| PAGE SUMMARY |
|---|
| SEE ALSO |
This page explains how to use Ant to run the self tests of JHDL. It also explains how to create new tests to add to the JHDL tests.
Running JHDL tests with Ant is very easy. Simply invoke Ant indicating that the test target should be run:
ant test
This action will cause Ant to scan the build/classes
directory for classes which are either recognized to be tests or
which are explicitly listed in the test target
of the build.xml file. Each of those tests is in
turn executed in a separate Java Virtual Machine. Output from
the test will be piped to the display through Ant.
When Ant is run with the test target invoked, it first compiles the ExecTest class in byucc/jhdl/util/ant. This class is a custom Ant task (see Ant documentation) which serves to execute tests from within Ant. After the ExecTest class is compiled, it is registered as an Ant task under the name exectest.
The exectest task requires a nested fileset
element to pass it a list of tests to execute. The fileset
element used in our build.xml file looks for classes that it
recognizes as tests (their names match a certain pattern or
patterns). Then the exectest task will simply invoke
the main method of each test.
Each test is executed in it's own Java virtual machine. This helps to ensure that a test will not be affected by anything performed by a previous test (e.g. changing the value of a static member of a class which both tests use).
Some tests require some special treatment. The test target in the build.xml file also performs the extra operations to support those tests.
Note that the test target in byucc/jhdl/build.xml depends on the compile target of the same buildfile. That ensures that the normal build process as described in How To Build JHDL
Often, especially when developing just a small portion of JHDL, you will not want to wait for all of the tests to execute, a process that can be very lengthy. There a few ways to restrict which tests will be executed.
The easiest way to limit the tests which get executed is to set
the start.path property when running Ant. For
example, the following will run only the tests found in
byucc/jhdhl/base/test and all of its subdirectories:
ant -Dstart.path=byucc/jhdl/base/test test
To have more precise control over the tests which will be executed, you can set the test.pattern, test.includes, and test.excludes properties.
The default for the test.pattern property is
"**/TEST*.class". This is normally
concatenated on the end of the start.path
property to form the test.pattern.path
property. You can modify the test.pattern
property to be more specific to your needs.
You might also find it useful to simply set test.pattern to a bogus pattern, and use the test.includes and test.excludes properties. These properties should be set to the name of a file which will contain patterns or explicit paths, separtated by line terminators, which will determine which tests will be executed and which will not.
The following will execute only the tb_TestObservable class:
ant -Dtest.pattern=/base/test/Observable/tb_TestObservable.class test
The following will execute all of the tests that match the
pattern byucc/jhdl/Fsm/test/TEST*.class except for
the test byucc/jhdl/Fsm/test/TESTParity2.class.
tests file:
byucc/jhdl/Fsm/test/TEST*.class
notest file:
**/TESTParity2.class
ant -Dtest.pattern=bogus -Dtest.includes=tests -Dtest.excludes=notest test
For now, we do not have a set standard for the implementation of tests of JHDL. Therefore, for the time being, you should simply make tests that make the most sense to validate the performance of your code.
However, the actual execution of tests is currently made easiest if you follow three simple patterns.
TEST*.java
When the exectest task is run, by default it
looks for all classes that have a name matching the pattern
TEST*. If you do not name your test such that it
begins with TEST, then you will need to explicitly
add your test's path and name pattern to the fileset
element nested in the exectest task in the
byucc/jhdl/build.xml antfile. That approach is not
recommended because it requires extra maintenance in the JHDL
tree, requiring upkeep not only in the directory of your tests,
but also in the buildfile itself.
Make sure your tests do not need any arguments. The
exectest task will attempt to find a method
matching the prototype public static void main( String[] ).
It will then try to invoke that method, passing to it an array of
String of length zero. If your test really needs to have
arguments, you should consider making your TEST* class be a
wrapper around the actual test. The wrapper can then invoke the
main method of the wrapped class, passing to it arguments as
needed. These arguments could even be supplied by an external
file resource, which Ant will copy into the directory of the
test. If you want to execute a single test several times, with
different arguments each time, consider creating a wrapper class
for each execution. Multiple wrapper classes is preferable over
a single wrapper that invokes a test multiple times because it
ensures that each execution of the test will be performed in a
separate Java virtual machine. This is often crucial because
static variables modified by one test could affect subsequent
tests in ways that are very difficult to debug.
Two other things should be considered when creating tests:
You should assign your tests to packages because of the way Ant compiles Java code. When Ant compiles a .java file, it compiles the class, then it places the class in the destination directory within the subdirectory(ies) which match its package path. A class that is not assigned to a package will therefore be assigned to the default package (.) and its .class file will be placed at the head of the destination directory when it is compiled. This could cause problems, especially if the class needs external resources (i.e. files); Ant will, by default, copy those resources to the exact path in which they were found in the source tree. So not assigning a package to a class could separate the class from the resources that it expects to find. Additionally, if everyone did not assign a package to their classes, there would most likely be naming conflicts at compile time as Ant tries to put all of those classes at the head of the destination directory.
We strongly recommend that you place your tests in a subdirectory called "test". (For example, if you are writing tests for the package byucc.jhdl.Xilinx.VirtexXV, you should put your tests in the directory and package corresponding to byucc.jhdl.Xilinx.VirtexXV.test.) Separating test code from application code has two advantages. First, it helps to create deliverable JHDL packages; when creating a JAR file of JHDL, we can just throw away all classes and resources in test subdirectories because we know they are not necessary for actual execution of the JHDL application code. Second, to have test code reside outside of the package being tested reduces the privileges of the tests. Since the tests don't have package-scope privileges to the classes in the tested package, you can better guarantee that the tested package can perform as expected without having to have any special internal powers and knowledge. This is generally the rule to follow, but there may, as always, be exceptions.