BYU Configurable Computing Home || Internal Lab Pages

Brigham Young University Configurable Computing

How To Build JHDL

Overview

JHDL is built (compiled, etc.) using the Ant build system. Ant is a program comparable to make (except that it is infinitely better) that is useful for building large projects written in Java. Builds using Ant are generally easier, faster, and more fun than builds using make!

Besides showing how to use Ant to build JHDL, this page also contains some tips for how to program with the Ant build process in mind.

Introduction To Ant

Ant works a lot like make. When run, Ant will look for a file that contains a list of directives to tell it what to do. Instead of using makefiles, Ant uses buildfiles (or antfiles if you like that better). The default name for an Ant buildfile is build.xml.

XML? Ant buildfiles are written in XML. XML looks a lot like HTML. The advantage of XML is that it is extremely customizable. XML is also relatively easy to read and understand (as opposed to makefiles, which can be rather cryptic). When you look at an XML file meant to be used for Ant, you will see tags such as <project>, <target>, <property>, and so forth.

When Ant is run, it first parses the buildfile, then it goes to the default target tag (or the target specified by the user) and begins executing the commands specified by nested tags within that target. All of the commands themselves are represented by XML tags. Targets can also indicate that they depend on other targets (with a depends attribute). Ant ensures that all of the targets listed in the depends attribute are executed before the current target is run.

Some of the commands that Ant recognizes by default include java, javac, exec, copy, delete, and echo. Many of these tags can have special attributes and nested tags to further define the behavior of the command to be executed.

Setting Up Ant To Compile JHDL

This setup information is valid for the Linux machines in the FPGA lab. Similar principles may be applied to other platforms, but some significant alterations will be required. If you are building JHDL on a computer other than the lab's Linux machines, you will have to obtain and install Ant on your own. Ant version 1.5 or higher is required to build JHDL. You can get Ant from http://jakarta.apache.org/ant/. You will also need to get Apache's Xerces XML parser version 1.4.3 or 1.4.4

Ant is already installed on most of the Linux machines in the lab. It is located in /opt/ant. This installation of Ant on our machines includes the Xerces XML parser from Apache instead of the default XML parser from the Ant distribution. The easist way to include Xerces in your build is to place the xerces.jar file in the lib directory of your Ant installation. Of course our build process also needs to have Java available on the system. If you are attempting to build JHDL on some other system, you should try to set up the system as similarly as you can.

To use Ant to build JHDL you must do the following:

• Include /opt/ant/bin in your command path
• Set the JAVA_HOME environment variable to the location of Java on your machine
  On most of our machines, you should set it to /usr/local/java.
• Clean out your CLASSPATH so that it does not point to any JHDL trees (besides the one you will be building if you wish).
• Check out a fresh copy of the JHDL tree. You may also update a previously checked out copy of the tree.

If you wish to use jikes in the compilation process, make sure that your environment is set up for that as well. For this, you should have jikes in your command path and $JAVA_HOME/jre/lib/rt.jar should be in either your CLASSPATH or in the optional JIKESPATH environment variable.

Additionally, you may wish to set the ANT_OPTS variable. At times, the Ant build will fail because the JVM runs out of stack memory. (This most often occurs during the compile target, because the approximately 3000 source files to compile simply demand a lot of memory resources for Ant.) To allocate more memory to the Ant process, you may wish to do one of the following:

For the bash shell:

export ANT_OPTS="-Xms32M -Xmx128M"

For the csh/tcsh shell:

setenv ANT_OPTS "-Xms32M -Xmx128M"

The values 32 and 128 in these ANT_OPTS examples can be adjusted to whatever minimum and maximum stack size you prefer or that you find works on your system. You may also set any other options in the ANT_OPTS environment variable which you would like to have as JVM options when Ant is run.

Compiling JHDL Using Ant

Run the following commands to compile JHDL using Ant.

1. cd byucc/jhdl
2. ant

Windows-Specific Instructions

Installing the Tools

Ant version 1.5 or higher and Xerces version 1.4.3 or higher are required to build JHDL in Windows. After obtaining the latest versions of Ant and Xerces:

• Unzip Ant into a desired location on your hard drive (ex: C:\apps\)
• Rename the directory into which Ant was unzipped to ant (ex: C:\apps\ant)
• Go to the lib directory where Ant was unzipped (ex: C:\apps\ant\lib) and rename the xml parser called: xercessimpl.jar to: xercessimpl.old
• Install Xerces to a desired directory on your hard drive (ex: C:\jars)

Environment Variables

• Set ANT_HOME to the directory where Ant was installed (ex: C:\apps\ant)
• Ensure that $ANT_HOME/bin is included in PATH
• Ensure that xerces.jar (located in the directory where Xerces was installed - ex: C:\jars) is included in your CLASSPATH
• Set JAVA_HOME to directory where java is installed (ex: C:\jdk1.3.1)

Build JHDL

• Check out a copy of JHDL
• cd to: byucc/jhdl
• To speed up the build process, remove the directories:
  • CSRC
  • TERA
  • sc
  • synth
• Run: ant

What Ant Does On A JHDL Build

Several things occur when the JHDL tree is built using Ant. This section explains the events and actions that occur in that process.

The Short Description:

Ant copies lots of files to a directory called build and compiles the tree into a subdirectory called classes. You can point to that directory in your CLASSPATH to use JHDL

The Long Description:

The default target for the build of JHDL is the compile target. The compile target depends on several other targets which build a special build directory structure, copy only the desired source files, and prepares other files, such as XMAC-generated code and resource files. After those other targets complete their tasks, the compile target will compile the desired Java source code to the build/classes directory.

The following figure depicts the directory structure built up by the Ant build process. The labels for each directory indicate the purpose of the directory on the first line, the path on the second line, and the Ant property name on the third line. (The Ant properties can be customized as described in the Advanced Topics section.)

Advanced Topics Using Ant To Build JHDL

To see a list of the major targets that you can run, execute the command

        ant usage
    

To see a more comprehensive list of the targets that you can run, execute the command

        ant -projecthelp
    

The build process is quite customizable through the setting of properties. To set a property for Ant, just use the -Dproperty.name=value commandline option for Ant. For example, if you want to enable the output of deprecation notices during compilation you can run the following:

        ant -Ddeprection=on
    

You may also set properties in a build.properties file. This file may contain the name/value pairs for all of the properties which you would like to have set every time you run Ant to build JHDL. Put your build.properties file in the base directory.

The following is a list of properties you can set to customize your build.

There are many other properties that can be set. Most of them won't matter to even the most advanced users. (i.e. the default values are sufficient.) But to learn more you should look through the build.xml file.

Customizing the JHDL Build

This sections gives a brief overview of how to build JHDL with more precision over what portions of JHDL are built. The *.include and *.exclude properties described in the Advanced Topics section are used for this.

The motivation for custom JHDL builds come of the fact that nobody needs everything from the JHDL tree for their projects. For example, JHDL currently has several technology libraries (CSRC, XC4000, Virtex, etc.). But if you are only developing for the Xess XSV board, you would only care about the Virtex library. Likewise, if your work involves just putting together a lightweight applet demonstration of JHDL, then a lot of extras like the EDIF and XNF parsers, the design rule checker, etc. would not be useful for your work. Using custom includes and excludes, you can fine-tune the JHDL build process to suit your specific needs.

The following example illustrates how to customize the build so that only the Virtex II library is included in the build. Once all of the files have been set up as shown, simply running Ant from the byucc/jhdl directory will produce the desired effect.

Developing JHDL With The Build In Mind

It's important to keep in mind the build process while you are developing on the JHDL tree. Be sure to remember the following points.

• Put all test files and examples that demonstrate correct functionality in a subdirectory called "test". For more information about writing and executing tests, see How To Test JHDL
• If you have resources that compliment your javadoc comments, put them in a subdirectory called "docs". This is a good place for .gif files, additional .html files, and other resources to include with the javadoc information
• If a file is not needed anymore, run cvs remove on it. If the file is of historical interest, or can otherwise be useful as reference, move it to a subdirectory called "old".
• Most of the time you will not need to worry about modifying the build.xml file in byucc/jhdl. However, if your changes to JHDL require special treatment for the build, be sure to update the build.xml file.

The "test," "docs," and "old" directories are treated in special ways by Ant, as directed by the buildfile. The test directories are not compiled by default. The docs directories are not copied over to build/src (so they could contain .java files if necessary); instead they are copied to the same directory in which javadoc-generated HTML documentation is made. The old directories are simply ignored by Ant altogether.

Making Your Builds Faster

In most cases, Ant is much faster than using other build systems. However, under certain conditions, even Ant has its speed limitations. This section gives some tips for speeding up your Ant builds by making good file system choices and by using the start.path property or a script to to that for you.

NFS vs. Local Drives

Ant builds can be very I/O-bound, especially if performed on an NFS (network file system) drive. The JHDL build involves 2000-3000 files. For normal builds, the copy and compilation dependencies are verified for each of these files. If the file queries are done over your network connection, as with NFS, the process of merely checking timestamps on files quickly adds up to be a very big job.

To avoid this problem, you should consider performing your Ant builds on the local drive of the computer you are using. All of the Linux and HP-UX machines have a /tmp drive available for such purposes. Many Windows machines may have a C:\temp\ directory. Also, some of the Linux machines in the lab may have a /home drive available. You can copy your CVS-checked-out files to one of these drives (or just perform the CVS chekout directly on the drive). With the file I/O no longer using the network, you will find your Ant builds going much faster, as well as your overall system performance improving. (The network can be a real bottleneck.)

WARNING: The /tmp drives on Linux and HP-UX machines (as well as other variants of Unix) are often cleaned out by a process running in the background. That process looks for files that are older than some relative date and deletes them. If you choose to work on a /tmp drive and you care about the work you have done there, you must remember to copy your directory back to a network drive when you are done. Alternatively, you could use a program or script that recursively updates the timestamps of all of your files to trick the /tmp scrubber. Also, beware that the /tmp and /home drives are not backed up by the lab or the Spice sysops. (Only the network drives are backed up.)

Subdirectories

By default, the Ant build of JHDL will check the dependencies of every file to compile in the whole JHDL tree. With approximately 3000 files, this can take a long time (especially over NFS). However, most of the time, you may only care about compiling some subdirectory of the JHDL tree, depending on where you are developing and working. You can use the start.path property to tell Ant to focus on just that subdirectory and the child directories in it. For example, to compile just the byucc/jhdl/Logic/Modules directory, you could run the following:

       ant -Dstart.path=byucc/jhdl/Logic/Modules
    

A handy tool for doing this from any directory is a script that Aaron Stewart wrote. This script, called build (or whatever you rename it, if you choose), allows you to go to any part of the JHDL tree in which you may be working, edit some files, then just run the build script from that same directory. The script automatically determines where you are in the byucc/jhdl directory hierarchy and runs Ant with the start.path property set to that value.

To use this script, do the following:

1. Download the script into a location in your command path. (For example, if ~yourUserName/bin is in your command path, you could put it there.) You may have to run the rehash command from the command line to refresh your terminal's path setting or modify the initialization file (e.g. .cshrc) which sets your path if you are adding a new directory to your path.
2. Go into your local copy of the JHDL tree and into a subdirectory where you are working.
3. Run build.

BYU Configurable Computing Home || Internal Lab Pages

What's Make?

In the past, we have used makefiles to direct the build process of the JHDL tree. Makefiles are still useful in a few circumstances related to JHDL, so it's a good idea for you to know what they are.

The make program reads in makefiles that contain variable declarations and instructions to execute. Makefiles then behave much like a script to execute. What makes makefiles useful is that the make program is pretty good at determining dependencies.

Here's an example of a makefile:

       CC=/usr/bin/gcc
       CC_FLAGS=-g
       INPUT_CODE=my_source_code.c
       EXECUTABLE=my_program
       COMMAND_LINE=hello_world
       CHMOD=chmod
       CHMOD_VALUE=755
       RM=rm

       ${EXECUTABLE}: ${INPUT_CODE}
             ${CC} ${CC_FLAGS} -o ${EXECUTABLE} ${INPUT_CODE}
	     ${CHMOD} ${CHMOD_VALUE} ${EXECUTABLE}

       run: ${EXECUTABLE}
             ${EXECUTABLE} ${COMMAND_LINE}

       clean:
             ${RM} -f ${EXECUTABLE}

    

Before we attempt to explain this makefile, you should ask yourself, "Does this make any sense?!" The answer should be no. That is why we migrated over to Ant for a build system. The makefile system is inherently difficult to maintain, understand, and use. That doesn't mean that Ant is absolutely perfect, but it is a much better alternative to make for Java development.

Now the explanation... First, this makefile sets up several variables. (CC, CC_FLAGS, etc.) The variables are accessed by wrapping ${} around their names. This makefile contains three targets: ${EXECUTABLE}, run, and clean. The default target (because it is the first) is ${EXECUTABLE}. When the make program is run it will see if ${EXECUTABLE} is more uptodate than ${INPUT_CODE}. If not, it will run the commands for the ${EXECUTABLE} target. If the user runs make, specifying the run target, make will attempt to execute the run target. First it makes sure that ${EXECUTABLE} is uptodate. If it is not, then the ${EXECUTABLE} target is run first. Then it runs the ${EXECUTABLE} with ${COMMAND_LINE} as an argument. Notice that the clean target doesn't have any dependencies. It is run no matter what when the user specifies that target.

Note that the makefiles used in JHDL are WAY more complicated than this example. By the way, don't just cut and paste this text to test it out as a makefile. The syntax for makefiles is very picky. It is based on the proper use of tabs for indentation. This example actually does not use correct syntax because the indentation is done with spaces.