FG Version 1.4
(released July 2006)

ABCDEFG (Asynchronous Buffered Computation Design and Engineering Framework Generator), which we shorten to FG and pronounce "effigy," is a programming environment for pipeline-structured programs. Such programs pass buffers among asynchronous stages, so that while one stage accesses high-latency data, other stages can make progress. The FG project is supported by DARPA, IBM, and the National Science Foundation. You can download FG version 1.4 from this page.

About FG

FG is a programming environment for asynchronous programs that run on clusters and fit into a pipeline framework. It enables the programmer to write a series of synchronous functions and represents them as stages of an asynchronous pipeline. FG mitigates the high latency inherent in interprocessor communication and accessing the outer levels of the memory hierarchy. It overlaps separate pipeline stages that perform communication, computation, and I/O by running the stages asynchronously. Each stage maps to a thread. Buffers, whose sizes correspond to block sizes in the memory hierarchy, traverse the pipeline. FG makes such programs easier to write, smaller, and faster.

New features in FG version 1.4 include disjoint and intersecting pipelines. FG allows you to run more than one pipeline concurrently on one processor. This flexibility enables cluster computations in which a given processor may send more or less data than it receives. Please see pages 32–36 in the FG tutorial for details.

May 2011 update: FG version 2 is being developed. The API for FG 2 will be completely different from the FG version 1.X API, and we plan to have a graphical front end.

People FG Alumni FG Papers Download FG
Here is everything you need for FG version 1.4. We have a tutorial to get you started, and you can follow the instructions below to install FG.

If you download and use FG, we invite you to send email to fg@cs.dartmouth.edu to let us know that you're using it.

You will need the standard pthread interface library and g++ to install and run FG. Download fg1.4.tgz. Then run the following command from your terminal to unzip the tarball:

tar -xvzf fg1.4.tgz
This command will unzip the FG files into a directory called fg1.4. Now you can adjust debugging options and make the FG library. There are three debugging options that you can set: FGLOG for the log file, FGDEBUG for debugging options, and FGTIMER for timing options. If you choose to leave all three options unset, you can make the FG library with the following command:
make libfg.a
You can set the debugging options explicitly in the Makefile, or you can set them on the command line. The FGLOG variable may be set to -DLOGA or -DLOGW, which you would set on the command line with one of the following:
make FGLOG=-DLOGA libfg.a
make FGLOG=-DLOGW libfg.a
If you set either option for FGLOG, FG prints information about your program, including warnings and errors, to a log file. With the first argument, FG appends all the information to the log file, and with the second it overwrites the existing log file.

The FGDEBUG variable may be set to -DVERBOSE:

make FGDEBUG=-DVERBOSE libfg.a
With this setting, FG continually prints information about the program as it progresses, along with any errors or warnings. It also prints timestamps along with each message. If FGLOG is set, FG prints this information to the log file. Otherwise, it prints to standard output.

Finally, you may set the FGTIMER variable to -DTIMERPIPELINE or -DTIMERALL, which you would set on the command line with one of the following:

make FGTIMER=-DTIMERPIPELINE libfg.a
make FGTIMER=-DTIMERALL libfg.a
The first option times the the pipeline when the first stage begins executing until the final stage finishes executing. The second option times the pipeline from the very first call to FG until the pipeline is dismantled. If FGLOG is set, FG prints the timings to the log file. Otherwise, it prints the timings to standard output.

You may combine any of the three debugging options in any way. If you would rather not use the command line, you can change the settings directly in the Makefile, which is in the fg1.4 directory.

By default, the FG log file is named FG.log, and it is stored in the directory where you run your program. You may change both the name and the directory with environment variables, however. FG uses the environment variable FGLOGNAME for the filename and FGLOGDIR for the directory.

Any files you create that need to use FG classes and methods must include the file fg1.4/FG.h. When compiling a program written with FG, you must link in the FG library file. If your program is called fgtest.cc, and your fg1.4 directory is located at /net/grad/laney/fg1.4/, you would build your FG program as follows:

g++ fgtest.cc -o fgtest -L/net/grad/laney/fg1.4 -lfg -lpthread
Then you can run the executable file fgtest as usual.

Because FG creates a multi-threaded program (even if you specify only one thread), you should ensure that all the software you link with FG works in a multi-threaded environment. Be particularly careful if you are using MPI for interprocessor communication, for not all implementations of MPI are thread-safe.

Because they are multi-threaded, FG programs also can create performance problems associated with disk I/O. On many systems, when you run multi-threaded code, C stdio locks a file every time a thread reads or writes it and unlocks the file when the thread finishes reading or writing. The use of locks can slow down performance considerably, especially in a program where disk I/O is a time sink. It can be useful to instead use the unlocked versions of C stdio functions. We have found that using fread_unlocked() for reading and fwrite_unlocked() for writing can be significantly faster than using the locked counterparts. Check the man page entry for unlocked_stdio to determine the support in your system for unlocked I/O.

Any questions or comments, please email fg@cs.dartmouth.edu.

Sample Code

The following are sample FG programs from the FG tutorial. You can download these files and build them with the FG library as described above.