James Amundson
Computing Division
Fermi National Accelerator Laboratory
Manual Version 1.02
December 14, 1999
This manual describes SoftRelTools version 2. It is intended to augment the documentation in ``A UNIX Based Software Management System,'' edited by Robert Harris, Computing Division Note: GU0013, which provides a more general introduction to SoftRelTools. It is available at
The primary purpose of SoftRelTools version 2 is to provide a backward-compatible replacement for the original SoftRelTools written by Bob Jacobsen. SoftRelTools is designed to be easier to use and to maintain than the original SoftRelTools. Ease of use and maintainability are enhanced by increased functionality and modular structure. Maintainability is further enhanced by the seperation of the basic tool from project and site specific settings through an inheritance heierarchy.
The modules and their relationships are described by the following figure:
The inheritance heierarchy from base to project and site-specific settings is orthogonal to the separation of the tool into modules. In the figure below, each pane represents the collection of modules shown in the figure above.
The SoftRelTools base is largely self-contained. The project and site specific behavior is implemented through optional additional packages, SRT_$PROJECT and SRT_SITE.
All of the publicly available SoftRelTools files live in a distribution directory. For the purposes of the manual we we refer to this directory as $SRT_DIST. In order to use SoftRelTools, users must have sourced the file $SRT_DIST/srt/srt.csh or $SRT_DIST/srt/srt.sh, depending on the users' shell. Of course, this assumes that a SoftRelTools distribution in $SRT_DIST has been installed. (See SoftRelTools for librarians.) After srt.(c)sh has been sourced, the command ``srt_setup'' will set up SoftRelTools for use. srt_setup is described in the srt_setup man page and in the section on the SoftRelTools environment.
The following example shows how a user can work with a package ``Hello'' that is part of a release ``development''. It assumes that SRT has been set up.
cd myrelease
read user srtrc
Creating a test release "myrelease" in the directory
/home/amundson/work
Linking tmp to /tmp/myrelease/tmp
Linking bin to /tmp/myrelease/bin
Linking lib to /tmp/myrelease/lib
> cd myrelease/
> addpkg Hello
Release development uses Hello version HEAD, will check that out
Adding package "Hello" to ".".
cvs checkout: Updating Hello
U Hello/GNUmakefile
U Hello/Hello.cc
U Hello/Hello.h
U Hello/HelloExample.cc
> srt_setup -a
> gmake
<**include**>
<**include**> Hello
<**lib**>
<**lib**> Hello
<**compiling**> Hello.cc
<**building library**> libHello
<**bin**>
<**bin**> Hello
<**compiling**> HelloExample.cc
<**building**> HelloExample
>
To learn more about the commands newrel, addpkg, etc., read the section on the code management system. To learn more about building your own packages, read the section on the build system.
(The following information is duplicated in the file README.install in the SoftRelTools/install directory.)
This version of SoftRelTools needs two things to get started:
There are several new files that can hold settings that apply to all users.
SoftRelTools will look for the package SRT_$SRT_PROJECT (i.e., SRT_D0, SRT_CDF, etc.). This package can be used to augment and/or replace behavior of the central SoftRelTools. The mechanism for this is described in the section on the inheritance hierarchy.
The package SRT_SITE (note the fixed name) is similar to the project-specific package. It is only intended for situations where two different machines for the same project need to behave differently. Its behavior supersedes both the default behavior and the project-specific behavior.
The version of SoftRelTools at Fermilab is modified frequently - on the order of once per day. These changes fall into two categories: 1) Bug fixes. 2) Local changes in settings, etc. The goal of the inheritance hierarchy in SoftRelTools is to allow projects to be able to make both kinds of changes quickly, without having to worry about affecting other projects. Changes of the first type can be incorporated into the base package in a controlled manner. Changes of the second type can stay with the projects, where they belong.
As the term implies, the inheritance hierarchy is based on the concepts of object-oriented design. Think of the site-specific parts of SoftRelTools inheriting from the project-specific parts, which inherit from the base. However, SoftRelTools is implemented in primarily in GNU Make and Bourne Shell - neither of which are very well-suited to object-oriented programming. ``Inheritance'' in SoftRelTools should really be considered an analogy. The analogy will break down if pushed too far.
All of the modules in SoftRelTools can be modified first at the project level then the site level. The site has the final word. There are two mechanisms available:
All makefile fragments in SoftRelTools/include look for the a corresponding file in SRT_$SRT_PROJECT/special. Files in subdirectories of SoftRelTools/include go into the corresponding subdirectories of SRT_$SRT_PROJECT/special. A few special files can be specialized both at the beginning and the end:
All of the scripts in SoftRelTools consist of a set of subroutines, including one called ``main''. If specialization files are found in the ``scripts'' subdirectory of the project-specific package, they are sourced before any of the subroutines are called. The specialization files can contain replacements for old subroutines and/or new subroutines.
echo ``bar''
echo "Now executing top secret extra actions"
}
main () {
script_defaults
process_args $*
actions
ods_actions
}
SoftRelTools includes two commands, srt_setup and srt_environment, for examining and manipulating the user environmental variables. The former is actually an alias that calls the latter with certain options. The default behavior for srt_environment is to print the current settings. Sample output is below.
Variables for backward compatibility:
BFARCH = Linux2-KCC_3_3
BFDIST = /home/amundson/work/dist
BFCURRENT = development
Automatic and derived variables:
PATH = /home/amundson/work/myrelease/bin/Linux2-KCC_3_3:/home/amundson/work/dist
/releases/development/bin/Linux2-KCC_3_3:/fnal/ups/prd/kai/v3_3f_1/Linux+2/KCC_B
ASE/bin:/home/amundson/work/dist/releases/boot/bin/generic:/opt/kde/bin:/fnal/up
s/prd/ups/v4_3/Linux+2/bin:/opt/kde/bin:/home/amundson/work/dist/releases/boot/b
in/generic:/home/amundson/bin:/usr/sbin:/bin:/usr/bin:/etc:/usr/etc:/usr/bin/X11
:/usr/local/bin:.:/home/t1/amundson/bin:/home/t1/amundson/bin
LD_LIBRARY_PATH = /home/amundson/work/myrelease/lib/Linux2-KCC_3_3:/home/amundso
n/work/dist/releases/development/lib/Linux2-KCC_3_3:
SRT_PRIVATE_CONTEXT = /home/amundson/work/myrelease
SRT_PUBLIC_CONTEXT = /home/amundson/work/dist/releases/development
MAKEFILES = "SoftRelTools/preamble.mk"
MAKEFLAGS = "-r -I/home/amundson/work/myrelease/SRT_ODS -I/home/amundson/work/di
st/releases/development/SRT_ODS -I/home/amundson/work/myrelease/include -I/home/
amundson/work/dist/releases/development/include"
CVSROOT = /home/amundson/repository
SRT_SUBDIR = Linux2-KCC_3_3
SRT_PROJECT = ODS
SRT_ARCH = Linux2
SRT_ENV_SET = yes
User settable variables:
SRT_LOCAL = /home/amundson/work/myrelease
SRT_DIST = /home/amundson/work/dist
SRT_BASE_RELEASE = development
SRT_CXX = KCC_3_3
SRT_QUAL = default
The first time srt_setup is called, variables are set to their default settings, SRT_CXX=$DEFAULT_SRT_CXX, etc. The defaults can be restored later by ``srt_setup -d''.
Users can alter the values of variables by putting one or more assignments on the command line
srt_setup SRT_QUAL=maxopt SRT_BASE_RELEASE=test
srt_setup and srt_environment source the file $SRT_DIST/srt/srt_envrc. This allows the librarian to set defaults for an entire project. For example, the srt_envrc might contain the lines
DEFAULT_SRT_CXX=KCC_3_3
DEFAULT_SRT_BASE_RELEASE=development
Since the file is sourced by the shell script, it can contain any shell commands. Only the resulting value of the variables matter.
srt_setup and srt_environment also source the file $HOME/.srt_envrc. (Note the presence of a leading dot in the user file, but not in the system file.) This allows the user to set his/her own defaults. Again, the file is sourced by the shell script, so it can contain any shell commands.
The scripts for the code management system are described by the man pages. Additionally, every script will describe its own actions and options when invoked with ``-help'' or an argument it does not understand. The ``newrel'' command looks for system and user preferences.
Scripts with the prefix ``srt_int'' are used internally by SoftRelTools. They will not generally be useful to users. Note that the srt_int_querypkg script is technically part of the build system, not the code management system. That means that changes to the build system can affect srt_int_querypkg. The code management scripts are designed to be independent of the build system.
newrel sources the file $SRT_DIST/srt/srtrc for directory creation preferences. If the variable ``extra_dirs'' is defined, it is merged with the list of directories to be created. It should be in the form of a space-separated list of directory names. extra_dirs has two purposes:
The variable ``release'' (the name of the new release) is guaranteed to be available when srtrc is sourced.
newrel also sources the file $HOME/.srtrc (note the leading dot) for directory creation preferences. See above for details. The following example .srtrc file is useful:
"$extra_dirs tmp>/tmp/$release/tmp bin>/tmp/$release/bin lib>/tmp/$release/lib "
Introduction
The build system in SoftRelTools is based on GNU Make. Make is a very flexible tool with a steep learning curve. SoftRelTools allows users to build and install a variety of objects without learning the intricacies of Make. At the same time, the power of Make is available for users who need to go beyond simple functionality. SoftRelTools provides a method to build libraries, binaries, standalone object files, man pages and documentation files. It also provides a method for packages to use multiple subdirectories and subpackages. Packages make their header files available to other packages through a configurable directory.
In order to use the SoftRelTools build system, the user must create a GNUmakefile. The GNUmakefile must at least include the line
All of the header files that are to be made available to other packages need to be placed in one subdirectory. The subdirectory may be the main package directory. SoftRelTools will use the subdirectory indicated by the variable PACKAGE_INCLUDE. If PACKAGE_INCLUDE is not found, SoftRelTools will look for a subdirectory with the same name as the package directory. If that fails, it will then look for subdirectories named include, then src, in that order. As a last resort, it will use the package directory itself as the header directory.
Exported headers can be used as follows
Packages can use an arbitrary hierarchy of subdirectories. SoftRelTools will attempt to launch builds in the subdirectories listed in the variable SUBDIRS. SoftRelTools will only launch make in directories containing a GNUmakefile. All subdirectories put their intermediate build products (object files, dependency files, etc.) in the same temporary directory.
Subpackages are very similar to subdirectories. A subpackage is distinguished by setting the variable SUBPACKAGE to the subpackage name. Each subdirectory of the subpackage must define SUBPACKAGE. Subpackages have their own temporary areas. Packages can have a mixture of subdirectories and subpackages, but this is not necessarily encouraged.
If the variable ``SORT_SUBDIRS'' is set, subdirectories are built in alphabetical order. Otherwise they are built in the given order.
SoftRelTools can build static and shared libraries. To build the library libFoo, include the line
LIB=libFoo.a
libFoo will be static, shared or both, if the variable LIB_TYPE is ``static'', ``shared'', or ``both'', respectively. (``Both'' means generating two libraries, e.g., libFoo.a and libFoo.so.) The default is to create static libraries. The suffix on the filename is irrelevant - SoftRelTools will give the library the appropriate suffix depending on library type and platform. Static and shared libraries can also be built by setting the variables SHAREDLIB and STATICLIB, respectively. The value of LIB_TYPE is ignored for SHAREDLIB and STATICLIB.
The contents of the created libraries are defined by the following variables:
As described above, libraries specified by the LIB variable will be built static, shared, or both depending on the value of the LIB_TYPE variable. The library rules in the old SoftRelTools included all the object files found in the library temporary directory into the library. It is preferable to link exactly those objects requested, however many packages rely on the old behavior. SoftRelTools will use the old behavior (all objects in the directory) if the variable CATCHALL_LIBS is set. Otherwise, it will only link the requested objects.
SoftRelTools will attempt to build all files listed in the BIN variable during the bin stage. SoftRelTools will attempt to build all files listed in the TBIN variable during the tbin stage. The tbin stage is not normally built by ``gmake all''; it must be invoked explicitly.
SoftRelTools provides rules for generating three different kinds of ``binary'' files.
Files listed in the SCRIPTS variable are assumed to be scripts. They are copied into the binary destination directory and made executable. Note that normal usage requires listing the script in the BIN variable (to tell SoftRelTools that it needs to be built during the bin stage) and the SCRIPT variable (to tell SoftRelTools how to build it.) A single directory can build an arbitrary number of scripts.
Each file listed in the SIMPLEBINS variable is built into a single binary of the same name. It looks for a single source file with the given name and a suffix .cc, .cpp, .cxx, .c or .f. They will each be linked with BINLIBS, described below. A single directory can build and arbitrary number of simple binaries. Note, however, that all simple binaries in a package are built in the same temporary directory. As with the other binary types, normal usage requires listing simple binaries in both the BIN variable and the SIMPLEBINS variable.
A subdirectory may specify in COMPLEXBIN one complex binary to be built. The following contents may be specified:
All binaries in a directory will be linked with the contents of BINLIBS. (Note that specifying extra libraries at link time is not usually a problem.) Local libraries should be listed as dependencies of the files that use them. Local libraries mean libraries built by SoftRelTools. External libraries, however, should not normally be listed as dependencies. A change in external libraries should be followed by a clean build. SoftRelTools places the contents of BINLIBS on the binary dependency line, but with the contents of NODEP_LIBS filtered out. Having libraries listed in NODEP_LIBS that are not in BINLIBS is perfectly acceptable. For backward compatability, if BINLIBS is empty, LOADLIBES is used instead.
All of the rules for for binaries apply to test binaries, also.
If OLD_BIN_RULES is defined, SoftRelTools includes the binary rules from the original SoftRelTools.
Standalone objects are built directly in the library directory.
Files listed in the MANPAGES variable will be installed during the bin stage. foo.1 will be installed in man/man1, foo.2 will be installed in man/man2 directory, etc.
Files listed in the DOCS variable will be installed into the doc directory during the bin stage.
Files listed in the INC variable will be built during the include stage. SoftRelTools does not provide the rules for building the files.
Files listed in the CODEGENFILES variable will be built during the codegen files. SoftRelTools does not provide rules for code generation by default, but rules can be included by including the files SoftRelTools/idl.mk, SoftRelTools/java.mk and SoftRelTools/yacc.mk.
SoftRelTools version 2 defines
Qualifiers are named sets of flags. The name is placed in the SRT_QUAL environment variable. The value of SRT_QUAL passed to srt_setup determines which set of qualifiers are being used for the other packages. The local package can be built with any value of SRT_QUAL.
Building the hello package.
> srt_setup -a
> gmake
<**compiling**> Hello.cc
<**building library**> libHello
<**building**> HelloExample
> gmake clean
> gmake
<**compiling**> Hello.cc
<**building library**> libHello
<**building**> HelloExample
> gmake clean
> gmake SRT_QUAL=maxopt
<**compiling**> Hello.cc
<**building library**> libHello
<**building**> HelloExample
SoftRelTools provides the following debugging aids:
Normally, SoftRelTools executes its commands silently. If the variable VERBOSE is defined SoftRelTools will print each command before it is executed. The value of VERBOSE is irrelevant; it only has to be non-null.
Typing gmake echo_FOO will echo the value of the variable FOO at target execution time. It also prints the value of the make ``origin'' command for the variable FOO. Obviously, it works for any variable.
sortecho_FOO is similar to echo_FOO, but it sorts the contents of FOO and prints them in a single column suitable for input to diff. It is useful for comparing the values of complicated flags.
This is really a function of make. The debug flag causes make to generate extremely verbose output. However, SoftRelTools works hard to minimize the extraneous content of the output.
Several example packages are available:
The external package system breaks down into two parts: the interface to the standard compilers and linkers and the interface to external libraries, etc.
The external interface to arch_spec.mk is essentially unchanged: the build system includes arch_spec.mk, which sets a variables for the compilers, flags, etc. The most important change is that SoftRelTools relies on the SRT_ variables as input instead of the BF variables. The SoftRelTools variables have been maintained.
The original SoftRelTools contained all settings for all architecture/compiler combinations in one file. This has been substantially rearranged in SoftRelTools version 2. arch_spec.mk has been split into three: the main arch_spec.mk, the C++ compiler files and the platform files. The compiler and platform files live in the compiler and platform subdirectories of the include directory of SoftRelTools. Since some settings depend on both compiler and platform, a choice had to be made. The convention is that compiler-specific information lives in the compiler file, even if it is platform-specific. There are still if statements in the compiler files, but they are very simple.
SoftRelTools now defines macros for compilation. All permutations of (C++, c, Fortran, Preprocessed Fortran) +
(pic, non-pic)+ (with depends, without depends) are defined. Additionally, both on-the-fly and separate dependency generation are included. For details, see arch_spec.mk.
Qualifiers are named sets of flags. The name is placed in the SRT_QUAL environment variable. By default, SoftRelTools defines two sets:
SoftRelTools attempts to create a standard for arch_spec_*.mk files where none existed before. arch_spec_*.mk files should:
A few packages do not follow these guidelines for legacy reasons. Each one has a comment to that effect.
SoftRelTools is available for anonymous cvs access at
The SoftRelTools boot distribution is available at
This document was generated using the LaTeX2HTML translator Version 98.2 beta6 (August 14th, 1998)
Copyright © 1993, 1994, 1995, 1996,
Nikos Drakos,
Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998,
Ross Moore,
Mathematics Department, Macquarie University, Sydney.
The command line arguments were:
latex2html -split 0 SoftRelTools-Manual.tex
The translation was initiated by James Amundson on 1999-12-14