|
Size: 33867
Comment:
|
Size: 33865
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 269: | Line 269: |
=== Using CVS === When you want to commit your changes, use the cvs commit command. However, you must first check to see if the file(s) in the archive have changed since the last time you checked them out. To do so, run the cvs update command on the files you have changed. The -P option will purge (remove) files that have been removed from the CVS. {{{ cvs update -P }}} If any have changed, you will see the letter U followed by the file name. {{{ U myFile.c }}} This means somebody else committed a newer version of the file, and your copy was just updated. If you and somebody else have made changes on the same file that conflict each other, you will see the letter C along with a message about the conflicts. {{{ C myFile.c }}} If the message states the conflicts have been resolved, you don't have to worry: cvs has merged the differences properly. If it says it couldn't merge the differences, you need to open the file in question, search for the string >>>, and merge the differences yourself. Once the up to date check has been performed, you can commit your changes. If you are in a directory and want to commit all the files in that directory that you have changed, just use: {{{ cvs commit }}} This will commit changes in the current directory and any subdirectories. If you want to commit only certain files, use: {{{ cvs commit <filename> }}} Or you can only commit the code in the current directory with the -l switch: {{{ cvs commit -l }}} Each time, an editor will open and ask you to enter a log message. To bypass this, use the -m option and enter your log message on the command line: {{{ cvs commit -m "some descriptive changelog comment" <filename> }}} A synonym for commit is ci, so you can just: {{{ cvs ci -m "fixed stuff" <filename> }}} Periodically, you should update your working copy to be in sync with the archive. CVS will look at all your files and see if the ones in the archive are different. If so, it will update your copy. CVS will not delete local modifications you have made. To do this update: {{{ cvs update -d -P }}} The -d switch adds any new directories and is not strictly necessary, but is a good habit, because if you don't use it you will update existing files but will now add new directories. The -P option purges directories of files that have been removed in the past. Similarly, you can update an entire directory or a single file. To commit all the code in the current directory (code that has not been changed will not be commited): {{{ cvs ci -m "fixed many things" }}} To restore an edited version to the version in the archive, either delete the file and cvs update, and the archive version will be copied to the local work area, or use the -C option to cvs update to replace your copy with one from the archive. To add a new file to the archive, use the add command and then commit the file: {{{ cvs add <filename> cvs commit -m "added the file" <filename> }}} To remove a file, remove in cvs and locally (with the -f flag), and commit: {{{ cvs remove -f <filename> cvs commit -m "removed the file" <filename> }}} That covers most of the CVS commands you'll have to to use. Additional stuff: To tag a group of code with a text label: {{{ cvs tag alpha-1 }}} To retrieve a tagged group of code: {{{ cvs update -r alpha-1 }}} There are more useful commands, but these are the most commonly used ones. See http://www.cvshome.org for more info. For more info on CVS, check out www.cvshome.org or the CVS man pages. Summary: check in a file {{{ cvs commit }}} get newest versions. merges newer versions than the ones you had with your working versions. {{{ cvs update -A }}} tag working files: {{{ cvs tag -c tag-name }}} if any working copies are edited, will report an error. check in first. checkout a tag: {{{ cvs checkout -r tag-name archive }}} split off a branch based on current revisions in working copy: {{{ cvs tag -b branch-name }}} note: doesn't make working copy the branch. split off a branch: {{{ cvs rtag -b -r tag-name branch-name archive }}} check out a branch: {{{ cvs checkout -r branch-name archive }}} switch working copy to a branch: {{{ cvs update -r branch-name archive }}} -or- {{{ BCD archive; cvs update -r branch-name }}} to find out what branch the working copy is on: {{{ cvs status }}} look for sticky tag field. join the changes in a branch to current working copy: {{{ cvs update -j branch-to-join file }}} best to tag merged branches after every merge. joing the changes in a revision to current working copy: {{{ cvs update -j rev file }}} i.e. two people check out 1.3, someone checks it in 1.4, you try to check it in and get a conflict, can join in the changes with cvs update -j 1.4 file.c add a new file: {{{ cvs add file; cvs commit file }}} remove a file (but keep in older revisions): {{{ rm file; cvs remove file; cvs commit file }}} rename a file: {{{ mv old new; cvs remove old; cvs add new; cvs commit old new }}} To receive an email notification about any file that has been commited by somebody else: {{{ cd dev cvs watch add }}} === Troubleshooting autoconf issues === [wiki:Self:AutoconfTroubleshooting Here's a list of common problems and solutions for autotools problems.] |
|
| Line 450: | Line 616: |
=== Using CVS === When you want to commit your changes, use the cvs commit command. However, you must first check to see if the file(s) in the archive have changed since the last time you checked them out. To do so, run the cvs update command on the files you have changed. The -P option will purge (remove) files that have been removed from the CVS. {{{ cvs update -P }}} If any have changed, you will see the letter U followed by the file name. {{{ U myFile.c }}} This means somebody else committed a newer version of the file, and your copy was just updated. If you and somebody else have made changes on the same file that conflict each other, you will see the letter C along with a message about the conflicts. {{{ C myFile.c }}} If the message states the conflicts have been resolved, you don't have to worry: cvs has merged the differences properly. If it says it couldn't merge the differences, you need to open the file in question, search for the string >>>, and merge the differences yourself. Once the up to date check has been performed, you can commit your changes. If you are in a directory and want to commit all the files in that directory that you have changed, just use: {{{ cvs commit }}} This will commit changes in the current directory and any subdirectories. If you want to commit only certain files, use: {{{ cvs commit <filename> }}} Or you can only commit the code in the current directory with the -l switch: {{{ cvs commit -l }}} Each time, an editor will open and ask you to enter a log message. To bypass this, use the -m option and enter your log message on the command line: {{{ cvs commit -m "some descriptive changelog comment" <filename> }}} A synonym for commit is ci, so you can just: {{{ cvs ci -m "fixed stuff" <filename> }}} Periodically, you should update your working copy to be in sync with the archive. CVS will look at all your files and see if the ones in the archive are different. If so, it will update your copy. CVS will not delete local modifications you have made. To do this update: {{{ cvs update -d -P }}} The -d switch adds any new directories and is not strictly necessary, but is a good habit, because if you don't use it you will update existing files but will now add new directories. The -P option purges directories of files that have been removed in the past. Similarly, you can update an entire directory or a single file. To commit all the code in the current directory (code that has not been changed will not be commited): {{{ cvs ci -m "fixed many things" }}} To restore an edited version to the version in the archive, either delete the file and cvs update, and the archive version will be copied to the local work area, or use the -C option to cvs update to replace your copy with one from the archive. To add a new file to the archive, use the add command and then commit the file: {{{ cvs add <filename> cvs commit -m "added the file" <filename> }}} To remove a file, remove in cvs and locally (with the -f flag), and commit: {{{ cvs remove -f <filename> cvs commit -m "removed the file" <filename> }}} That covers most of the CVS commands you'll have to to use. Additional stuff: To tag a group of code with a text label: {{{ cvs tag alpha-1 }}} To retrieve a tagged group of code: {{{ cvs update -r alpha-1 }}} There are more useful commands, but these are the most commonly used ones. See http://www.cvshome.org for more info. For more info on CVS, check out www.cvshome.org or the CVS man pages. Summary: check in a file {{{ cvs commit }}} get newest versions. merges newer versions than the ones you had with your working versions. {{{ cvs update -A }}} tag working files: {{{ cvs tag -c tag-name }}} if any working copies are edited, will report an error. check in first. checkout a tag: {{{ cvs checkout -r tag-name archive }}} split off a branch based on current revisions in working copy: {{{ cvs tag -b branch-name }}} note: doesn't make working copy the branch. split off a branch: {{{ cvs rtag -b -r tag-name branch-name archive }}} check out a branch: {{{ cvs checkout -r branch-name archive }}} switch working copy to a branch: {{{ cvs update -r branch-name archive }}} -or- {{{ BCD archive; cvs update -r branch-name }}} to find out what branch the working copy is on: {{{ cvs status }}} look for sticky tag field. join the changes in a branch to current working copy: {{{ cvs update -j branch-to-join file }}} best to tag merged branches after every merge. joing the changes in a revision to current working copy: {{{ cvs update -j rev file }}} i.e. two people check out 1.3, someone checks it in 1.4, you try to check it in and get a conflict, can join in the changes with cvs update -j 1.4 file.c add a new file: {{{ cvs add file; cvs commit file }}} remove a file (but keep in older revisions): {{{ rm file; cvs remove file; cvs commit file }}} rename a file: {{{ mv old new; cvs remove old; cvs add new; cvs commit old new }}} To receive an email notification about any file that has been commited by somebody else: {{{ cd dev cvs watch add }}} === Troubleshooting autoconf issues === [wiki:Self:AutoconfTroubleshooting Here's a list of common problems and solutions for autotools problems.] |
Index
This page is targeted at those who have access to the FreeSurfer source code, and wish to build this source code on their own Unix platform (whether it be an NMR Martinos Center platform, which has all required libraries, or on some arbitrary Unix platform, where the dependent libraries must be installed prior to building FreeSurfer).
1. CVS Checkout
If you have an NMR account, you can checkout the FreeSurfer source code from the NMR center using local CVS access or remotely by using SSH as the CVS remote connection method.
1.1. Local CVS Access
The CVS repository is /space/repo/1/dev. Use this as your CVSROOT. You can either set it as an environment variable:
setenv CVSROOT /space/repo/1/dev
or specify it in the checkout command with the -d option. Note that the CVS root is cached in a CVS checkout directory, so if you choose to use the -d method, you will only have to do it once during your first checkout.
Check out the code with the CVS checkout command. The archive name is dev.
cvs checkout -P dev
or
cvs -d /space/repo/1/dev checkout -P dev
This will copy the entire archive to your directory, creating a directory called dev/. The -P option will purge the old directories that have been removed from the CVS repository.
Generally it is handy to define an environment variable called FSDEV which is set to your Freesurfer development directory. Be sure also to define your FREESURFER_HOME environment variable set to the intended Freesurfer installation directory.
1.2. Remote CVS Access
Tell CVS to use SSH to access the archive by setting the following environment variable:
setenv CVS_RSH ssh
Use the following string as your CVS root:
setenv CVSROOT :ext:USER@MACHINE.nmr.mgh.harvard.edu:/space/repo/1/dev
Where USER is your username and MACHINE is one of the NMR machines visible to the outside. Then use the CVS commands normally.
Check out the code with the CVS checkout command. The archive name is dev.
cvs checkout -P dev
Note that using this method makes an SSH connection for every CVS command, and you will be required to enter your password every time. You may want to look into a utility to automatically authenticate SSH connections, such as SSH agent. See:
http://mah.everybody.org/docs/ssh
When working in a CVS checkout directory, cvs knows where the CVSROOT is for that directory. However, it doesn't know what CVS_RSH method to use, so if you are grabbing files remotely, you may want to set this in your login file.
2. Open Source Distribution
not available just yet
Download the file <put link here> and untar/gunzip. It will create a directory called 'dev'.
In advance of the availability of the open-source distribution, to get a jump-start on building this open-source distribution (ie, making sure the necessary 3rd-party packages exist on your system) the 'configure' script is available here:
ftp://surfer.nmr.mgh.harvard.edu/pub/dist/fs_supportlibs/
If you can get configure to run to completion successfully, then 90% of the work is done (that is, the integration of the 3rd-party packages is the biggest hassle in building the Freesurfer application).
3. Configuring
While not strictly necessary, NMR center developers are encouraged to run a pre-configure script, to create the platform specific tools required by configure (execute in the dev/ directory created by cvs checkout):
./setup_configure
This script runs a set of commands (aclocal, libtoolize, automake, autoconf) that creates the platform specific files for configure and puts them in the 'dev/config' directory. Those who are working with the open-source distribution should not need to run this script, as the default files should work for most platforms, but ./setup_configure is there if someone is having autoconf issues.
Now you need to configure your building parameters for your machine:
./configure
This runs the configure script which checks a bunch of local settings and libraries that are needed. You will probably need to supply some extra options:
--prefix=~/work/freesurfer
The prefix option determines where your build will be installed when you use the 'make install' command. This is your private copy of FreeSurfer, not a release for others, so put it in your own directory space. Generally, a developer will set their $FREESURFER_HOME to their working directory (in this case FREESURFER_HOME is ~/work/freesurfer).
If you are compiling in the NMR Center, pass the option:
--enable-nmr-install
This will look for and install some files that are only usable in the Center. These files are not included in external (open-source) release, so using this option on an external release will result in an error as required files will not be found.
There are options to specify where certain packages exist, such as: --with-mni-dir=/usr/pubsw/packages/mni/current These are often needed so configure can find libraries installed in /usr/pubsw. Note that you should have /usr/pubsw mounted on your machine. This is normal for NMR computers.
Freesurfer builds against the following set of open-sourced libraries, thus, these must be installed prior to building:
- GSL
- JPEG
- TIFF
- Tcl 8.4
- Tk 8.4
- Tix 8.4
- BLT
- GLut
- Qt 3.3.4
If these libraries are not available on your system, then they must be built and installed. Special bundles of this open-source code are available on the distribution site:
ftp://surfer.nmr.mgh.harvard.edu/pub/dist/fs_supportlibs/
Also in this 'bundle' directory is a Linux script called 'fs_lib_check' which will check your system for these libraries given some prior sysadmin installation. If you do not have a set of libs, then the .tar.gz files contain the open-source code from which they can be built. The files gsl-1.6.tar.gz, tcltktixblt.tar.gz, tiffjpegglut.tar.gz, and qt-x11-free-3.3.4.tar.gz each contain special build scripts that perform the work of building and installing these libraries. Refer to those scripts (with 'build_' in the filename preface) to see what they do. For the simplest installation, it is recommended to install these packages in a directory called /usr/pubsw/packages, as the Freesurfer setup script checks for this directory for its packages during its setup. Here is a table of recommended installation paths for the 3rd-party packages:
/usr/pubsw/packages/gsl/current |
gsl-1.6.tar.gz |
/usr/pubsw/packages/tcltktixblt/current |
tcltktixblt.tar.gz |
/usr/pubsw/packages/tiffjpegglut (Mac OS X only) |
tiffjpegglut.tar.gz |
/usr/pubsw/packages/qt/current |
qt-x11-free-3.3.4.tar.gz |
/usr/pubsw/packages/mni/current |
mni.srcbuild.tar.gz |
Freesurfer also builds against certain packages of the MINC toolkit. A special bundle of the MINC toolkit source code for the MINC packages required by Freesurfer has been assembled at MGH and is available from the download site:
ftp://surfer.nmr.mgh.harvard.edu/pub/dist/fs_supportlibs/
Look for mni.srcbuild.tar.gz, and mni.srcbuild.README.txt describes the build process.
A sample configure line for a Linux machine at the NMR center having appropriate NFS mount-points (ie. /usr/pubsw/packages):
./configure \
--enable-nmr-install \
--prefix="$FREESURFER_HOME" \
--with-mni-dir="/usr/pubsw/packages/mni/current" \
--with-tcl-dir="/usr/pubsw/packages/tcltktixblt/current" \
--with-gsl-dir="/usr/pubsw/packages/gsl/current" \
--with-qt-dir="/usr/pubsw/packages/qt/current"Before attempting to build and install the 3rd-party packages from source, it is recommended to just start by running:
./configure \
--enable-nmr-install \
--prefix="$FREESURFER_HOME"and observe the errors. configure will tell you which packages are already installed on your system. Read the file config.log, generated by configure, for error messages.
Be aware that the tcl/tk/tix/blt combo is particularly tricky. While many systems have tcl/tk installed, the tricky thing is getting tix and BLT to work with tcl/tk. Thus, our installation package (tcltktixblt.tar.gz) is the easiest route to successfully getting these four guys to work together.
4. Building
You can now run 'make', edit and build code, etc. The first thing you should do is 'make' to build libraries like libutils. (Note that all libs will be built to your local checkout directory.) To just build libutils, cd into utils/ and run make.
You can now compile individual programs or all of them at once. To compile all of them, just run 'make' from dev/. Binaries will automatically be placed in their individual subdirectories. Like this:
cd dev make
However you may just want to compile one of them at a time. To do so, cd to the directory of the program you want and use 'make' to compile it:
cd mri_info make
This creates mri_info in the mri_info/ directory.
5. Adding a new binary to the tree
Assuming that you have a source file MYPROG.c that compiles into MYPROG and want to add it to the FreeSurfer tree:
1) Make the directory in dev and copy the source file there. Name the directory MYPROG and the source file MYPROG.c.
mkdir dev/MYPROG cp MYPROG.c dev/MYPROG
2) Tell the autotools to build your program when you type make from the top dir.
a) Modify dev/configure.in to add MYPROG/Makefile to the list at the end of the file in the definition of AC_OUTPUT. Be sure to add a backslash at the end of line:
AC_OUTPUT( \ ... other files ... MYPROG/Makefile \ )
b) Modify dev/Makefile.am to add MYPROG to the SUBDIRS definition. (You can also alternatively add it to the end of MRISUBDIRS or MRISSUBDIRS if more appropriate.)
SUBDIRS= ... other directories ... MYPROG
c) Copy dev/dummy/Makefile.am into MYPROG/ and customize it, replacing 'dummy' with 'MYPROG'. Be sure to change:
bin_PROGRAMS = MYPROG
d) Copy in the additional testing file dev/dummy/myown.c. You can customize it for your test program later.
3) Run automake from dev/. You should get no errors. If you do, make sure you followed the above instructions properly. Also try the AutoconfTroubleshooting page. Verify that this stepped work by checking if MYPROG/Makefile.in was created.
4) Run autoconf to generate a new configure script that now includes your new MYPROG directory.
5) Run ./configure with the parameters you previously used. To check these out, run head config.log from dev/. The output should include the ./configure line you used. Copy it, but leave out the --no-create --no-recursion options if present.
[dev/]$ head config.log This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by Freesurfer configure 0.1, which was generated by GNU Autoconf 2.57. Invocation command line was $ ./configure --with-mni-dir=/usr/pubsw/packages/mni/current --prefix=/home/kteich/freesurfer/dev --no-create --no-recursion ## --------- ## ## Platform. ## [dev/]$ ./configure --with-mni-dir=/usr/pubsw/packages/mni/current --prefix=/home/kteich/freesurfer/dev
Note: Do not just copy this example, use what's in your own config.log file!
6) Run make and verify that your binary program MYPROG/MYPROG was created.
7) Check in your changes.
[dev/] cvs ci -m "Added MYPROG" configure configure.in Makefile.am Makefile.in [dev/] cvs add MYPROG [dev/] cd MYPROG [MYPROG/] cvs add Makefile.am Makefile.in MYPROG.c myown.c [MYPROG/] cvs commit -m "First checkin." Makefile.am Makefile.in MYPROG.c myown.c
6. Installing
To install all binaries and support files into your private FreeSurfer, use 'make install' from the toplevel dev/ directory, like this:
cd dev make install
This will make a directory called freesurfer/ in the directory specified by the --prefix option to configure, above. Note that if you do not specify this location, it will try to install to /usr/local, which you probably don't have permission to do. Even if you do, i.e. you are installing on a laptop, it's generally better to specify a prefix of /usr/local/freesurfer to keep everything in the same place.
Note that you can also run 'make release'. 'make install' makes and installs the NMR internal version of FreeSurfer, while 'make release' makes the public version which omits some stuff.
The first time you run 'make install', it will take a while to copy all the big data files to the new installation. Subsequent 'make installs' will only copy the changed files.
If you only want to install a single binary, run 'make install' from a subdirectory. For example, running 'make install' from the scuba/ directory will copy the scuba binary and its support script files to the proper locations. Running 'make install' from scripts/ will copy all the necessary scripts to the right location.
7. Using CVS
When you want to commit your changes, use the cvs commit command. However, you must first check to see if the file(s) in the archive have changed since the last time you checked them out. To do so, run the cvs update command on the files you have changed. The -P option will purge (remove) files that have been removed from the CVS.
cvs update -P
If any have changed, you will see the letter U followed by the file name.
U myFile.c
This means somebody else committed a newer version of the file, and your copy was just updated.
If you and somebody else have made changes on the same file that conflict each other, you will see the letter C along with a message about the conflicts.
C myFile.c
If the message states the conflicts have been resolved, you don't have to worry: cvs has merged the differences properly. If it says it couldn't merge the differences, you need to open the file in question, search for the string >>>, and merge the differences yourself.
Once the up to date check has been performed, you can commit your changes. If you are in a directory and want to commit all the files in that directory that you have changed, just use:
cvs commit
This will commit changes in the current directory and any subdirectories. If you want to commit only certain files, use:
cvs commit <filename>
Or you can only commit the code in the current directory with the -l switch:
cvs commit -l
Each time, an editor will open and ask you to enter a log message. To bypass this, use the -m option and enter your log message on the command line:
cvs commit -m "some descriptive changelog comment" <filename>
A synonym for commit is ci, so you can just:
cvs ci -m "fixed stuff" <filename>
Periodically, you should update your working copy to be in sync with the archive. CVS will look at all your files and see if the ones in the archive are different. If so, it will update your copy. CVS will not delete local modifications you have made. To do this update:
cvs update -d -P
The -d switch adds any new directories and is not strictly necessary, but is a good habit, because if you don't use it you will update existing files but will now add new directories. The -P option purges directories of files that have been removed in the past.
Similarly, you can update an entire directory or a single file. To commit all the code in the current directory (code that has not been changed will not be commited):
cvs ci -m "fixed many things"
To restore an edited version to the version in the archive, either delete the file and cvs update, and the archive version will be copied to the local work area, or use the -C option to cvs update to replace your copy with one from the archive.
To add a new file to the archive, use the add command and then commit the file:
cvs add <filename> cvs commit -m "added the file" <filename>
To remove a file, remove in cvs and locally (with the -f flag), and commit:
cvs remove -f <filename> cvs commit -m "removed the file" <filename>
That covers most of the CVS commands you'll have to to use.
Additional stuff:
To tag a group of code with a text label:
cvs tag alpha-1
To retrieve a tagged group of code:
cvs update -r alpha-1
There are more useful commands, but these are the most commonly used ones. See http://www.cvshome.org for more info.
For more info on CVS, check out www.cvshome.org or the CVS man pages.
Summary:
check in a file
cvs commit
get newest versions. merges newer versions than the ones you had with your working versions.
cvs update -A
tag working files:
cvs tag -c tag-name
if any working copies are edited, will report an error. check in first.
checkout a tag:
cvs checkout -r tag-name archive
split off a branch based on current revisions in working copy:
cvs tag -b branch-name
note: doesn't make working copy the branch.
split off a branch:
cvs rtag -b -r tag-name branch-name archive
check out a branch:
cvs checkout -r branch-name archive
switch working copy to a branch:
cvs update -r branch-name archive
-or-
BCD archive; cvs update -r branch-name
to find out what branch the working copy is on:
cvs status
look for sticky tag field.
join the changes in a branch to current working copy:
cvs update -j branch-to-join file
best to tag merged branches after every merge.
joing the changes in a revision to current working copy:
cvs update -j rev file
i.e. two people check out 1.3, someone checks it in 1.4, you try to check it in and get a conflict, can join in the changes with cvs update -j 1.4 file.c
add a new file:
cvs add file; cvs commit file
remove a file (but keep in older revisions):
rm file; cvs remove file; cvs commit file
rename a file:
mv old new; cvs remove old; cvs add new; cvs commit old new
To receive an email notification about any file that has been commited by somebody else:
cd dev cvs watch add
8. Troubleshooting autoconf issues
[wiki:AutoconfTroubleshooting Here's a list of common problems and solutions for autotools problems.]
9. Martinos Center issues
9.1. Making changes live
The current setup allows you to have your own private installation of FreeSurfer using configure --prefix=/your/private/directory. This lets you use make install to easily install the entire FreeSurfer environment into a local copy.
The purpose of having your own private installation of FreeSurfer is to use it as a testbed to make sure that (1) your updates, whatever they be, work in freesurfer/ setup that is just like a real one, and (2) your updates are properly installed using 'make install' and 'make release'. Once you are sure of these two things, check in your updated files, including any Makefile.am and Makefile.in changes, etc, and the nightly build script will take care of the rest.
The nightly build stuff automatically updates and installs the dev/ distribution on all the build platforms (rh9, centos4, centos4-x86_64 and mac os). The next morning, just check if your changes are in any /usr/local/freesurfer/dev; if not, email fsdev. If they are in one /usr/local/freesurfer, they will be in the rest of them. (If any build/installs fail, fsdev group is notified, and will do the build/install manually for that day.)
Things not to do:
- Don't test your local setup by manually copying files into your local freesurfer/. Otherwise you can't be sure that the automated build will correctly install your files with 'make install' and 'make release'. The only way files should get into your freesurfer/ distro as with 'make install' and 'make release'.
- Don't try to install into /usr/local/freesurfer yourself (i.e. by setting your .configure --prefix=... to target it). It's set up so the build scripts can do their job automatically. Also, you would only be installing on one of the platforms. It's a pain to do all three builds on all build machines and verify it all worked - that's what the nightly build script is for, so let it do it for you.
If you absolutely need a 'hot fix' right away, and need to make changes live in any of the /usr/local/freesurfers before the nightly build stuff goes, let KevinTeich or NickSchmansky know and they'll do it.
9.2. How the nightly build works
First a note on the directory structure. At the NMR Center, in /space/freesurfer/build there is a directory for each build machine. Each builds a platform in /space/freesurfer, i.e. /space/freesurfer/rh9. Currently these are as follows:
Build machine |
Platform |
Processor |
Nightly build time |
martinos01 |
Linux-rh7.3 |
i686 dual p3 500MHz, 1GB |
6:00AM |
kani |
Linux-rh9 |
i686 dual p3 1GHz, 2GB |
1:00AM |
fishie |
Linux-centos4.2 |
i686 dual p4 3GHz, 3.5GB |
3:00AM |
minerva |
Linux-centos4.2_x86_64 |
x86_64 dual opteron 2.7GHz, 8GB |
5:00AM |
storm |
Darwin-tiger10.4.3 |
powerpc dual g5 2.5GHz, 8GB |
4:00AM |
Inside each /space/freesurfer/build/MACHINE is trunk/ and stable/. The file configure_options.txt contains command-line arguments for configure for that platform, and source_before_building.csh (optional) contains environment variables that must be set (mainly things like CPPFLAGS that are used by configure but can't be passed in to configure via the command line from a cronjob). The trunk and stable directories contain cvs checkouts (use -P flag, ie 'cvs checkout -P', to purge deleted cvs directories).
In /space/freesurfer/build/scripts is build_dev.csh and build_betapub.csh. Each build machine runs build_dev.csh nightly. It basically goes to /space/freesurfer/build/MACHINE/trunk/dev and does the following:
cvs update -d
make distclean
rm -rf autom4te.cache
aclocal
autoconf
automake
setenv FREESURFER_HOME /usr/local/freesurfer/dev
./configure `cat ${BUILD_DIR}/configure_options.txt` \
--prefix=$FREESURFER_HOME \
--bindir=$FREESURFER_HOME/bin-new
make
make install
rm -rf $FREESURFER_HOME/bin_old
mv $FREESURFER_HOME/bin $FREESURFER_HOME/bin_old
mv $FREESURFER_HOME/bin-new $FREESURFER_HOME/binA few things to note about this script. The prior /bin dir is saved to /bin-old. This is for two reasons: 1) for processes running against this nightly build, this prevents those binaries from core-dumping; 2) for builds that create a binary that core-dumps (due to some bug that was just introduced), the prior nights binary can be recovered and copied to /bin, until the bug is fixed. The second thing to note is that the binaries are built into /bin-new, and only after make install is complete (which installs the binaries into /bin-new), is /bin-new moved to /bin. This is, again, to accommodate recon-all scripts running overnight (running against the nightly build environment), so that a binary does not appear 'missing' during the switchover time to the new binaries.
The build_betapub.csh works in /space/freesurfer/build/MACHINE/stable/dev and also does:
make release prefix=/usr/local/freesurfer/pub
All the autotools stuff guarantees that enough of the Makefiles will be regenerated to ensure that new directories will be added to all Makefiles. This wouldn't normally be done with a simple ./config.status because configure may need to be regenerated to know about new directories.
Note that all machines can run these scripts at any time from the command line. This lets maintainers (KevinTeich and NickSchmansky) explicity do a build and install on any machine if a 'hot fix' is needed.
Additionally, a developer can go to each machine's /space/freesurfer/build/MACHINE/trunk/dev, update code, and do a make install from an individual binary directory if a hot fix is needed for a single binary before the nightly build. This is the quick and dirty way of doing an update for a single binary without waiting for the nightly build. For example, to update tkmedit, the developer would, for each build machine:
cd /space/freesurfer/build/MACHINE/trunk/dev/tkmedit cvs update make install
This would install the files to /usr/local/freesurfer/dev. However, the developer must know about file dependencies. In this case, the script files that tkmedit may also need to be updated, so the developer would also have to cvs update and make install from dev/scripts.
9.3. How the daily testing works
Note: work-in-progress!
The long-term goal is to run a suite of tests on each platform to ensure that Freesurfer is consistently producing the same results. This ideal is difficult to achieve, given the wide variety of process streams that exist. However, the basic testing involves running recon-all on one or more reference subjects, and checking that the results (volume and surface files, and stats) are identical to past results (the reference).
The test platforms are:
Test machine |
Platform |
Processor |
Day of Week and Time of Test |
jc |
Linux-centos4.2 |
i686 quad xeon 2.8GHz, 4GB |
Mon@6AM Tues@noon Wed@6PM Thurs@midnight Sat@6AM |
hades |
Linux-centos4.2_x86_64 |
x86_64 quad xeon 3.2GHz, 8GB |
Mon@noon Tues@6PM Wed@midnight Fri@6AM Sat@noon |
mist |
Darwin-tiger10.4.3 |
powerpc dual g5 2.7GHz, 8GB |
Mon@6PM Tues@midnight Thur@6AM Fri@noon Sat@6PM |
The script space/freesurfer/build/scripts/test_recon-all.csh runs the testing on a platform. Basically, it will:
source /usr/local/freesurfer/dev/SetUpFreeSurfer.csh recon-all -i <reference_src_mri_orig_files> -s <test_subject> -all mri_diff <reference_volumes> <test_subject_volumes> mris_diff <reference_surfaces> <test_subject_surfaces> diff <reference_stats_files> <test_subject_stats_files>
Results (eventually) will be posted to a [http://public.kitware.com/Dart/HTML/Index.shtml Dart] server which will post a webpage of build/test results.
9.4. Updating the stable branch
This section covers how to manage the different branches of the FreeSurfer code and how to make in-house public releases.
The CVS repository for FreeSurfer has a stable branch. This is a path of code that we will maintain and compile for use as FreeSurferStable that will not receive any new features, only bug fixes. If a bug is found in the stable version, the bug fixes will have to be done on the stable branch of code, and then merged into the main branch.
You should have two complete copies of the code repository, one for the main branch or trunk, and one for the stable branch. It is suggested that you create an fsdev/ directory and put two subdirectories in it, stable/ and trunk/ . In fsdev/trunk, do:
cvs checkout -P dev
to get the main trunk of code (and purge prior cvs-removed files).
In fsdev/stable, do:
cvs checkout -P -r stable2 dev
This assumes that 'stable2' is the name of the branch. (As of Jan 2005, the most current stable branch is 'stable2', but the next one will be 'stable3', etc.). This will create a sticky copy of the stable branch. All changes made with 'cvs commit' and accesses made with 'cvs update' in fsdev/stable will apply to the stable branch, not the trunk.
When a developer needs to make edits to the stable branch to fix a bug, cd into fsdev/stable/dev and do a cvs update to make sure you have the most current copy of the stable branch. Then make your edit as you normally would and cvs commit it.
Now the bug fix exists in the stable branch, but not the main trunk. You can use CVS's merging functionality to merge the changes from the stable branch to the trunk. This must be done on a file-by-file basis. (*)
- (*) Not technically, but in our repository we have binary and text files. Keywords are turned off for binary files, since we don't want binary data that happens to match keywords expanded by CVS, as this would change the binary unpredictably. Using the -kk option works for text files, but using it with binary files clears the flag that tells cvs not to use keyword expansion on the file. So, I suggest merging on a file-by-file basis and avoid batch merges to avoid accidentally using the -kk flag with a binary file and clearing the binary flag.
The files in the stable branch have their own revision history; there may be several revisions in the stable branch of a single file. However, we want to only merge the changes in the last revision to the file in the main trunk. Without specifying which revision to use, CVS will try to merge the changes in all revisions, which could be bad. So what we'll do is tag the file in the stable branch every time it's merged into the main trunk. (Note that the first time you update from a file in the stable branch, it will not have this tag.)
We'll call that tag 'merged-from-stable2', and every time we merge, we'll tell CVS to merge all the changes from the 'merged-from-stable2' tag to the most current revision into the file in the main trunk.
Say you edited mri.c in the stable branch. cd to fsdev/trunk/dev/utils. The command:
cvs update -kk -j merged-from-stable2 -j stable2 mri.c
will merge the changes in the stable version of mri.c with the tag 'merged-from-stable2' to the trunk version of mri.c. The -kk option prevents CVS keywords in the file (such as the Id keyword) from causing conflicts. The first -j tells us to start merging with the revision specified, 'merged-from-stable2', and the second -j tells us to merge up to the end of the 'stable2' branch.
If this is the first time you're merging in this branch, you'll get an error message saying that the tag 'merged-from-stable2' doesn't exist. If this is the case, use the 'pre-stable2-branch' tag, which was created before the stable2 merge:
cvs update -kk -j pre-stable2-branch -j stable2 mri.c
Note that you may get a conflict message, in which case you should resolve them by editing the file. Once the merge is successful, cvs commit it to make the change in the main trunk.
With a successful merge, we need to tag the file in the stable branch with the 'merged-from-stable2' tag. So go back to fsdev/stable/dev and tag that file with the command:
cvs tag -F merged-from-stable2 mri.c
Note that again you should specify the file name. Normally CVS will not tag a file with a tag name if that tag name already exists in the file. Since we basically want to create a moving tag, we want CVS to delete the original tag location and move it to the current revision. The -F option does this.
In summary, to edit a file (in this example, mri.c) in the stable branch and merge the change to the main trunk:
- cd fsdev/stable/dev -- move to stable branch
- cvs update -- update stable branch
- edit mri.c to make the change, build, and install into freesurfer_beta
- cvs commit utils/mri.c -- commit change to stable branch
- cd fsdev/trunk/dev -- move to main trunk (dev)
- cvs update -kk -j merged-from-stable2 -j stable2 utils/mri.c
- -- merge changes from stable into main
- cvs update -kk -j pre-stable2-branch -j stable2 utils/mri.c
- resolve conflicts in utils/mri.c, test build
- cvs commit utils/mri.c -- commit change to main trunk
- cd fsdev/stable/dev -- move to stable branch
- cvs tag -F merged-from-stable2 utils/mri.c
- -- tag the file in the stable branch
To build a version for InternalFreeSurferDistribution (FreeSurferStable / FreeSurferDev), use the public build directories on one of the build machines. Note that you must do the build on all versions so each platform distribution is updated. See the section on nightly builds for this information.
In /space/freesurfer/build, there are directories that represent each build. In each directory, there are two more directories: stable and trunk. These represent the build branches.
If you want to make a release of FreeSurferStable, ssh into each of the build machines, go to space/freesurfer/build/$MACHINE/stable/dev, then cvs update, make, and make install. The configuration files are already set up to install the software into the proper path in /usr/local/freesurfer. Note that you may need to cvs update -d to get new directories, then run the ./setup_configure script found in the /dev directory (runs autoconf, automake, etc.).
A script exists to do this, so, for example, to make a FreeSurferStable release:
- ssh minerva
- type: /space/freesurfer/build/scripts/build_betapub.csh
And repeat for other build machines.
