Stijn Hoop

stijn@win.tue.nl

2001 Stijn Hoop $Date$ $FreeBSD: doc/en_US.ISO8859-1/articles/cvs-freebsd/article.sgml,v 1.1 2002/01/07 20:15:05 keramida Exp $ This article describes the steps I took to setup a CVS repository that uses the same scripts the FreeBSD project uses in their setup. This has several advantages over a stock CVS setup, including more granular access control to the source tree and generation of readable email of every commit. Most of the open source software projects use CVS as their source code control system. While CVS is pretty good at this, it has its share of flaws and weaknesses. One of these is that sharing a source tree with other developers can quickly lead to a system administration nightmare, especially if one wishes to protect parts of the tree from general access. FreeBSD is one of the projects using CVS. It also has a large base of developers located around the world. They developed some scripts to make management of the repository easier. Recently, these scripts were revisited and normalized by Joseph Karthauser to make it easier to reuse them in other projects. This article describes one method of using the new scripts. To make the most use of the information in this article, you need to be familiar with the basic method of operation of CVS. It might be best to first perform this procedure with an empty test repository, to make sure you understand all consequences. As always, make sure you have recent, readable backups! The first thing to do when setting up a new repository is to tell CVS to initialize it: &prompt.user; cvs -d path-to-repository init This tells CVS to create the CVSROOT administrative directory, where all the customization takes place. Now we will create the group which will own the repository. All committers need to be in this group, so that they can write to the repository. We will assume the FreeBSD default of ncvs for this group. &prompt.root; pw groupadd ncvs Next, you should &man.chown.8; the directory to the user and group you just added: &prompt.root; chown -R :ncvs path-to-your-repository This ensures that no one can write to the repository without proper group permissions. Now you need to obtain the CVSROOT directory from the FreeBSD repository. This is most easily done by checking it out from a FreeBSD anonymous CVS mirror. See the relevant chapter in the handbook for more information. Let us assume that the sources are stored in CVSROOT-freebsd in the current directory. Next, we will copy the FreeBSD CVSROOT sources into your own repository. If you are accustomed to CVS, it might occur to you to try and import the scripts, in an attempt to make synchronizing with later versions easier. However, it turns out that CVS has a deficiency in this area: when importing sources into the CVSROOT directory, it will not update the needed administrative files. In order to make it recognize those, you will need to checkin each file after importing them, losing the value of cvs import. Therefore, the recommended method is to simply copy over the scripts. It does not matter if the above paragraph did not make sense to you—the end result is the same. Simply check out your CVSROOT and copy the FreeBSD files over your local (untouched) copies: &prompt.user; cvs -d path-to-your-repository checkout CVSROOT &prompt.user; cd CVSROOT &prompt.user; cp ../CVSROOT-freebsd/* . &prompt.user; cvs add * Note that you will probably get a few warnings about some directories not being copied; this is normal, you don't need those. Now you have in your working directory an exact copy of the scripts that the FreeBSD project itself uses for their repository. A summary of what each file is used for is included below. access - this file is not used in the default setup. It is used in the FreeBSD project specific setup, where it controls access to the repository. You can remove this file if you do not wish to use this setup. avail - this file controls access to the repository. In this, you can specify groups of people that are allowed access to the repository, as well as disallow commits on a per-directory basis. You should tailor it to contain the groups and directories that will be in your repository. cfg.pm - this file parses your configuration, and provides the default configuration. You should not make changes to this file. Instead, put your configuration changes in cfg_local.pm. cfg_local.pm - this file contains all configurable parameters of the system. You should configure all sorts of settings here, such as where commit mail is send, on what hosts people can commit, and others. More information on this below. checkoutlist - this files lists all files under control of CVS in this directory. You should edit this to remove some FreeBSD specific files. commit_prep.pl - this script performs various pre-commit checks, based on whether you enabled them in your cfg_local.pm. You should not have to touch this. commitcheck - this script is invoked directly from CVS. It first checks if the committer has access to the specified part of the tree, and then runs commit_prep.pl for the various pre-commit checks. If those are OK, CVS will allow the commit to proceed. You should not have to touch this file. commitinfo - this file is used by CVS to determine which script to run before a commit. You should not have to touch this file. config - the configuration file for this repository. You should change this as needed, but most administrators can probably leave the defaults. More information on the options that can be set here can be found in the CVS manual. cvs_acls.pl - this script determines the committers identity, and whether he/she is allowed access to the tree. It does this based on the avail file. You should not have to touch this file. cvsignore - this file specifies files that CVS should not checkin in the repository. You can edit this as you wish. More information about this file is available in the CVS manual. cvswrappers - this file is used by CVS to enable or disable keyword expansion, or whether a file should be considered binary. You can edit this as you wish. More information about this file is available in the CVS manual. edithook - this file is not used any more, but kept for historic reasons. You should not have to touch this file. editinfo - CVS uses this file for editor overrides. FreeBSD does not use this functionality, as parsing the log message is done in the verifymsg and logcheck files. You should not have to touch this file. exclude - this file lists regular expressions that are used to determine files which cannot contain a revision header. In the FreeBSD setup, all files under revision control need to have a revision header (like $FreeBSD$). All filenames that match one of the lines in this file are exempted from this check. You should add expressions to this file as you checkin files that cannot have a revision header. For the purpose of installing the scripts, it may be best to exclude CVSROOT/ from header checks. log_accum.pl - this is a script that takes the log message as provided by the logcheck script, and appends it to a log file in the repository for backup purposes. It also handles mailing out a message to an email address you provide (in cfg_local.pm). You should not have to touch this file. logcheck - this file parses the commit log message that committers provide, and attempts to sanitize it somewhat. This script depends on a local FreeBSD hack of CVS: this version reads the log message back in after this script has modified it. The stock version of CVS apparently does not, which would make this script useless. loginfo - this file is used by CVS to control where log information is sent. You should not have to touch this file. modules - this file retains its traditional meaning in CVS. You should remove the FreeBSD modules from the stock version. You can edit this as you wish. More information about this file is available in the CVS manual. notify - this file is used by CVS in case someone sets a watch on a file. It is not used in the FreeBSD repository. You can edit this as you wish. More information about this file is available in the CVS manual. options - this file is specific to the FreeBSD version of CVS. It contains the keyword to expand in revision headers. You should alter this to match the keyword you specified in cfg_local.pm (if you use that feature, which is FreeBSD specific for now). rcsinfo - this file maps directories in the repository to template files. By default, FreeBSD uses one template for the whole repository. You can add others to this file if you wish. rcstemplate - this file is the actual template committers will see when they make a checkin. You should edit this to describe the various extra parameters you defined in cfg_local.pm. tagcheck - this files controls access to tagging in the repository. The stock FreeBSD version disallows tags with names of RELENG*, because of the release engineering process. You should edit this file as desired. taginfo - this file maps tag operations on repository directories to access control scripts. You should not have to touch this file. unwrap - this script is needed to automatically unwrap binary files (see cvswrappers) on checkout. It is not used in the current FreeBSD setup. You should not have to touch this file. verifymsg - this file maps repository directories to post processor scripts of log messages. You should not have to touch this file. wrap - this script is needed to automatically wrap binary files (see cvswrappers) on checkin. It is not used in the current FreeBSD setup. You should not have to touch this file. The next step is to setup the scripts so that they work in your environment. You should go over all files in the directory and make your customizations. In particular, you might want to do edit the following files: If you do not wish to use the FreeBSD specific features of the scripts, you can safely remove the access file: &prompt.user; cvs rm -f access Edit avail to contain the various repository directories in which you want to control access. Make sure you retain the avail||CVSROOT line, otherwise you will lock yourself out in the next step. The other thing you can add in this file are committer groups. By default, FreeBSD uses the access file to list all its committers in, but you can use any file you wish. You can also add groups if you want (the syntax is specified at the top of cvs_acls.pl). Edit cfg_local.pm to contain the options you want. In particular, you should take a look at the following configurable items: %TEMPLATE_HEADERS - these get processed by the log scripts, and inserted below the commit mail if present and non-empty in the commit message. You can probably remove the PR and MFC after entries. And of course you can add your own. $MAIL_BRANCH_HDR - if you want to insert a header into each commit mail describing the branch on which the commit was made, define this to match your setup. Or leave it empty if you don't want such a header. @COMMIT_HOSTS - define this to be a list of hosts on which people can commit. $MAILADDRS - set this to the admin or list address that should receive commit mail. @LOG_FILE_MAP - change this array as you wish - each regexp is matched on the directory of the commit, and the commit log message gets stored in the commitlogs subdirectory in the filename mentioned. $COMMITCHECK_EXTRA - if you do not want to use the FreeBSD specific access checks, you should remove the definition of $COMMITCHECK_EXTRA from this file. Changing the $IDHEADER parameter is only guaranteed to work on FreeBSD platforms; it depends on FreeBSD specific modifications to CVS. You can check cfg.pm to see which other options can be changed, but the above is a reasonable subset. Edit exclude to remove the FreeBSD specific entries (such as all lines beginning with ^ports/ etc.). Furthermore, comment out the lines beginning with ^CVSROOT/, and add one line with only ^CVSROOT/ on it. After the wrapper is installed, you can add your header to the files in the CVSROOT directory and restore these lines, but for now they will only be in the way when you try to commit later on. Edit modules, and delete all FreeBSD stuff. Add your own modules if you wish. This step is only necessary if you specified a value for $IDHEADER in cfg_local.pm (which only works using a FreeBSD modified CVS). Edit options to match the tag you specified in cfg_local.pm. A global search and replace of FreeBSD with your tag should suffice. Edit rcstemplate to contain the same keywords as specified in cfg_local.pm. Optionally remove the FreeBSD checks from tagcheck. You can simply add exit 0 to the top of the file to disable all checks on tagging. The last thing to do before you are finished, is to make sure the commitlogs can be stored. By default these are stored in the repository, in the commitlogs subdirectory of the CVSROOT directory. This directory needs to be created, so do the following: &prompt.user; mkdir commitlogs &prompt.user; cvs add commitlogs Now, after careful review, you should commit your changes. Be sure that you have granted yourself access to the CVSROOT directory in your avail before you do this, because otherwise you will lock yourself out. So make sure everything is as you intend, and then do the following: &prompt.user; cvs commit -m '- Initial FreeBSD scripts commit' You are ready for the first test: a forced commit to the avail file, to make sure everything works as expected. &prompt.user; cvs commit -f -m 'Forced commit to test the new CVSROOT scripts' avail If everything works, congratulations! You now have a working setup of the FreeBSD scripts for your repository. If CVS still complains about something, go back and recheck if all of the above steps have been performed correctly. The FreeBSD project itself uses a slightly different setup, which also uses files from the freebsd subdirectory of the FreeBSD CVSROOT. The project uses this because of the large number of committers, which all would have to be in the same group. So, a simple wrapper was written which ensures that people have the correct credentials to commit, and then sets the group id to that of the repository. If your repository also needs this, the steps to set this up are documented below. But first an overview of the files involved. access - this file controls access information. You should edit this file to include all members of your project. freebsd/commitmail.pl - this file is not used any more, but kept for historic reasons. You should not have to touch this file. freebsd/cvswrap.c - this is the source to the CVS wrapper that you will need to install to make all access checks actually work. More information on this below. You should edit the paths in the ACCESS and REALCVS macros to match your setup. freebsd/mailsend.c - this file is needed by the FreeBSD setup of the mailing lists. You should not have to touch this file. Edit the access file to contain only your username. Edit cvswrap.c to contain the correct path for your setup. This is defined in a macro named ACCESS. You should also change the location of the real cvs binary if it is not appropriate to your situation. The stock cvswrap.c expects to be a replacement for the systemwide cvs command, which will be moved to /usr/bin/ncvs. My copy of cvswrap.c has this: #define ACCESS "/local/cvsroot/CVSROOT/access" #define REALCVS "/usr/bin/ncvs" Next up is installing the wrapper to ensure you become the correct group when committing. The sources for this live in cvswrap.c in your CVSROOT. Compile the sources that you edited to include the correct paths: &prompt.user; cc -o cvs cvswrap.c And then install them (you have to be root for this step): &prompt.root; mv /usr/bin/cvs /usr/bin/ncvs &prompt.root; mv cvs /usr/bin/cvs &prompt.root; chown root:ncvs /usr/bin/cvs /usr/bin/ncvs &prompt.root; chmod o-rx /usr/bin/ncvs &prompt.root; chmod u-w,g+s /usr/bin/cvs This installs the wrapper as the default cvs command, making sure that anyone who wants to use the repository has to have the correct access levels. You can now remove everyone from your repository group. All access control is done by your wrapper, and this wrapper will set the correct group for access. Your wrapper should now be setup. You can of course test this by making a forced commit to the access file: &prompt.user; cvs commit -f -m 'Forced commit to test the new CVSROOT scripts' access Again, if this fails, check to see whether all of the above steps have been executed correctly.