cvs.sgml

<!--
$Header: /cvsroot/pgsql/doc/src/sgml/cvs.sgml,v 1.24 2002/10/20 20:58:02 momjian Exp $
CVS code repository
Thomas Lockhart
-->

<appendix id="cvs">
 <docinfo>
  <authorgroup>
   <author>
    <firstname>Marc</firstname>
    <surname>Fournier</surname>
   </author>
   <author>
    <firstname>Tom</firstname>
    <surname>Lane</surname>
   </author>
   <author>
    <firstname>Thomas</firstname>
    <surname>Lockhart</surname>
   </author>
  </authorgroup>
  <date>1999-05-20</date>
 </docinfo>

 <title>The <productname>CVS</productname> Repository</title>

 <para>
  The <productname>PostgreSQL</productname> source code is stored and managed using the
  <productname>CVS</productname> code management system.
 </para>

 <para>
  At least two methods,
  anonymous CVS and <productname>CVSup</productname>,
  are available to pull the <productname>CVS</productname> code tree from the
  <productname>PostgreSQL</productname> server to your local machine.
 </para>

 <sect1 id="anoncvs">
  <title>Getting The Source Via Anonymous <productname>CVS</productname></title>

  <para>
   If you would like to keep up with the current sources on a regular
   basis, you can fetch them from our <productname>CVS</productname> server
   and then use <productname>CVS</productname> to
   retrieve updates from time to time.
  </para>

  <procedure>
   <title>Anonymous CVS</title>

   <step>
    <para>
     You will need a local copy of <productname>CVS</productname>
     (Concurrent Version Control System), which you can get from
     <ulink url="http://www.cyclic.com/">http://www.cyclic.com/</ulink> or
     any GNU software archive site.
     We currently recommend version 1.10 (the most recent at the time
     of writing). Many systems have a recent version of
     <application>cvs</application> installed by default.
    </para>
   </step>

   <step>
    <para>
     Do an initial login to the <productname>CVS</productname> server:

     <programlisting>
$ cvs -d :pserver:anoncvs@anoncvs.postgresql.org:/projects/cvsroot login
     </programlisting>

     You will be prompted for a password; just press <literal>ENTER</literal>.
     You should only need to do this once, since the password will be
     saved in <literal>.cvspass</literal> in your home directory.
    </para>
   </step>

   <step>
    <para>
     Fetch the <productname>PostgreSQL</productname> sources:
     <programlisting>
cvs -z3 -d :pserver:anoncvs@anoncvs.postgresql.org:/projects/cvsroot co -P pgsql
     </programlisting>

     which installs the <productname>PostgreSQL</productname> sources into a
     subdirectory <filename>pgsql</filename>
     of the directory you are currently in.

     <note>
      <para>
       If you have a fast link to the Internet, you may not need
       <option>-z3</option>, which instructs
       <productname>CVS</productname> to use gzip compression for transferred data.  But
       on a modem-speed link, it's a very substantial win.
      </para>
     </note>
    </para>

    <para>
     This initial checkout is a little slower than simply downloading
     a <filename>tar.gz</filename> file; expect it to take 40 minutes or so if you
     have a 28.8K modem.  The advantage of
     <productname>CVS</productname>
     doesn't show up until you want to update the file set later on.
    </para>
   </step>

   <step>
    <para>
     Whenever you want to update to the latest <productname>CVS</productname> sources,
     <command>cd</command> into
     the <filename>pgsql</filename> subdirectory, and issue
     <programlisting>
$ cvs -z3 update -d -P
     </programlisting>

     This will fetch only the changes since the last time you updated.
     You can update in just a couple of minutes, typically, even over
     a modem-speed line.
    </para>
   </step>

   <step>
    <para>
     You can save yourself some typing by making a file <filename>.cvsrc</filename>
     in your home directory that contains

     <programlisting>
cvs -z3
update -d -P
     </programlisting>

     This supplies the <option>-z3</option> option to all cvs commands, and the
     <option>-d</option> and <option>-P</option> options to cvs update.  Then you just have
     to say
     <programlisting>
$ cvs update
     </programlisting>

     to update your files.
    </para>
   </step>
  </procedure>

  <caution>
   <para>
    Some older versions of <productname>CVS</productname> have a bug that
    causes all checked-out files to be stored world-writable in your
    directory.  If you see that this has happened, you can do something like
    <programlisting>
$ chmod -R go-w pgsql
    </programlisting>
    to set the permissions properly.
    This bug is fixed as of
    <productname>CVS</productname> version 1.9.28.
   </para>
  </caution>

  <para>
   <productname>CVS</productname> can do a lot of other things,
   such as fetching prior revisions
   of the <productname>PostgreSQL</productname> sources
   rather than the latest development version.
   For more info consult the manual that comes with
   <productname>CVS</productname>, or see the online
   documentation at
   <ulink url="http://www.cyclic.com/">http://www.cyclic.com/</ulink>.
  </para>
 </sect1>

 <sect1 id="cvs-tree">
  <title><productname>CVS</productname> Tree Organization</title>

  <para>
   <note>
    <title>Author</title>
    <para>
     Written by Marc G. Fournier (<email>scrappy@hub.org</email>) on 1998-11-05
    </para>
   </note>
  </para>

  <para>
   The command <command>cvs checkout</command> has a flag, <option>-r</option>,
   that lets you check out a
   certain revision of a module.  This flag makes it easy to, for example,
   retrieve the
   sources that make up release 6_4 of the module `tc' at any time in the
   future:

   <programlisting>
$ cvs checkout -r REL6_4 tc
   </programlisting>

   This is useful, for instance, if someone claims that there is a bug in
   that release, but you cannot find the bug in the current working copy.

   <tip>
    <para>
     You can also check out a module as it was at any given date using the
     <option>-D</option> option.
    </para>
   </tip>
  </para>

  <para>
   When you tag more than one file with the same tag you can think
   about the tag as <quote>a curve drawn through a matrix of filename vs.
   revision number</quote>.  Say we have 5 files with the following revisions:

   <programlisting>
             file1   file2   file3   file4   file5

             1.1     1.1     1.1     1.1  /--1.1*      <-*-  TAG
             1.2*-   1.2     1.2    -1.2*-
             1.3  \- 1.3*-   1.3   / 1.3
             1.4          \  1.4  /  1.4
                           \-1.5*-   1.5
                             1.6
   </programlisting>

   then the tag <literal>TAG</literal> will reference
   file1-1.2, file2-1.3, etc.

   <note>
    <para>
     For creating a release branch, other then a
     -b option added to the command, it's the same thing.</para>
   </note>
  </para>

  <para>
   So, to create the 6.4 release
   I did the following:

   <programlisting>
$ cd pgsql
$ cvs tag -b REL6_4
   </programlisting>

   which will create the tag and the branch for the RELEASE tree.
  </para>

  <para>
   For those with <productname>CVS</productname> access, it's simple to
   create directories for different versions.
   First, create two subdirectories, RELEASE and CURRENT, so that you don't
   mix up the two.  Then do:

   <programlisting>
cd RELEASE
cvs checkout -P -r REL6_4 pgsql
cd ../CURRENT
cvs checkout -P pgsql
   </programlisting>

   which results in two directory trees, <filename>RELEASE/pgsql</filename> and
   <filename>CURRENT/pgsql</filename>. From that point on,
   <productname>CVS</productname>
   will keep track of which repository branch is in which directory tree, and will
   allow independent updates of either tree.
  </para>

  <para>
   If you are <emphasis>only</emphasis> working on the <literal>CURRENT</literal>
   source tree, you just do
   everything as before we started tagging release branches.
  </para>

  <para>
   After you've done the initial checkout on a branch

   <programlisting>
$ cvs checkout -r REL6_4
   </programlisting>

   anything you do within that directory structure is restricted to that
   branch.  If you apply a patch to that directory structure and do a

   <programlisting>
cvs commit
   </programlisting>

   while inside of it, the patch is applied to the branch and
   <emphasis>only</emphasis> the branch.
  </para>
 </sect1>

 <sect1 id="cvsup">
  <title>Getting The Source Via <productname>CVSup</productname></title>

  <para>
   An alternative to using anonymous CVS for retrieving
   the <productname>PostgreSQL</productname> source tree
   is <productname>CVSup</productname>.
   <productname>CVSup</productname> was developed by
   John Polstra (<email>jdp@polstra.com</email>) to
   distribute CVS repositories and other file trees for
   <ulink url="http://www.freebsd.org">the FreeBSD project</ulink>.
  </para>

  <para>
   A major advantage to using
   <productname>CVSup</productname> is that it can reliably
   replicate the <emphasis>entire</emphasis> CVS repository on your local system,
   allowing fast local access to cvs operations such as <option>log</option>
   and <option>diff</option>. Other advantages include fast synchronization to
   the <productname>PostgreSQL</productname> server due to an efficient
   streaming transfer protocol which only sends the changes since the last update.
  </para>

  <sect2>
   <title>Preparing A <productname>CVSup</productname> Client System</title>

   <para>
    Two directory areas are required for <productname>CVSup</productname>
    to do it's job: a local <productname>CVS</productname> repository
    (or simply a directory area if you are fetching a snapshot rather
    than a repository; see below)
    and a local <productname>CVSup</productname> bookkeeping
    area. These can coexist in the same directory tree.
   </para>

   <para>
    Decide where you want to keep your local copy of the
    <productname>CVS</productname> repository. On one of our systems we
    recently set up a repository in <filename>/home/cvs/</filename>,
    but had formerly kept it under a
    <productname>PostgreSQL</productname> development tree in
    <filename>/opt/postgres/cvs/</filename>. If you intend to keep your
    repository in <filename>/home/cvs/</filename>, then put

    <programlisting>
setenv CVSROOT /home/cvs
    </programlisting>

    in your <filename>.cshrc</filename> file, or a similar line in
    your <filename>.bashrc</filename> or
    <filename>.profile</filename> file, depending on your shell.
   </para>

   <para>
    The <application>cvs</application> repository area must be initialized.
    Once <envar>CVSROOT</envar> is set, then this can be done with a
    single command:

    <programlisting>
$ cvs init
    </programlisting>

    after which you should see at least a directory named
    <filename>CVSROOT</filename> when listing the
    <envar>CVSROOT</envar> directory:

    <programlisting>
$ ls $CVSROOT
CVSROOT/
    </programlisting>
   </para>
  </sect2>

  <sect2>
   <title>Running a <productname>CVSup</productname> Client</title>

   <para>
    Verify that
    <application>cvsup</application> is in your path; on most systems
    you can do this by typing

    <programlisting>
which cvsup
    </programlisting>

    Then, simply run
    <application>cvsup</application> using:

    <programlisting>
$ cvsup -L 2 <replaceable class="parameter">postgres.cvsup</replaceable>
    </programlisting>

    where <option>-L 2</option> enables some status messages so you
    can monitor the progress of the update,
    and <replaceable class="parameter">postgres.cvsup</replaceable> is
    the path and name you have given to your
    <productname>CVSup</productname> configuration file.
   </para>

   <para>
    Here is a <productname>CVSup</productname> configuration file
    modified for a specific installation, and which maintains a full
    local <productname>CVS</productname> repository:

    <programlisting>
# This file represents the standard CVSup distribution file
# for the <productname>PostgreSQL</> ORDBMS project
# Modified by lockhart@fourpalms.org 1997-08-28
# - Point to my local snapshot source tree
# - Pull the full CVS repository, not just the latest snapshot
#
# Defaults that apply to all the collections
*default host=cvsup.postgresql.org
*default compress
*default release=cvs
*default delete use-rel-suffix
# enable the following line to get the latest snapshot
#*default tag=.
# enable the following line to get whatever was specified above or by default
# at the date specified below
#*default date=97.08.29.00.00.00

# base directory where CVSup will store its 'bookmarks' file(s)
# will create subdirectory sup/
#*default base=/opt/postgres # /usr/local/pgsql
*default base=/home/cvs

# prefix directory where CVSup will store the actual distribution(s)
*default prefix=/home/cvs

# complete distribution, including all below
pgsql

# individual distributions vs 'the whole thing'
# pgsql-doc
# pgsql-perl5
# pgsql-src

   </programlisting>
   </para>

   <para>
    The following is a suggested <productname>CVSup</productname> config file from
    <ulink url="ftp://ftp.postgresql.org/pub/CVSup/README.cvsup">the <productname>PostgreSQL</> ftp site</ulink>
    which will fetch the current snapshot only:

    <programlisting>
# This file represents the standard CVSup distribution file
# for the <productname>PostgreSQL</> ORDBMS project
#
# Defaults that apply to all the collections
*default host=cvsup.postgresql.org
*default compress
*default release=cvs
*default delete use-rel-suffix
*default tag=.

# base directory where CVSup will store its 'bookmarks' file(s)
*default base=<replaceable class="parameter">/usr/local/pgsql</replaceable>

# prefix directory where CVSup will store the actual distribution(s)
*default prefix=<replaceable class="parameter">/usr/local/pgsql</replaceable>

# complete distribution, including all below
pgsql

# individual distributions vs 'the whole thing'
# pgsql-doc
# pgsql-perl5
# pgsql-src

    </programlisting>
   </para>
  </sect2>

  <sect2>
   <title>Installing <productname>CVSup</productname></title>

   <para>
    <productname>CVSup</productname> is available as source, pre-built
    binaries, or Linux RPMs. It is far easier to use a binary than to
    build from source, primarily because the very capable, but
    voluminous, Modula-3 compiler is required for the build.
   </para>

   <procedure>
    <title><productname>CVSup</productname> Installation from Binaries</title>

    <para>
     You can use pre-built binaries
     if you have a platform for which binaries
     are posted on
     <ulink url="ftp://ftp.postgresql.org/pub">the <productname>PostgreSQL</productname> ftp site</ulink>,
     or if you are running FreeBSD, for which
     <productname>CVSup</productname> is available as a port.

     <note>
      <para>
       <productname>CVSup</productname> was originally developed as a
       tool for distributing the <productname>FreeBSD</productname>
       source tree. It is available as a <quote>port</quote>, and for those running
       FreeBSD, if this is not sufficient to tell how to obtain and
       install it then please contribute a procedure here.
      </para>
     </note>
    </para>

    <para>
     At the time of writing, binaries are available for
     Alpha/Tru64, ix86/xBSD,
     HPPA/HP-UX 10.20, MIPS/IRIX,
     ix86/linux-libc5, ix86/linux-glibc,
     Sparc/Solaris, and Sparc/SunOS.
    </para>

    <step>
     <para>
      Retrieve the binary tar file for
      <application>cvsup</application>
      (<application>cvsupd</application> is not required
      to be a client) appropriate for your platform.
     </para>

     <substeps>
      <step performance="optional">
       <para>
	If you are running FreeBSD, install the <productname>CVSup</productname> port.
       </para>
      </step>

      <step performance="optional">
       <para>
	If you have another platform, check for and download the appropriate binary from
	<ulink url="ftp://ftp.postgresql.org/pub">the <productname>PostgreSQL</productname> ftp site</ulink>.
       </para>
      </step>
     </substeps>
    </step>

    <step>
     <para>
      Check the tar file to verify the contents and directory
      structure, if any. For the linux tar file at least, the static binary
      and man page is included without any directory packaging.
     </para>

     <substeps>
      <step>
       <para>
	If the binary is in the top level of the tar file, then simply
	unpack the tar file into your target directory:

	<programlisting>
$ cd /usr/local/bin
$ tar zxvf /usr/local/src/cvsup-16.0-linux-i386.tar.gz
$ mv cvsup.1 ../doc/man/man1/
	</programlisting>
       </para>
      </step>

      <step>
       <para>
	If there is a directory structure in the tar file, then unpack
	the tar file within /usr/local/src and move the binaries into
	the appropriate location as above.
       </para>
      </step>
     </substeps>
    </step>

    <step>
     <para>
      Ensure that the new binaries are in your path.

      <programlisting>
$ rehash
$ which cvsup
$ set path=(<replaceable>path to cvsup</replaceable> $path)
$ which cvsup
/usr/local/bin/cvsup
      </programlisting>
     </para>
    </step>
   </procedure>
  </sect2>

  <sect2>
   <title>Installation from Sources</title>

   <para>
    Installing <productname>CVSup</productname> from sources is not
    entirely trivial, primarily because most systems will need to
    install a Modula-3 compiler first.
    This compiler is available as Linux <productname>RPM</productname>,
    FreeBSD package, or source code.

    <note>
     <para>
      A clean-source installation of Modula-3 takes roughly 200MB of disk space,
      which shrinks to roughly 50MB of space when the sources are removed.</para>
    </note>
   </para>

   <procedure>
    <title>Linux installation</title>

    <step>
     <para>
      Install Modula-3.
     </para>

     <substeps>
      <step>
       <para>
	Pick up the <productname>Modula-3</productname>
	distribution from
	<ulink url="http://m3.polymtl.ca/m3">Polytechnique Montréal</ulink>,
	who are actively maintaining the code base originally developed by
	<ulink
	 url="http://www.research.digital.com/SRC/modula-3/html/home.html">the DEC Systems Research Center</ulink>.
       The <productname>PM3</productname> <productname>RPM</productname> distribution is roughly
	30MB compressed. At the time of writing, the 1.1.10-1 release
	installed cleanly on RH-5.2, whereas the 1.1.11-1 release is
	apparently built for another release (RH-6.0?) and does not run on RH-5.2.

	<tip>
	 <para>
	  This particular rpm packaging has
	  <emphasis>many</emphasis> <productname>RPM</productname> files,
	  so you will likely want to place them into a separate
	  directory.
	 </para>
	</tip>
       </para>
      </step>

      <step>
       <para>
	Install the Modula-3 rpms:

	<programlisting>
# rpm -Uvh pm3*.rpm
	</programlisting>
       </para>
      </step>
     </substeps>
    </step>

    <step>
     <para>
     Unpack the cvsup distribution:

      <programlisting>
# cd /usr/local/src
# tar zxf cvsup-16.0.tar.gz
      </programlisting>
     </para>
    </step>

    <step>
     <para>
      Build the cvsup distribution, suppressing the GUI interface
      feature to avoid requiring X11 libraries:

      <programlisting>
# make M3FLAGS="-DNOGUI"
      </programlisting>

      and if you want to build a static binary to move to systems
      that may not have Modula-3 installed, try:

      <programlisting>
# make M3FLAGS="-DNOGUI -DSTATIC"
      </programlisting>
     </para>
    </step>

    <step>
     <para>
      Install the built binary:

      <programlisting>
# make M3FLAGS="-DNOGUI -DSTATIC" install
      </programlisting>
     </para>
    </step>
   </procedure>
  </sect2>
 </sect1>
</appendix>

<!--
> It became clear that I had a problem with my m3 installation; some
> X11 libraries were not being found correctly.

By the way, you can build the client without the GUI by doing this
in the "client" subdirectory:

    m3build -DNOGUI

If you build it that way, then it doesn't need the X11 libraries and
it's quite a bit smaller.  The GUI is fun to watch, but it's not
very useful.  I originally implemented it because it made debugging
the multi-threaded client program much easier.

To build a statically-linked client, edit <filename>client/src/m3makefile</filename>
to add <literal>build_standalone()</literal>
 just before the <literal>program()</literal> entry near
the end of the file:

<programlisting>
build_standalone()
program(cvsup)
</programlisting>

Then, if cvsup has already been built, remove the machine-specific build directory
(e.g. <filename>LINUXELF/</filename>) and rebuild:

<programlisting>
rm -rf LINUXELF
m3build -DNOGUI -v
cp -p LINUXELF/cvsup /usr/local/bin
</programlisting>

> Anyway, with the reinstall and the two-line patch above (and that
> one include-file _POSIX_SOURCE workaround from the previous try),
> things built cleanly.

Good!

> Now I just need a server somewhere to test.

If you want to try it out, there are public servers for the FreeBSD
source repository at cvsup.freebsd.org and cvsup2.freebsd.org.
Here's a suggested supfile:

*default host=cvsup.freebsd.org compress
*default release=cvs
*default base=/home/jdp/cvsup-test	# FIX THIS
*default delete use-rel-suffix
# *default tag=.
src-bin

This will fetch you the source repository for the programs that get
installed into "/bin".  I chose it because it's one of the smaller
pieces of the system.  Make an empty directory someplace for
testing, and change the "FIX THIS" line to specify that directory
after the "base=".

If you are on a T1 or better, you should probably delete the
"compress" keyword in the first line.

As shown, it will get the repository (RCS) files.  If you uncomment
the line containing "tag=." then it will instead check out the
latest version of each file.  There's a bunch more information about
what you can do in
<ulink url="http://www.freebsd.org/handbook/cvsup.html">the CVSup Handbook</ulink>.

There is one other thing I want to send you, but not tonight.  I
discovered the hard way that you need a malloc package that is
thread-safe with respect to the Modula-3 threads package.  The
Modula-3 runtime takes care to do the proper mutual exclusion around
all calls it makes to malloc.  But if you call certain functions in
the native C library which in turn call malloc, then the mutual
exclusion gets bypassed.  This can lead to rare but baffling core
dumps.

For FreeBSD, I solved this by adding a thread-safe malloc package
into the Modula-3 runtime.  The package is quite portable, and I'm
sure it will work well for Linux with very few changes (probably
none at all).  I want to send it to you along with instructions
for making it a part of the "libm3core" library.  It's very simple,
but I've run out of steam for tonight. :-)  Once you have this
malloc in place, the CVSup system should be rock solid.  We have
servers that have been up for weeks and have served many thousands
of clients without any observed problems.

> We hope to have the PostgreSQL tree using CVSup within a month or
> so, and hope to retire sup in September...

Great!  I'll do my best to help make sure you don't regret it.

John

Tom,

I'm appending the sources for the thread safe version of malloc that
I told you about.  I believe that it will simply compile and work
under Linux, but I've never had an opportunity to test it there.
I urge you to put it into your Modula-3 system; otherwise, you
are guaranteed to get occasional mysterious core dumps from cvsupd.

As a first step, I'd suggest simply trying to compile it under
Linux, like this:

    cc -O -c malloc.c

You shouldn't get any errors or warnings.  If you do, contact me
before you waste any more time on it.

Assuming it compiles OK, copy malloc.c into this directory of your
Modula-3 source tree:

    m3/m3core/src/runtime/LINUXELF

In that same directory, edit "m3makefile" and add this line to the
end of the file:

    c_source       ("malloc")

Then chdir up into "m3/m3core" of the Modula-3 tree and type
"m3build".  (I'm assuming you already have a working Modula-3
installation.)  After that finishes, become root and type "m3ship"
to install it.

That's all there is to it.  If you built cvsup and cvsupd to use
shared libraries, you don't even need to re-link them.  They'll pick
up the change automatically from the updated shared library.

Let me know if you run into any problems with it.

By the way, this is a very good malloc in its own right.  It's worth
using even aside from the thread safety of it.

Regards,
John

I've deposited a statically built cvsup client executable (and man pages
and test configuration) in

  /pub/incoming/cvsup-15.1-client-linux.tar.gz

This was built and linked on Linux/v2.0.30, RH/v4.2, gnulib/v5.3.12 and
includes the thread-safe malloc provided by John Polstra. I'll forward
to you the malloc code and an additional installation e-mail from John.

The Modula-3 installation takes a good bit of room (~50MB?) and the
build environment is unique to Modula-3, but suprisingly enough it
pretty much works.

The cvsup Makefiles do not work on my machine (they are not portable
yet) but each major package (there are 4) can be built without needing
the makefiles with two commands each. Not difficult at all. John gives
some hints in his e-mail on how to build a static executable, and on how
to shrink the size of the executable by leaving out the GUI support.
Again, easy to do.

My client test case, picking up a sub-tree of the FreeBSD distribution,
worked flawlessly. I haven't tried running a server.

Thanks to John for getting me going.

			- Tom


For the thread-safe malloc, do the following:
1) install Modula-3
2) add the enclosed file "malloc.c" to m3/m3core/src/runtime/LINUXELF
3) edit the last line of m3makefile in the same directory to add
     c_source       ("malloc")
4) do an "m3build" and an m3ship from the appropriate directory.

From what John said, the malloc problem can be noticable for the
server-side running cvsupd. Clients may not need it.

Unfortunately I seem to have lost John's original good instructions for
this, so am doing this from memory. May need to ask John to give
instructions again...

			- Tom

-->


<!-- Keep this comment at the end of the file
Local variables:
mode:sgml
sgml-omittag:nil
sgml-shorttag:t
sgml-minimize-attributes:nil
sgml-always-quote-attributes:t
sgml-indent-step:1
sgml-indent-data:t
sgml-parent-document:nil
sgml-default-dtd-file:"./reference.ced"
sgml-exposed-tags:nil
sgml-local-catalogs:("/usr/lib/sgml/catalog")
sgml-local-ecat-files:nil
End:
-->