5. Incorporating the updates

To incorporate the updates, you need write access to the distribution directory from a Linux machine, with a working version of rpm installed. There are three steps involved:

1. Correct the file protection modes.
2. Replace updated RPMs.
3. Generate the hdlist file

If you maintain a mirror of the updates directory, you can at any time produce a CD including the current updates by repeating these steps.

5.1 Correcting the file protection modes

During the installation process, some programs are run directly off the CD. Unfortunately, the FTP program does not always preserve the protection modes of the files and directories that are copied. Therefore, it is necessary to make sure that execute permission is given to programs, shell scripts and shared libraries, before the directory is burned on the CD. This is done by running the updatePerm script on your local copy of the distribution:

#!/bin/bash

RHVERSION=6.1

LIST=/tmp/er3hd3w25
RHROOT=/jaz/redhat-${RHVERSION}

# Find all directories, and make sure they have +x permission
find $RHROOT -type d -exec chmod -c 755 {} \;

# Find all files that are executables, shell, python or perl scripts
find $RHROOT -type f | file -f - | grep -v RPM \
   | egrep -i 'executable|perl|bourne|shell|python' | cut -f1 -d: > $LIST
#   ====>     NEEDED FOR RedHat 6.1        ^^^^^^^

# Find shared libraries
find $RHROOT -name \*.so >> $LIST

# Make them executable
while read file
do
   if [ ! -x $file ] ; then
      chmod -c 755 $file
   fi
done < $LIST

/bin/rm $LIST

exit 0

5.2 Replacing the updated RPMS

The following script called updateCD copies all files from the update directory to the RPMS directory. The script uses some nifty rpm tricks to determine what packages in the updates directory are more recent. Older packages are moved to the ${OLD} directory.

#! /bin/bash
# This script updates rpms in a RedHat distribution found in $RPMDIR.
# The old rpms will be placed in $OLDDIR.
# The new rpms should be located in $UPDDIR.
# The new images are in $IMGDIR
# The images to be updated are in $OMGDIR 
# The architechture is $ARCH.

RHVERSION=6.1
ARCH=i386

RHROOT=/jaz/redhat-${RHVERSION}
RPMDIR=${RHROOT}/${ARCH}/RedHat/RPMS
UPDDIR=${RHROOT}/updates/${ARCH}
IMGDIR=${RHROOT}/updates/images/${ARCH}
OMGDIR=${RHROOT}/${ARCH}/images
OLDDIR=${RHROOT}/old

if [ ! -d $OLDDIR ] ; then
   echo making directory $OLDDIR
   mkdir $OLDDIR
fi

allow_null_glob_expansion=1

for rpm in ${UPDDIR}/*.rpm ; do
  NAME=`rpm --queryformat "%{NAME}" -qp $rpm`
  unset OLDNAME
  for oldrpm in ${RPMDIR}/${NAME}*.rpm ; do
    if [ `rpm --queryformat "%{NAME}" -qp $oldrpm` = "$NAME" ]; then
      OLDNAME=$oldrpm;
      break
    fi
  done
  if [ -z "$OLDNAME" ]; then 
    echo $NAME is new
    cp -pv $rpm $RPMDIR
  else
    if [ `basename $rpm` != `basename $OLDNAME` ]; then
      mv $OLDNAME $OLDDIR
      cp -pv $rpm $RPMDIR
    fi
  fi
done


# Copy new boot image files to the right place...
for newfile in ${IMGDIR}/* ; do
  file=${OMGDIR}/$(basename ${newfile})
  if [ $newfile -nt $file ] ; then 
     cp -pv $newfile $file
  fi
done

exit 0

Important note for RedHat 6.0, 6.1

Certain RPMs, specifically the kernel and kernel-smp packages, include the platform in the filename but not in the package name. For example, the "kernel" package comes in several flavors:

 
kernel-2.2.5-22.i386.rpm
kernel-2.2.5-22.i586.rpm
kernel-2.2.5-22.i686.rpm

but for all three rpm -qp returns just "kernel" for the package name.

As you can see, this "outsmarts" the updateCD script. The result is that only the last one gets properly copied. The first two get copied, but are then moved to the $OLD directory! Preferably, RedHat should name these differently. But for now, the easy solution is to move the packages by hand, after running updateCD. (Thanks to Kyle B. Ferrio <kyle@U.Arizona.EDU>)

Joshua Sarro <mthed@shore.net> has contributed a perl script called updateMirror.pl which can deal with the situation. You can fetch it here: http://imsb.au.dk/~mok/linux/doc/updateMirror.pl.

5.3 Generating a new hdlist file

When installing from the CD, the installation program on the CD relies on the file RedHat/base/hdlist describing what RPM packages are available on the CD. The hdlist file can be generated by the program misc/src/install/genhdlist. This program must be run with the absolute path to the root of the distribution as the only argument. Here is the updateHdlist script which calls that program:

#!/bin/bash

RHVERSION=6.1
ARCH=i386

echo generating hdlist...
RHROOT=/jaz/redhat-${RHVERSION}
GENHDDIR=${RHROOT}/${ARCH}/misc/src/install

chmod u+x ${GENHDDIR}/genhdlist
chmod 644 ${RHROOT}/${ARCH}/RedHat/base/hdlist
${GENHDDIR}/genhdlist ${RHROOT}/${ARCH} || echo "*** GENHDLIST FAILED ***"

exit 0

NOTE: After having incorporated the updates in the main RedHat/RPMS directory, your copy of the distribution is no longer a mirror of the Red Hat distribution site. Actually, it is more up-to-date! Therefore, if you attempt to mirror the distribution, older versions of the RPM's that have been updated will be downloaded once more, and the updates deleted.

Important note for RedHat 6.1

The installation in RedHat 6.1 is completely changed from earlier versions, and RedHat have introduced a system called ``anaconda''. The genhdlist program is now found in a different place, so in the script above, use

GENHDDIR=${RHROOT}/${ARCH}/misc/src/anaconda/utils

The updatePerm script must be changed to include ``python'' in line 13. This has been done in the listing above, but if you've cut the script from earlier versions of this document (before 1.30), you need to make the change!

In some cases, genhdlist fails to run, because the executable is not statically linked. In such a case, you can add a new line ${RHROOT}/${ARCH}/RedHat/instimage/usr/lib in /etc/ld.so.conf and run ldconfig -v.

Another solution is to recompile genhdlist. The following modification to the updateHdlist script worked under RedHat 5.2:

#!/bin/bash

RHVERSION=6.1
ARCH=i386

RHROOT=/misc/redhat/redhat-${RHVERSION}
GENHDDIR=${RHROOT}/${ARCH}/misc/src/anaconda/utils

echo Compiling genhdlist...
sed -e 's/FD_t/int/' \
    -e 's/fdOpen/open/' \
    -e 's/fdClose/close/' \
    -e 's/fdFileno//' < ${GENHDDIR}/genhdlist.c > /tmp/genhdlist.c
cc -o /tmp/genhdlist -I/usr/include/rpm /tmp/genhdlist.c -lrpm -lz

echo generating hdlist...
chmod 644 ${RHROOT}/${ARCH}/RedHat/base/hdlist
/tmp/genhdlist ${RHROOT}/${ARCH} || echo "*** GENHDLIST FAILED ***"

exit 0

In this version of the script, a copy of the C source of genhdlist.c is piped through sed to create a copy in /tmp that will compile under RedHat 5.2. This version of genhdlist is then used to create the hdlist file

Important note for RedHat 5.2

As distributed with RedHat version 5.2 and earlier, genhdlist CRASHES if there are files in the RedHat/RPMS directory which are not RPM files! This causes problems, because in the 5.2 distribution, there are a couple of non-RPM files named ls-lR and ls-lR.gz in RedHat/RPMS. Therefore, you must remove all non-RPM files from the directory. Alternatively, you can apply the following patch to misc/src/install/genhdlist.c and do a make. The patch will cause genhdlist to ignore any non-RPM files.

*** genhdlist.c.orig    Fri Nov 27 12:08:13 1998
--- genhdlist.c Fri Nov 27 12:08:20 1998
***************
*** 12,23 ****
--- 12,26 ----
  
  #define FILENAME_TAG 1000000
  
+ /* Not used apparently...
+ 
  int tags[] =  { RPMTAG_NAME, RPMTAG_VERSION, RPMTAG_RELEASE, RPMTAG_SERIAL,
                RPMTAG_FILENAMES, RPMTAG_FILESIZES, RPMTAG_GROUP,
                RPMTAG_REQUIREFLAGS, RPMTAG_REQUIRENAME, RPMTAG_REQUIREVERSION,
                RPMTAG_DESCRIPTION, RPMTAG_SUMMARY, RPMTAG_PROVIDES,
                RPMTAG_SIZE, RPMTAG_OBSOLETES };
  int numTags = sizeof(tags) / sizeof(int);
+ */
  
  int main(int argc, char ** argv) {
      char buf[300];
***************
*** 26,34 ****
--- 29,39 ----
      struct dirent * ent;
      int fd, rc, isSource;
      Header h;
+     /* not used 
      int count, type;
      int i;
      void * ptr;
+     */
  
      if (argc != 2) {
        fprintf(stderr, "usage: genhdlist <dir>\n");
***************
*** 74,79 ****
--- 79,85 ----
  
            rc = rpmReadPackageHeader(fd, &h, &isSource, NULL, NULL);
  
+           if (!rc) {
            headerRemoveEntry(h, RPMTAG_POSTIN);
            headerRemoveEntry(h, RPMTAG_POSTUN);
            headerRemoveEntry(h, RPMTAG_PREIN);
***************
*** 110,115 ****
--- 116,122 ----
            headerWrite(outfd, h, HEADER_MAGIC_YES);
            headerFree(h);
            close(fd);
+           }
        }
  
        errno = 0;

5.4 The comps file

The comps file defines how the packages are bundled during the installation. In the Red Hat distribution, this is done according to the functionality they provide, for example:

• Printer Support
• X Window System
• GNOME
• KDE
• Mail/WWW/News Tools
• ...
• Kernel Development
• Extra Documentation

Sometime during the installation process, the user is presented with a dialog called "Components to install". Some of the components have been preselected, and others not. The last item on the components list is called "Everything". According to the Red Hat documentation, selecting every package will require close to 1 Gb of free disk space.

On the dialog box, there also is an option that enables the user to customize exactly what packages will be installed. Customizing the installation by hand, or selecting "Everything" in the components list is the only way to have your own packages installed unless you modify the RedHat/base/comps file.

Format of comps file in RedHat versions < 6.1

The format of the comps file currently starts with a header describing the version of the comps format, followed by an empty line.

0.1
<empty line>

After this, the components are listed, separated by empty lines:

<component 1>
<empty line>
<component 2>
<empty line>
.
.
<component n>
<empty line>
EOF

Each component has the following definition:

(0|1) (--hide)? <name>
<RPM 1>
<RPM 2>
...
<RPM n>
end

Before the name of each component, 0 or 1 is given. A value of 1 here means that the component is chosen by default, and 0 means it's not. The option "--hide" means that you will not see the entry, unless you choose "expert" installation. The first component is called "Base", and that is special, in the sense that it must be present and it does not show up in the dialog (you can't deselect the base installation, which makes sense...)

Next follows a list of rpm packages belonging to that component. Note that this is the package name stored in the rpm file, and not any part of the file name of the package (although it is often the same).

By adding your packages to the comps file, you can customize your own distribution, and make sure that your packages will be installed by default. One thing to be careful about is interdependence among your packages, but here, you are on your own :-) A word of warning: be careful not to add or remove extra whitespace in the file. Examine the existing comps file (make a copy of the original) to see how it's done (or check i386/misc/src/install/pkgs.c if you want to see how the file is parsed).

Format of comps file in RedHat version 6.1

With RedHat version 6.1, the format of the comps file has changed. We have not yet disected it to discover the file format. With the above information, however, it should be easy enough to do it yourself. The decoding takes place in ${RHROOT}/${ARCH}/misc/src/anaconda/comps.py.