My thoughts on Slackware, life and everything

Tag: mirror

My new mirror

I have a new mirror server.

I have configured a VPS (a virtually hosted server using QEMU) which was donated to me by a Slackware supporter who wishes to remain anonymous. The physical server is on a gigabit Internet connection, so I guess I can offer a speedy  mirror service!

In fact, the mirrors are already complete. With a re-sync of several times a day, I hope to offer an uptodate service. I am mirroring the following:

  • My own complete package repository and tools – including my “restricted SlackBuilds” area, containing things like lame, libdvdcss etc.
  • My “Ktown” repository of KDE packages – where I made available new packages for KDE 4.5.1 earlier today.
  • The full SlackBuilds.org repository.
  • Slackware-current – both 32-bit and 64-bit architectures will be mirrorred.

Plus, I am using my mirror-slackware-current.sh script to create bootable DVD ISO images of slackware-current (32-bit and 64-bit). These ISO images will be re-generated every time there is an update to the -current ChangeLog.txt.
These ISO images do not contain source code. This allows me to keep the ISO file size below 2 GB which will make it a worthwhile download for you.

And most important – mirror access:

Hammer it good!

Cheers, Eric

Local Slackware mirror

You will not be surprised if I tell you that I install Slackware a lot.  Inbetween the public updates to slackware-current which you can read in the ChangeLog.txt there is usually a lot of private testing.

For that purpose I keep a local mirror of the slackware-current tree.  I have written a script that keeps this local copy synchronized, and also creates the ISO images which I use for the installs.  The script is widely used by other people as well, because there are no official “slackware-current” ISO images available for download.  It is much more economical to create your own ISO images when you already have a local mirror of all the packages.

The script is appropriately called “mirror-slackware-current.sh“. If you want to know what it can do, just run it with the “-h” parameter:

$ /usr/bin/mirror-slackware-current.sh -h
-----------------------------------------------------------------
$Id: mirror-slackware-current.sh,v 1.70 2009/04/14 10:24:48 root Exp root $
-----------------------------------------------------------------
Usage:
  /usr/bin/mirror-slackware-current.sh [OPTION] ...
or:
  SLACKROOTDIR=/your/repository/dir /usr/local/sbin/mirror-slackware-current.sh [OPTION] ...

The SLACKROOTDIR is the directory that contains the directories
  slackware-<RELEASE> and slackware-<RELEASE>-iso
Current value of SLACKROOTDIR : /mirrors

You can change the script defaults in a file 'mirror-slackware-current.conf'

The script's parameters are:
  -h            This help.
  -b <number>   Limit bandwidth usage to <number> KBytes/sec.
  -c            Check for newer version of this script.
  -e            Use 'boot-load-size=32' instead of the value 4.
                 (32 is a more standard value, but a value of 4
                 will let the ISO boot with old 'broken' BIOSes).
  -f            Force sync and the creation of new ISO image(s)
                even if no update of the ChangeLog.txt was found.
                This is how you resume after an aborted attempt.
                Note: this will also create any missing local
                      directories needed for the mirror.
  -i            Only generate ISO images from our local copy;
                do not attempt to contact the remote server.
  -l <localdir> The root directory where you keep your local
                Slackware mirror; this directory contains
                slackware-<RELEASE> and slackware-<RELEASE>-iso
  -m <uri>      The rsync URI that you want to use instead of
                the script default. Example:
                -m mirrors.tuxq.com::slackware/slackware-current
                (no trailing slash!)
  -n            Only show the changes in the ChangeLog.txt
                but don't sync anything and don't generate ISOs.
  -o <iso_type> The type of ISO that you want to generate.
                iso_type can be one of:
                CDROM: produce 3 CDROM images (KDE on CD3);
                DVD  : produce a single DVD image;
                ALL  : produce CDROM and DVD images;
                NONE : produce no images at all (just sync).
                The default iso_type is CDROM.
  -p            Remove old ISO's before building the new ones
                (in case you're suffering from low free space).
  -r <release>  The release ('current' by default); use '-r 12.2'
                if you want to mirror and image slackware 12.2
  -q            Non-verbose output (for cron jobs).
  -s            Additional ssh options, in case rsync needs to
                login to the remote server using ssh. Example:
                -s "-l alien -o IdentityFile=/home/alien/.ssh/id_rsa"
  -v            Verbose progress indications.
  -w            Write a .conf file containing script defaults.
                It will be created in the script's directory,
                as 'mirror-slackware-current.conf'
  -X <xfile>    File 'xfile' contains a list of exclude patterns
                for directories that you do not want mirrored.
                Note: this will override the default exclusion of
                the 'pasture' directory so if you still want that
                excluded, add it explicitly to the file 'xfile'.
                If your intention is *not* to exclude '/pasture'
                from the mirror, use '-X none'.

An example: I have defined a cronjob which checks a Slackware mirror server once a day, and if the script detects a change in the ChangeLog.txt it will bring my local mirror back in sync, and creates a bootable DVD ISO image for me. If no changes are found, the script does not produce any output, but in case of an update it will show it’s actions on standard output. The cron daemon will mail the output of any job to the user account which runs the script, so in the morning I will find a comforting message in my inbox:

Date: Sat, 28 Mar 2009 15:44:37 +0100
From: root@darkstar
To: root@darkstar
Subject: cron: /usr/bin/mirror-slackware-current.sh -q -X none -o DVD

0a1,17
> Fri Mar 27 20:30:48 CDT 2009
> a/module-init-tools-3.6-i486-2.tgz:  Patched modprobe to not complain if
/sys
>   is not mounted.  This is none of modprobe's business.  ;-)
> l/glib2-2.18.4-i486-1.tgz:  Upgraded to glib-2.18.4 and patched.
>   This library has been patched to fix overflows that are
>   possible security holes.
>   For more information, see:
>     http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-4316
>   (* Security fix *)
> l/libxml2-2.7.3-i486-1.tgz:  Upgraded to libxml2-2.7.3.
> xap/mozilla-firefox-3.0.8-i686-1.tgz:
>   Upgraded to firefox-3.0.8.
>   This fixes some security issues.
>   For more information, see:
>     http://www.mozilla.org/security/known-vulnerabilities/firefox30.html
>   (* Security fix *)
> +--------------------------+
Sat Mar 28 15:22:06 CET 2009 [17616]: ChangeLog.txt has been updated,
starting mirror of slackware-current.
*** Using rsync.osuosl.org::slackware/slackware-current ***
Sat Mar 28 15:24:09 CET 2009 [17616]: Done mirroring slackware-current
 (exit code 0).
Sat Mar 28 15:28:04 CET 2009 [17616]: Creating DVD ISO image for
 slackware-current...
Sat Mar 28 15:38:27 CET 2009 [17616]: DVD ISO created (exit code 0) ...
Sat Mar 28 15:38:37 CET 2009 [17616]: Computing MD5 checksums of the ISO's
 (time consuming).
Sat Mar 28 15:44:31 CET 2009 [17616]: Resulting ISO files:
-rw-r--r--  1 root root 4479451136 Mar 28 15:38
/mirrors/slackware-current-iso/slackware-current-install-dvd.iso
Sat Mar 28 15:44:31 CET 2009 [17616]: Done!

This is the command line which I added to my crontab:

# Keep the Slackware -current tree in sync:
22 5 * * * /usr/bin/mirror-slackware-current.sh -q -X none -o DVD

The script used to create 2 CDROM ISO images if you wanted those (leaving out the kdei/ series, as well as the packages in extra/), but lately, the size of Slackware has increased so much that the packages no longer fit on two CDROMs. I have modified the script so that it will now produce 3 CDROM ISO images (with the kde/ series on the third CDROM) if you want to burn CD’s.

TIP: If you want to change the default values of the script, do not edit the script itself.  Instead run “mirror-slackware-current.sh -w” which will create the file “mirror-slackware-current.conf” in the same directory as “mirror-slackware-current.sh”. If you edit the variable values in “mirror-slackware-current.conf” the script will use those.

Have fun with this script,

Eric

© 2024 Alien Pastures

Theme by Anders NorenUp ↑