My thoughts on Slackware, life and everything

Tag: slackbuild (Page 1 of 3)

Ten years of SlackBuilds.org

sbologo This week marks the tenth anniversary of SlackBuilds.org.

Many of us will remember the time when a true Slacker did not bother herself with build scripts. The “configure && make && make install” mantra was at the forefront of everyone’s mind when it came to installing new software. Slackware made this possible. Unlike the other big distros, the Slackware package management did (and does!) not get in your way. If you want to compile your software by hand, bypassing the package management ‘database’ (which in Slackware is nothing more than a directory), then nothing or no one is stopping you.

But when you compile and upgrade a lot of software, or when you have to re-install your computer, all that hand-compiled stuff is quite tricky, if not pretty hard, to replicate unless you wrote down all that you did. And if you had been writing your compile and build steps down anyway, then it was a small step towards formalizing this documentation into an actual build script. And Slackware was (mostly) developed using build scripts, called “SlackBuilds” because of these scripts’ extension “.SlackBuild”. So, these scripts used by Slackware became gradually more common among users of Slackware. And once you had compiled a package using a build script, other advantages would become obvious: if you owned more than one computer you did not have to compile your software twice. You would transfer the package to your other computer(s) and use the Slackware ‘pkgtools’ to install or upgrade the package. Fast and deterministic.

In those days, there was a web site called “linuxpackages.net” – formerly known as “linuxmafia.org” but I guess that name had a negative vibe to it. Linuxpackages.net would offer binaries for Slackware, nicely packaged in case you were not inclined to compile stuff yourself. The site was pretty popular, but it had a major disadvantage: the packages did not have a proper quality control and it would happen ever so often that people’s computers broke after installing one of their packages. The ‘LP‘ packages were apparently built on unclean systems, introducing unknown dependencies, and the build scripts would often be absent. Good old NoobFarm has a lot of nice LP related quotes taken from the regular Slackware meeting places on the Net.

Also in those days, somewhere end of May, beginning of June 2006 (don’t have exact records of our first contact) there was a relative Slackware freshman called Robby Workman. He and his pal Erik Hanson (who initiated the GWare project, a Gnome build for Slackware after Slackware dropped that Desktop Environment from the distro) envisioned a web site where one would be able to find – not binaries, but SlackBuild scripts. These scripts would be quality-tested by a small team of people who carried trust in the community. High-quality scripts and skilled, trustworthy people at the helm, this would certainly gain some traction and be generally beneficial to the Slackware ecosystem. Robby and Erik contacted me because I knew Pat, some of my work had landed in Slackware releases, and my SlackBuilds repository (scripts plus packages) was already quite well-known but not well-organized (no proper ChangeLog.txt).

At that time I was not yet a part of the Slackware coreteam by the way – that would take until August of that same year. Robby and Erik asked me if I would be interested to join them and become part of the admin team of this new SlackBuilds.org (SBo) project they had in mind and I said “yes”. We talked this through with Pat and in particular, we discussed our scope. We wanted Pat to be sympathetic of our cause and we did not want to become a possible burden to him on the long run. For that reason, the golden rule was established that all SlackBuild entries at slackbuilds.org would follow the style of the “mother” scripts. Basically, a SBo script should be transferable into the Slackware core distro without feeling out of place. I think it was this rule that made Pat give the nod of approval. Just think of the scenario where the SBo site would become popular using a style of SlackBuild scripts that did not look at all like Slackware’s own. There was a good chance that people would start demanding that Slackware must adopt the scripting style of SBo. This was a big no-no.

In contrast, my own SlackBuild scripts do a whole lot more than a basic Slackware script – notably my scripts will download the source tarballs for you. You will not find any of those scripts on slackbuilds.org for the exact reason I just explained.

I dug through my email logs and the oldest record I still have is this, the first test email of the then already functional project:

testing_SBo

Coïncidentally this day (8 June 2006) also marks the start of formalizing my own SlackBuilds repository. I had been working on my gen_repos_files.sh script since April 2006 and used it for the first time on June 8th. The first entry in the ChangeLog.txt says:

+--------------------------+
Thu Jun  8 15:10:57 UTC 2006
Starting the ChangeLog.txt for Alien's SlackBuild repository.

Ten years have passed since those days, and SlackBuilds.org has brought a lot of good to Slackware. SBo submissions find their way into the Slackware distro core quite regularly. Which is nice, but more important is the fact that users of the distro are no longer tied to the “small” (other distros’ judgement) official repository but can tap into a vast repository of high-quality build scripts. I don’t think that anybody can concieve today of the idea that there would not be a SlackBuilds.org repository.

The fact that the SlackBuild scripts of SBo remained so basic allowed several “third party” tools to be conceived that wrap themselves around the SBo content as it were, providing queue control and automation for the process of building your own packages as well as all their dependencies. Well-known examples are sbopkg, sbotools and slpkg. The queue management feature is special. Slackware is a distro that does not concern itself with dependency management – you install the full distro, it is small enough, and that fulfills all the dependencies. Working with 3rd party packages and scripts is different though, and these tools around SBo have found ways to build and install all the required dependencies along with the package that you want to have in the first place. All of that is made possible by the SBo “info” file which is part of every submission. It contains the essential information about the software that is to be packaged, it is easily parseable and therefore ideal source material for the 3rd party tools:

PRGNAM="name of application"
VERSION="version of application"
HOMEPAGE="homepage of application"
DOWNLOAD="direct download link(s) of application source tarball(s) arch-independent or x86"
MD5SUM="md5sum(s) of the source tarball(s) defined in DOWNLOAD"
DOWNLOAD_x86_64="direct download link(s) of application source tarball(s), x86_64 only"
MD5SUM_x86_64="md5sum(s) of the source tarball(s) defined in DOWNLOAD_x86_64"
REQUIRES="%README%"
MAINTAINER="name of SlackBuild script maintainer"
EMAIL="email address of author"

I have since retired as a SBo admin, although I still hang out in the users- and admin-channels on IRC. Others have come and gone as well, and the current team is still going strong (and with Robby & Erik still at the wheel). I admire these guys a lot, because just look at the size of the repository! The package database for Slackware 11.0 (which we started with in 2006) contains 429 submissions. Compare that number to the Slackware 14.1 repository’s 5745 entries. That’s more than 13 times as many! And now imagine that this small team (the admin team has grown a bit bigger since the beginning but not much) still QA’s every script by compiling it and checking the outcome. On top of that, the team has to test all these scripts against the upcoming Slackware 14.2 release and make sure that there will be a high-quality SBO repository for Slackware 14.2 when that gets released. The project is using GIT to manage this transition process. Learning about and understanding git is one of the most important things that I took from the SBo project.

These guys need your praise and support. Go tell them in the slackbuilds-users mailing list!

 

Speeding up my vlc.SlackBuild script

In a comment to my previous post about the VLC media player where I announced packages for VLC 2.0.0, Nille asked if I could explain a bit more about the SlackBuild script I use to create these packages.

In my scripts, I sometimes hide interesting side-tracks or shortcuts which you would not notice if you just run the script and wait patiently until the packages have been created.

The VLC SlackBuild script is huge. Apart from VLC sources, it also compiles over 50 (fifty!) other libraries and embeds them into the VLC package. This makes VLC such a versatile media player – it supports many media formats and input sources, has audio visualisation inputs and other cool stuff.

The penalty for having such a big script, is that compiling it all takes quite a bit of time. And if you just want to update VLC to a newer snapshot, it is aggravating when the script also recompiles all of the other internal dependencies. Furthermore, in order to compile the browser plugin, the script needs Mozilla’s xulrunner libraries. Since Slackware does not ship those, the xulrunner SDK has to be compiled as well and that is time-consuming- only to be thrown away after the compilation has ended.

I added a few pieces of code which allow me to considerably shorten the compilation time of my VLC packages.

  1. Allow the use of a tarball containing a pre-built xulrunner SDK
  2. Allow the use of a tarball containing prebuilt “contribs”, i.e. all the internal libraries VLC depends on

Let me explain both.

Pre-compiled mozilla SDK.

If you run the vlc.SlackBuild, and enable the creation of the browser plugin “npapi-vlc” (this is enabled by default in the script) then the SlackBuild script will first compile the xulrunner SDK. This takes a long, long time. So, in order to speed things up, the script will look for a pre-compiled version of the xulrunner SDK in the directory containing “vlc.SlackBuild” and use that instead if available. You can find this SDK tarball after the xulrunner compilation has ended; check the “./dist/sdk” subdirectory in the xulrunner source directory (normally that would be something like “/tmp/build/tmp-vlc/mozilla-<someversion>/dist/sdk/“) and copy the file “xulrunner-*.sdk.tar.bz2” you’ll find there back into the directory containing vlc.SlackBuild. Next time you run vlc.SlackBuild, it will extract and use the contents of that SDK file instead of compiling the SDK all over again. Big timesaver!

Pre-compiled contribs.

I usually update the internal libraries (the VLC developers call them the “contribs”) only when there is a reason. For instance, when VLC requires a newer version of such a library, or because a newer version brings speed improvements (think of ffmpeg). But inbetween minor releases of VLC I usually leave the contribs as they are, and only compile the new VLC sources. In order to save time, the vlc.SlackBuild has a commandline switch which tells it to only build the contribs, but skip the VLC sources. This switch is “–wrapvlcdeps”. If you run the command:

./vlc.SlackBuild –wrapvlcdeps

… then the script will compile all internal libraries, and then “wrap” the compiled contribs into a single tarball in your $OUTPUT directory (which is /tmp by default) with the name “vlc-vlcdeps-$ARCH.tar.bz2“. Here, “$ARCH” is the architecture for which you are compiling (usually i486 or x86_64). Note that this command will not create an actual VLC package! All it does is compile the internal dependencies and wrap those into a tarball for later use.

Now, if you copy that tarball back into the directory which contains vlc.SlackBuild, the script will be able to use its pre-compiled binary contents when it is time to create a VLC package. When you run the following command:

./vlc.SlackBuild –unwrapvlcdeps

… then the script will “unwrap” the contents of the “vlc-vlcdeps” tarball and won’t bother with compiling all those contribs again. Instead, the script will proceed with compiling the VLC sources and creating the final package. Big timesaver!

Note:

You may be aware that I need to compile two versions of VLC for my repositories, thanks to stupid US software patents (Europeans, do not forget to sign the anti-ACTA petition at https://secure.avaaz.org/en/eu_save_the_internet_spread/ !). One version contains code which would violate the US software patents, such as mp3 and aac audio encoders, while the other version is free of software patent issues and therefore free to distribute inside the US. The vlc.SlackBuild script can distinguish between “restricted” and “unrestricted” tarballs containing the “contribs” or vlcdeps as I call them myself. The script has this piece of code:

  if [ "${USE_PATENTS}" == "YES" ]; then
    DEPSFILE="$SRCDIR/$PRGNAM-vlcdeps_restricted-$ARCH.tar.bz2"
  else
    DEPSFILE="$SRCDIR/$PRGNAM-vlcdeps_unrestricted-$ARCH.tar.bz2"
  fi
  [ ! -f $DEPSFILE ] && DEPSFILE="$SRCDIR/$PRGNAM-vlcdeps-$ARCH.tar.bz2"

So, if you too need to create two versions of the package (for instance if you distribute your package inside the US) it is up to you to rename the DEPSFILE “vlc-vlcdeps-$ARCH.tar.bz2” after you created it. You should add the string “restricted” or “unrestricted” to its name depending on whether it contains patented code or is unrestricted.

Have fun! Eric

KDE bugfixes and how to use my modular KDE.SlackBuild

KDE 4.7.2 fixes

In the past week, fixes have been posted to the KDE packagers mailing list, asking us to apply those to our packages. Both fixes are important enough that I updated my kdelibs and kdepim-runtime packages.

There is a fix for kdelibs of which Sebastian Trueg states “Hi packagers, sadly I introduced a serious issue into kdelibs 4.7.2 which was intended to be a fix. This bug prevents any query which does NOT use wide unicode characters to fail. The attached patch provides a workaround for this issue and will make all queries work“. Another patch is for the kdepim-runtime package. It fixes bug 283467Kmail has duplicated folders after migration from previous version

Both updated packages are now available in the KDE 4.7.2 section of my “ktown” repository.

The 32bit packages:

The 64bit packages:

KDE.SlackBuild HOWTO

Some people have asked how to work with the modularized KDE.SlackBuild script. It is not immediately clear to everybody for instance how to apply patches, or how to rebuild individual packages. I will use this  opportunity to explain a bit about the innards of the new framework.

The KDE.SlackBuild concept is loosely based on Patrick’s modular X.Org build. For a user of the script, there are not really that many differences compared to building X.Org. Let me tell you more:

The directory structure

  • KDE.SlackBuild :this is the script which you run to build the complete KDE Software Compilation, or any number of its individual packages.
  • KDE.options : this file contains common options for all packages. For instance, it contains the version of KDE we are compiling, and the overall build number (usually all the packages will get a build number “1”)
  • build/ : If a package needs another build number than defined in KDE.options, you should create a new file in the “build” directory that must carry the name of this package. Iinside the file you write the desired build number.
  • cmake/ : This directory contains the actual compilation script, called “cmake”.. The cmake program is used to configure the source code for Qt/KDE based applications (make will be called afterwards to compile the configured soiurces). If an application needs a non-standard set of cmake/make parameters or commands, you must create a new file in this directory, carrying the name of the application, and inside you add the custom compilation commands. Use the default “cmake” script as your starting point. The KDE.SlackBuild script will then use this custom file instead of the default “cmake” file.
  • docs/ : The KDE.SlackBuild script will add a couple of common documentation files from the source tarball into the resulting package’s “/usr/doc” diretory. Think of files like “AUTHORS* CONTRIBUTING* COPYING* HACKING* INSTALL* MAINTAINERS README* NEWS* TODO*“. If an application’s source tarball contains other documentation files that you want to have added instead, then you should create a new file in the “docs/” directory, containing the application’s name and inside you add the files that the KDE.SlackBuild should add to the package’s “/usr/doc” directory.
  • doinst.sh/ : To every package a generic “doinst.sh” script will be added. This is the script which is run by “installpkg” and “upgradepkg” after installing the package (for instance to refresh the KDE menu or to register new MIME types). If an application needs its own “doinst.sh” script “doinst.sh” then you add it to the “doinst.sh” directory, the file should carry the name of the application (I am repeating myself a lot but that’s because the concept is identical for all these subdirectories)
  • makepkg/ : Creating the actual package after its compilation has been completed is accomplished by calling the “makepkg” command. If you have a package that needs a custom makepkg commandline you can add that here.
  • modularize : KDE Software Collection is built out of modules (think of modules like “kdelibs”, “kdenetwork”, etc…). These modules used to be distributed as big “monolithic” source tarballs in the past, and we would create one Slackware package for every module. This has changed since KDE 4.7.0 where the sources of these modules were split off into a lot of separate tarballs. The KDE.SlackBuild still builds one package per module if you do not tell it to do otherwise. If you need/want to split up a module into sub-packages, then you edit section in the “modularize” file which applies to this module. Every name you add to the “modularize” file (which must correspond to a source tarball of the same name) will be built as a separate package. for instance, the “kdebase” section contains the following lines, which means that compiling the “kdebase” module will result in the additional packages: konsole, kate, kde-wallpapers, kde-workspace, kde-runtime: These separate programs used to be included in the kdebase package itself previously:

# kdebase:
konsole
kate
kde-wallpapers
kde-workspace
kde-runtime

  • modules/ : This directory contains the KDE modules, one file per module. Inside each file you will find the basenames of the individual source tarballs which make up this module. If a source tarball is listed inside one of these files but is not listed in the “modularize” file (see the previous bullet) then the binaries resulting from compiling this source will be included into the main module package and not be split off into its own separate package.
  • noarch : Most of the KDE packages will be built for a specific machine architecture (i486 or x86_4 for instance). If a package does not contain machine-dependant code then its architecture type should be “noarch“. In that case you should add the name of the package to the “noarch” file.
  • package-blacklist :  The collection of source tarballs is bigger than what we support on the Slackware Linux platform (some of the sources apply to other distros only or to MS Windows, such as the “csharp” tarball for instance). The “package-blacklist” file is where you enter the names of source tarballs to skip compiling  Just the application’s basename – no version.number is needed.
  • patch/ : This directory contains patch scripts for applications (see further down for more detail).
  • post-install/ : A few packages need additional work after the “cmake && make && make install” sequence. If you need that, you can add a file with the name of the application inside the “post-install” directory containing those commands.
  • pre-install/ : A few packages need additional preparation before the “cmake && make && make install” sequence. If you need that, you can add a file with the name of the application inside the “pre-install” directory containing those commands.
  • slack-desc/ : Every package that is going to be created needs a descriptive file (the package’s slack-desc file). They go into the “slack-desc” directory, each carrying the name of their application.
  • src/ : All the KDE sources go in here. It does not matter if you put some of the sources into their own subdirectory, the KDE.SlackBuild script will figure out where they are. For instance, I moved the “extragear” sources (these are the applications which do not belong to the core of KDE SC) into a subdirectory “extragear” but in fact the name can be arbirtrarily chosen.

Compiling one or more single packages

By default, the compilation script “KDE.SlackBuild” will compile all the packages for the KDE Software Compilation. If you want to build only one module, or only a few of a module’s sub-packages, then you have to call the script with the appropriate parameters. If you want to build a sub-package you have to specify to which module it belongs. A module and a sub-package are separated by a colon (:). Multiple sub-packages are separated by a comma (,).

The generic method is:

# ./KDE.SlackBuild [ module[:subpackage1,subpackage2,…] [ module[:subpackage1,subpackage2,…] …] ]

For instance:

  • To compile only the “kdelibs” module, you run:

# ./KDE.SlackBuild kdelibs

  • To compile the “kdelibs” module as well as the kdepim-runtime package which is a component of the “kdepim” module:

# ./KDE.SlackBuild kdelibs kdepim:kdepim-runtime

  • To compile packages for kalgebra, kstars and marble which are all components of the “kdeedu” module, you run:

# ./KDE.SlackBuild kdeedu:kalgebra,kstars,marble

You see that it is actually quite simple, although a bit different from the old method (until KDE 4.6) of building individual packages.

Applying a patch to a KDE package

If you need to patch a package in the KDE Software Compilation, then you must first create a subfolder in the “patch” directory with the name of the package. Then add the patch file into that subdirectory. Finally create a file <packagename>.patch (or edit the file if it already exists) in the “patch” directory, The file “<packagename>.patch” must perform the actual patching, and the patch command must be appended with the following series of commands in order for KDE.SlackBuild to determine if the patch failed or not:

 || { touch ${SLACK_KDE_BUILD_DIR}/${PKGNAME}.failed ; continue ; }

And to make the new package name differ from the old version, you have to update its BUILD number. Edit (or create) a file in the “build” directory for the package and write the new BUILD number in that file. Usually the new BUILD number is one higher than the previous if that was an integer value.

 

Checking if your sources match the build files

Suppose you are not certain if the source tarballs you added to the “src/” directory match the available slack-desc files and module files, you can run the build script with an additional environment variable. If you run the following command, the KDE.SlackBuild script will compare the content of the “src” directory with the content of the “slack-desc” directory and the content of the “modules” directory and will complain if it finds anything amiss (and abort the script):

# PRECHECK=yes ./KDE.SlackBuild

 

I hope this clears up things a bit for you!

Cheers, Eric

New packages – should I build them for Slackware 13.37 or for 13.1?

Hi!

A number of big packages are coming up fairly soon. There will be a new LibreOffice (3.4, a major release), and VLC as well will probably have another release shortly. Also, I should be working on KDE 4.6.3.

I am looking at ways to make this manageable for me. Compiling LibreOffice packages takes forever, even for a single Slackware release. Building LibreOffice packages for both Slackware 13.1 and 13.37 will simply take too much of my time.

So, I am interested to find who is running my LibreOffice 3.3.2 package on Slackware 13.37. Do you have issues? Is it working well? I can tell you that I have no issues with that package, and I am using it on my Slackware 13.37 laptop. I will make a decision based on the feedback I will get from you.

For any other package, I will probably switch to creating them for Slackware 13.37 exclusively. You will have no problem compiling your own version of a package for an older Slackware, using my sources. It’s just that you do not want to compile something like LibreOffice yourself!

Please leave your comments below the article.

Thanks, Eric

Release candidate 1 of KDE SC 4.5

The sources of KDE Software Compilation 4.4.90 (which is  in fact Release Candidate 1 of the upcoming KDE 4.5) have been available on KDE’s ftp server for some days now.

There is still no official announcement from KDE team about this RC1 but it will without doubt appear soon on kde.org after the weekend.

In the meantime I have been building packages (32-bit and 64-bit) that you can install on Slackware 13.1 or -current. See my ktown repository for installation/upgrade instructions.

Please note that this release candidate will likely still have bugs – if you find any please report them to bugs.kde.org so that we can expect a great 4.5 release in august.

You can download 4.4.90 from the server using http:

lftp -c “open http://alien.slackbook.org/ktown/ ; mirror 4.4.90”

or using rsync (mind the dot at the end):

rsync -av rsync://alien.slackbook.org/alien-kde/4.4.90 .

Also next week, there will be a release of KDE SC 4.4.5 (the last in the 4.4 series) and if everything goes as planned, this should end up in slackware-current – meaning, I won’t have to build packages for 4.4.5.

Cheers, Eric

« Older posts

© 2024 Alien Pastures

Theme by Anders NorenUp ↑