My thoughts on Slackware, life and everything

Day: February 19, 2012

Speeding up my vlc.SlackBuild script

In a comment to my previous post about the VLC media player where I announced packages for VLC 2.0.0, Nille asked if I could explain a bit more about the SlackBuild script I use to create these packages.

In my scripts, I sometimes hide interesting side-tracks or shortcuts which you would not notice if you just run the script and wait patiently until the packages have been created.

The VLC SlackBuild script is huge. Apart from VLC sources, it also compiles over 50 (fifty!) other libraries and embeds them into the VLC package. This makes VLC such a versatile media player – it supports many media formats and input sources, has audio visualisation inputs and other cool stuff.

The penalty for having such a big script, is that compiling it all takes quite a bit of time. And if you just want to update VLC to a newer snapshot, it is aggravating when the script also recompiles all of the other internal dependencies. Furthermore, in order to compile the browser plugin, the script needs Mozilla’s xulrunner libraries. Since Slackware does not ship those, the xulrunner SDK has to be compiled as well and that is time-consuming- only to be thrown away after the compilation has ended.

I added a few pieces of code which allow me to considerably shorten the compilation time of my VLC packages.

  1. Allow the use of a tarball containing a pre-built xulrunner SDK
  2. Allow the use of a tarball containing prebuilt “contribs”, i.e. all the internal libraries VLC depends on

Let me explain both.

Pre-compiled mozilla SDK.

If you run the vlc.SlackBuild, and enable the creation of the browser plugin “npapi-vlc” (this is enabled by default in the script) then the SlackBuild script will first compile the xulrunner SDK. This takes a long, long time. So, in order to speed things up, the script will look for a pre-compiled version of the xulrunner SDK in the directory containing “vlc.SlackBuild” and use that instead if available. You can find this SDK tarball after the xulrunner compilation has ended; check the “./dist/sdk” subdirectory in the xulrunner source directory (normally that would be something like “/tmp/build/tmp-vlc/mozilla-<someversion>/dist/sdk/“) and copy the file “xulrunner-*.sdk.tar.bz2” you’ll find there back into the directory containing vlc.SlackBuild. Next time you run vlc.SlackBuild, it will extract and use the contents of that SDK file instead of compiling the SDK all over again. Big timesaver!

Pre-compiled contribs.

I usually update the internal libraries (the VLC developers call them the “contribs”) only when there is a reason. For instance, when VLC requires a newer version of such a library, or because a newer version brings speed improvements (think of ffmpeg). But inbetween minor releases of VLC I usually leave the contribs as they are, and only compile the new VLC sources. In order to save time, the vlc.SlackBuild has a commandline switch which tells it to only build the contribs, but skip the VLC sources. This switch is “–wrapvlcdeps”. If you run the command:

./vlc.SlackBuild –wrapvlcdeps

… then the script will compile all internal libraries, and then “wrap” the compiled contribs into a single tarball in your $OUTPUT directory (which is /tmp by default) with the name “vlc-vlcdeps-$ARCH.tar.bz2“. Here, “$ARCH” is the architecture for which you are compiling (usually i486 or x86_64). Note that this command will not create an actual VLC package! All it does is compile the internal dependencies and wrap those into a tarball for later use.

Now, if you copy that tarball back into the directory which contains vlc.SlackBuild, the script will be able to use its pre-compiled binary contents when it is time to create a VLC package. When you run the following command:

./vlc.SlackBuild –unwrapvlcdeps

… then the script will “unwrap” the contents of the “vlc-vlcdeps” tarball and won’t bother with compiling all those contribs again. Instead, the script will proceed with compiling the VLC sources and creating the final package. Big timesaver!

Note:

You may be aware that I need to compile two versions of VLC for my repositories, thanks to stupid US software patents (Europeans, do not forget to sign the anti-ACTA petition at https://secure.avaaz.org/en/eu_save_the_internet_spread/ !). One version contains code which would violate the US software patents, such as mp3 and aac audio encoders, while the other version is free of software patent issues and therefore free to distribute inside the US. The vlc.SlackBuild script can distinguish between “restricted” and “unrestricted” tarballs containing the “contribs” or vlcdeps as I call them myself. The script has this piece of code:

  if [ "${USE_PATENTS}" == "YES" ]; then
    DEPSFILE="$SRCDIR/$PRGNAM-vlcdeps_restricted-$ARCH.tar.bz2"
  else
    DEPSFILE="$SRCDIR/$PRGNAM-vlcdeps_unrestricted-$ARCH.tar.bz2"
  fi
  [ ! -f $DEPSFILE ] && DEPSFILE="$SRCDIR/$PRGNAM-vlcdeps-$ARCH.tar.bz2"

So, if you too need to create two versions of the package (for instance if you distribute your package inside the US) it is up to you to rename the DEPSFILE “vlc-vlcdeps-$ARCH.tar.bz2” after you created it. You should add the string “restricted” or “unrestricted” to its name depending on whether it contains patented code or is unrestricted.

Have fun! Eric

LibreOffice 3.5.0 is out… no Slack packages yet

Last week, the Document Foundation released version 3.5.0 of their LibreOffice suite.

Read all about it in their official announcement “the best free office suite ever“. LibreOffice has made great strides ever since it was forked off OpenOffice. On LinuxQuestions.org, which hosts my favourite Slackware forum, LibreOffice was chosen by its member community as Office Suite of the Year (with 81.01% of the votes). Bravo!

Of course, I wanted to have Slackware packages ready ASAP. Silly me… the developers are changing the build process with each new release cycle. The 3.3 -> 3.4 switch gave me headaches and 3.4 -> 3.5 is no different. In fact, it is even worse. The build system is moving toward using standard autoconf/automake/make tooling, and piece-by-piece getting rid of java/dmake for its compilation. This means, I had to revise my libreoffice.SlackBuild script again.

Unfortunately I did not have the chance (due to time constraints) to test beta builds, so now that the release is there, I am faced with an inability to compile new packages…

I have been running compilations for days, breaking off after 12+ hours repeatedly, and currently I can’t even get past the “configure” stage… being stuck at a horribly broken SlackBuild script.

No doubt I will eventually succeed, and present you some nice packages, but not soon. Terribly sorry, but I thought I would at least let you know.

In the meantime, I did manage to build VLC packages (which you can read all about in my previous post) and soon some new QEMU and qemu-kvm packages. A lot of my time is currently spent on a new ARM port of Slackware. I hope to write some more about that too, in the near future.

Cheers, Eric

© 2024 Alien Pastures

Theme by Anders NorenUp ↑