My thoughts on Slackware, life and everything

Tag: git (Page 1 of 2)

How I setup cgit for a browsable liveslak repository

I received a request to document how I configured the backend for https://git.liveslak.org/ . This is where my git repository for liveslak is accessible and browseable using cgit as the engine.
Of course the server is also running an actual git repository service which people use to clone the liveslak files and for me to upload changes.
Setting up git is not hard, but it’s beyond the scope of this blog article. Also, setting up Let’s Encrypt to create a secure (https) web site is not in the scope of this article but you can read my Let’s Encrypt article of course.

I’ll explain the steps that are needed to create a cgit webserver in the text below.
First we build and install cgit as well as its dependencies; then we create the cgit configuration and add some customization. Lastly we configure Apache httpd so that it knows what to serve.

Let’s start with building the required packages in order. In case you are running Slackware -current (i.e. newer than 15.0) then the lua package is already included in the core distro, so you can skip compiling it now:

  • lua
  • luacrypto
  • lua-md5
  • highlight
  • cgit

You can find build scripts for all of them on the SlackBuilds.org (SBo) web site.
The lua* and highlight packages are needed for syntax highlighting, and lua is also needed for gravatar support.
The cgit package has customization added by the SBo admins which we are going to use, notably support for displaying the committers’ gravatar images.

Once you have built and installed the above packages, create a new directory to hold our custom cgit stuff:

# mkdir -p /home/www/cgit

Create some symlinks to the files that were installed by the Slackware cgit package:

# ln -s /var/www/cgi-bin/cgit.cgi /home/www/cgit/
# ln -s /var/www/cgi-bin/cgit.js /home/www/cgit/
# ln -s /var/www/cgi-bin/cgit.png /home/www/cgit/

But make a copy, not a symlink, of the CSS file:

# cp -ia /var/www/cgi-bin/cgit.css /home/www/cgit/

Some additional CSS code needs to be added to cgit.css to make the committer avatars hover properly. Here’s the lines to append to the file copy we just made (you will find this same code in the file /usr/doc/cgit-*/email-gravatar-sbo-additions.css which is part of the Slackware cgit package):

div#cgit span.gravatar img.onhover {
    display: none;
    border: 1px solid gray;
    padding: 0px;
    -webkit-border-radius: 4px;
    -moz-border-radius: 4px;
    border-radius: 4px;
    width: 128px;
    height: 128px;
}
div#cgit span.gravatar img.inline {
    -webkit-border-radius: 3px;
    -moz-border-radius: 3px;
    border-radius: 3px;
    width: 13px;
    height: 13px;
    margin-right: 0.2em;
    opacity: 0.8;
}
div#cgit span.gravatar:hover > img.onhover {
    display: block;
    position: absolute;
    margin-left: 1.5em;
    background-color: #eeeeee;
    box-shadow: 5px 5px 3px #bbb;
}

We also need to tweak the syntax-highlighting.sh script (part of the cgit package) a bit so it works better for us Slackers.
Make a copy of it, removing all comments:

# grep -Ev '(#$|^ *#)' /usr/share/cgit/filters/syntax-highlighting.sh > /home/www/cgit/syntax-highlighting.sh
# chmod +x /home/www/cgit/syntax-highlighting.sh

And then add these lines right before the final ‘exec’ line of the script:

# map SlackBuild to .sh
[ "$EXTENSION" == "SlackBuild" ] && EXTENSION=sh

To make the git.liveslak.org website stand out from others, I substituted my own frontpage image and a custom favicon image. These are created as /home/www/cgit/eric_hameleers.png (link) and /home/www/cgit/erichameleers_favicon.ico (link) and both filenames are referenced further down in the cgit configuration.
By default, cgit will look for a file called “/etc/cgitrc” to read its configuration, but I am running multiple sites on a single Slackware host (git.slackware.nl for instance), therefore I create a separate cgit configuration file for each site. For liveslak, it is called “/etc/cgitrc.git.liveslak.org” and then I instruct Apache httpd to look for that particular filename.

/etc/cgitrc.git.liveslak.org

# For more options, see cgitrc.5.txt in the docs folder
root-title=Alien BOB's
root-desc=Web interface to liveslak git repository
virtual-root=/
snapshots=tar.gz tar.xz

summary-branches=10
summary-log=10
summary-tags=10

repository-sort=date
commit-sort=date
branch-sort=age

enable-blame=1
enable-index-links=1
enable-commit-graph=1
enable-follow-links=1
enable-log-filecount=1
enable-log-linecount=1
max-stats=quarter

mimetype-file=/etc/httpd/mime.types
mimetype.rss=application/rss+xml
enable-html-serving=1

css=/cgit.css
logo=/eric_hameleers.png
favicon=/erichameleers_favicon.ico

about-filter=/usr/share/cgit/filters/about-formatting.sh

# for this to work you have to build cgit against lua
# and install luacrypto too
email-filter=lua:/usr/share/cgit/filters/email-gravatar-sbo.lua

# For this to work, you need lua and highlight packages:
source-filter=/home/www/cgit/syntax-highlighting.sh

repo.url=liveslak
repo.path=/path/to/repositories/liveslak.git
repo.desc=Slackware Live Edition by AlienBOB
repo.owner=alien@slackware.com
repo.clone-url=git://liveslak.org/liveslak.git
repo.readme=:README.txt

The actual definition for the git repository is the lines at the end starting with “repo.“.

Lastly, the Apache web server needs to be configured to serve our cgit stuff whenever someone visits git.liveslak.org.
These are the files used to accomplish that:

/etc/httpd/extra/git.liveslak.org.conf

(just an example filename, you will know how and where to add the content for your specific Apache configuration):

<VirtualHost *:80>
    ServerName git.liveslak.org
    ServerAdmin alien@slackware.com

    CustomLog /var/log/httpd/git.liveslak.org_access_log combined
    ErrorLog /var/log/httpd/git.liveslak.org_error_log

    # Include Let's Encrypt configuration:
    Include /etc/httpd/extra/letsencrypt.conf

    # Include the site definition (AFTER Let's Encrypt configuration!!):
    Include /etc/httpd/extra/git.liveslak.org_content.conf
</VirtualHost>
<VirtualHost *:443>
    ServerName git.liveslak.org
    ServerAdmin alien@slackware.com

    CustomLog /var/log/httpd/git.liveslak.org-ssl_access_log combined
    ErrorLog /var/log/httpd/git.liveslak.org-ssl_error_log

    # Include Let's Encrypt configuration:
    Include /etc/httpd/extra/letsencrypt.conf

    # Include web server generic SSL configuration for vhosts:
    Include /etc/httpd/extra/ssl_vhost.conf

    # Include the site definition (AFTER Let's Encrypt configuration!!):
    Include /etc/httpd/extra/git.liveslak.org_content.conf
</VirtualHost>

The file “git.liveslak.org_content.conf” is included twice in the above VirtualHost definition file for Apache. Using these “Include” lines prevents some duplication of code.
Here is the content of that included file:

/etc/httpd/extra/git.liveslak.org_content.conf

DocumentRoot /home/www/cgit
Options FollowSymlinks

# Use a custom name for the configuration - default is /etc/cgitrc:
SetEnv CGIT_CONFIG /etc/cgitrc.git.liveslak.org

DirectoryIndex cgit.cgi
AllowOverride none
Require all granted

SetHandler cgi-script

Options ExecCGI
Require all granted

Alias /cgit.css /home/www/cgit/cgit.css
Alias /cgit.png /home/www/cgit/cgit.png
Alias /erichameleers_favicon.ico /home/www/cgit/erichameleers_favicon.ico
Alias /eric_hameleers.png /home/www/cgit/eric_hameleers.png
Alias /robots.txt /home/www/cgit/robots.txt
Alias / /home/www/cgit/cgit.cgi/

I hope that this story may help some of you with getting your own browseable git repository online.

Good luck, Eric

Liveslak 1.8.0 – more filesystems supported, lots of fixes

Liveslak updates! Quite soon after my August ISO refresh, I used some free moments to implement a request of sorts and fix some longstanding bugs.
Version 1.8.0 of the liveslak scripts is now available, containing these enhancements and fixes.
Links to the liveslak git repository and download locations are at the bottom of this post. Of course, the new functionality and fixes are also present in a fresh batch of Slackware Live ISO images.

New features of liveslak 1.8.0

  • LUKS encrypted containers for your homedirectory and for persistence are now supporting many more filesystems, not just ext2/4 but also btrfs, f2fs, jfs or xfs. The lack of f2fs support was mentioned by a visitor of this blog and I thought that was useful feedback.
    The accompanying scripts iso2usb.sh and isocomp.sh now support this filesystem choice via a new ‘-F‘ switch.
    Note that the Linux partition of a USB Live thumbdrive remains ext4 formatted, because liveslak uses extlinux to boot on a BIOS computer.
  • The upslak.sh script can now extend the size of the  LUKS encrypted container files on your USB thumbdrive, in case you run out of storage there.
  • I fixed the pxeserver script and expanded its functionality:
    • PXE boot of a UEFI computer finally works.
      Note that this may not work for you out of the box, because there are two implementations in dnsmasq to support UEFI PXE boot… and sometimes, one will work and the other won’t. So, you may have to open the /usr/local/sbin/pxeserver script in an editor and look for the section where a number of ‘dhcp-match‘ and ‘dhcp-boot‘ lines is commented out. If you remove the comment characters from these 8 lines and instead, add a comment character ‘#’ in front of the ‘dhcp-match‘ and ‘pxe-service‘ lines a bit higher up, your UEFI computer might actually succeed in booting over the network.
    • NAT firewalling can be optionally enabled in case PXE clients wouldn’t have Internet access otherwise.
  • I fixed the hang during shutdown/reboot when liveslak is a PXE client.
  • I fixed UEFI boot using GRUB when the live ISO had been ‘dd’-ed or ‘cp’-ed directly to a USB thumbdrive. You would end at the GRUB prompt instead of booting into the OS.
    It took me quite a while because this bug was introduced in Feb 2019 (!), and I never found the time to investigate. Eventually looking into it during a weekend of solitude caused an epiphany.
  • I added support for a dark theme in KDE Plasma based ISOs, with the LEAN ISO as an example (that one also gets a new login/desktop background image taken from my photo collection with every ISO refresh).
  • Lots of other small fixes and enhancements, read the Git log for more information.

Booting from an on-disk ISO file

Liveslak supports Ventoy, a multi-boot manager for removable media like USB thumbdrives which lets you boot any ISO image you store on the disk, selecting from a GRUB menu it creates on the fly.
Because liveslak  implements the “Ventoy-compatible” guideline, any Slackware Live ISO works out of the box on Ventoy. Support for encrypted persistence and homedirectory containers on a Ventoy disk is offered by the ISO companion script ‘isocomp.sh‘.
You can find the details in the git commit message.

Install Slackware using a liveslak ISO

A quick reminder that you can use a liveslak ISO to install Slackware to your hard drive from its official package repository (using a network installation from a HTTP, FTP, NFS or Samba server).
You can also install the actual content of the Live ISO to your harddisk if you like. In this case all “live aspects” will be skipped during the copy, so that you will in fact end up with a completely regular Slackware on your harddisk.
The program to run these installations is available on all liveslak ISOs and is called ‘setup2hd‘. It is derived from the ‘setup‘ program of a Slackware install media, but has some nice enhancements: it launches cfdisk/cgdisk for you to partition your disks, lets you create a regular user, and allows you to configure a basic firewall.
Best of all, you can run setup2hd in an X-terminal and let the installer chomp through the packages while you browse the internet, watch a video or perform any other kind of leisurely activities on your Live OS.

And by the way: booting Slackware Live is the only way (using setup2hd) to install the official Slackware distro from a network server across a wireless connection. The official Slackware installer only supports wired network connections.

Get liveslak ISOs

The various variants of Slackware Live Edition can be found in the “latest” subdirectory at https://download.liveslak.org/ or its US mirror https://us.liveslak.org/ . A fast UK mirror is provided by Darren Austin at https://slackware.uk/liveslak/ .
You’ll be able to download ISO Live images of 32bit and 64bit Slackware proper, also of the small XFCE variant for (both architectures), and then CINNAMON, DAW, LEAN and MATE ISOs that only come in 64bits.
Big thanks to Willy Sudiarto Raharjo (willysr) for maintaining the Mate and Cinnamon Slackware package repositories.
Also have a look in the “bonus” subdirectory! There you’ll find Nvidia graphics and Broadcom wireless binary drivers, Wine 8 and multilib modules to add if you use the persistent version of liveslak.

All ISOs containing a 64bit Live Slackware have support for SecureBoot.

Get liveslak sources

The liveslak project is hosted in git. Its browsable cgit interface is here: https://git.liveslak.org/liveslak/

A set of the liveslak scripts can also be downloaded from http://www.slackware.com/~alien/liveslak/ or https://slackware.nl/people/alien/liveslak/

Have fun! Eric

Tracking development of slackware in git

Something had been nagging me for a long time, and I finally had enough of that itch and decided to deal with it.

As you know, there’s a private and a public side to Slackware’s development. The discussions and decisions are handled internally among the members of ‘the team’ and are not shared with the public at large until an update is done to the ‘slackware-current’ tree which can be found on every Internet mirror.
Thus you have access to the latest state of development always. But for some people it is a compelling idea to be able to access the development updates in a public repository like git – where you can track the changes over time.

A recent discussion on LinuxQuestions brought up the topic of SlackBuild scripts in Slackware-current. The scripts you can find in the -current directory tree on the Slackware mirrors are always the latest version. Sometimes there’s a good reason to want to go back in time and fetch an earlier version. In the thread post with the appropriate number “1337” it is ponce (Matteo Bernardini) who replies with a link to a git repository maintained by Adrien Nader which already has been tracking the development in -current for nearly 8 years!. So it’s quite a convenient way to retrieve a historical version of any script.

Me being me, it’s the existence of that repository which has been nagging me for a couple of years. Why? Because I wondered how it was done. And if I question an issue long enough, I will eventually create my own solution – as a learning exercise of course, but also to give back to the community.

And so, today arrived. I was pondering – if I were to create a git repository for tracking the developments in -current, what would I want in there? Exactly the same as Adrien’s? The answer has been “no” for a while. The most important capability that is missing from Adrien’s repo is that it contains a lot of compressed files that are impossible to read. Think of patches and doinst.sh scripts, and more. So I gave myself the task to implement a git repository with uncompressed files, as an improvement on the original effort. Also, it should track all relevant files in the complete tree, not just in the “./source/” subdirectory. In particular the documentation files (various .TXT files).

The result is a script, maintain_current_git.sh, and a repository, https://git.slackware.nl/current/ .
The repository just had its first commit. For those who want to check out a commit in order to compile a package from there, the maintain_current_git.sh script generates another script called ‘recompress.sh‘ and stores that script in the root directory of the repository. When you run this recompress script in the root directory of the repository, it will re-compress all the files that had been un-compressed before committing them to git. That way, a SlackBuild script will find the correct files and will function as intended. Note that you would still have to download the source tarballs from somewhere, because this repository of mine will only track the Slackware-specific files.

I decided that it is prudent and more respectful to not import Adrien’s work into my own repository. The two are similar but different and I think everyone of you can choose which repository suits your needs better.

I have scheduled the above script to run twice a day and update the git repository when new updates become available.
As with all my scripts, this one has a “-h” parameter to explain its usage. Let me know if it – and the git repository – are useful to you.
This particular script may be a bit messy because I have not spent a lot of time polishing it. I hope that’s OK 😉

Have fun!

Using git while working on KDE 5

Even though I caught a bad cold (luckily, it was not the flu as I feared at first) I managed to do a lot of prepping for new KDE 5 packages (Frameworks, Plasma, Applications) since last week.

The way I tackled KDE updates in the past, was to update the sources, check the continued need of patches used for prior releases, and then start compiling and checking the build logs for errors – and responding to errors with fixes to the scripts, new patches or even new or updated dependencies. The core of the KDE.SlackBuild framework would remain largely unchanged, and it never took lots of going back and forth to recompile packages and get rid of errors or missing dependencies. KDE 4 has always been very easy to update (well, easy – experience helps a lot, plus the fact that I wrote the KDE.SlackBuild framework and know its ins and outs).

Working with the first test release of KDE 5, last summer, was a bit of a challenge, but I took it like I did with every change to a new release cycle of KDE 4 (4.11 -> 4.12 etc…): finding all available documentation on the available sources, their inter-dependencies as well as their other needs, and reading the comments and patch proposals on the KDE packagers mailing list. Going from KDE 4 to 5 was a heck of a lot more work than I initially anticipated (the hardest part was figuring out the tiering concept of the Frameworks, and of course there was a lot of cursing at the immaturity of the Plasma sources) but I considered that first release for Slackware as nothing more than a play-test. The preview packages were designed not to mess up your existing KDE 4 configuration, and easily un-installable. For me the important thing with this proof of concept was to convince myself that the updates to the KDE.SlackBuild framework were sound and I understood and was able to respond to the new reality of fragmented release cycles.

This last week, I have been targeting a more serious outcome. The upcoming KDE 5 packages for Slackware which I will be releasing sometime later this month, are meant to replace your good old KDE 4 installment, and therefore I need to deliver something that you can work with on a daily basis – a full set of applications in a functional Desktop Environment.

For that reason, I decided to call upon git to guide me through the process of making changes to the build framework. I needed to be able to reproduce my previous build environments in order to back-track after encountering errors or dead ends, and take a fork from there and try again. Putting the whole KDE.SlackBuild tree in a git repository was something I did in advance – see my previous post on KDE 5 – and since then, I have been pushing my changes to the repository continuously.

Definitely, this has saved me from finding myself in a dead end. What sometimes happened was that some application would compile into a package one time, and fail to compile the next iteration. Usually the cause would be an update of a dependency, the introduction of a patch to another package and more of that vague stuff. My git updates enabled me to inspect all updates I had done since a previous compilation attempt, and revert or adapt some of the changes I had made. In some other cases, the changes I could track graphically in the ktown cgit web interface showed cause and effect with more ease than back in the time when I would keep multiple directory trees of past attempts. A time saver!

I am currently compiling the last couple of packages that are part of the Applications. If they succeed, it means that the KDE.SlackBuild is ready and I have a full set of packages. If you ask whether they produce a wonderful Desktop Environment… I do not know yet 🙂 That will be the next step: when all packages are conceptually OK, I will fire up an X session in the virtual machine and see if the stuff I compiled works at all. If I get into a functional KDE 5 environment, that will be cause for celebration. But I expect that I will have issues in the VM (with sddm and/or kwin). If I see failure, I will try installing the packages on a real computer and see how it fares there.

I won’t release any packages until I am fully satisfied, but feel free to tinker with the KDE.SlackBuild in advance.

Eric

Ktown for Slackware – development history available in git

kde4 For a long time I have been keeping copies of the full source directories for every KDE 4 release I have made for Slackware. That is amounting to a lot of megabytes, since I am also keeping the source tarballs, not just the scripts and patches. Traditionally, I have kept one KDE version publicly available for all recent Slackware releases, in my ‘ktown’ package repository at http://alien.slackbook.org/ktown/ . This repository is also available through rsync, not just http (using my primary mirror at rsync://taper.alienbase.nl/mirrors/alien-kde/).

But what was missing (because I am lazy) is a git interface. Now, you could argue that accessing the ktown scripts and patches through git is somewhat pointless, since you would either want the packages (through direct download or using slackpkg+) or the full sources (inclusing the source tarballs, not just the scripts) and the git interface I intend would not be offering that.

However,  with the advance of KDE 5, it starts making sense to use git: primarily for version control and history keeping.

The old KDE 4 releases were self-contained: all sources would be released at the same time, so that I could simply create a toplevel directory of, say, “4.14.3” and put everything that I needed to compile the packages below that directory. Easy-peasy. KDE 5 on the other hand, has abandoned the big coördinated release effort. Instead, the releases are now done for subsets of the Software Collection (old name, no longer used). You’ll find separate uncoördinated releases for the KDE Frameworks, Plasma and the Applications. Of course these are all dependent on each other and require specific  versions to work together, but there is no longer something like a “KDE 5.2.0”.

You saw in my initial release of KDE 5 packages that I simply named the toplevel directory “5”, i.e. without minor numbers. My plan for the future is to stick to that number until Frameworks and Plasma move away from 5.x. Inside the “5” directory you will see future partial updates: whenever new Frameworks, Plasma and Applications tarballs are released, I will update one or more of these at the same time, but not necessarily all of them at the same time.

This poses a problem for my internal version control. Copying directory trees every time I refresh a subsection, will create a big mess with greater risk of not being able to go back to a previous point in time, in particular for the “deps” packages but I also see a risk of not being able to retrace working combinations of Frameworks, Plasma and Applications.

Therefore I have decided to move my ktown sources and scripts into a git repository. What I did, was take every final increment of every KDE release since KDE 4.6 for which I have ever created packages, and build a git version control history using those sources. The git repository has a master branch which will be the release which I consider most recent and stable. At the moment, that is 4.14.3. Every release will have its own branch too. There are branches for 4.6.5, 4.7.4, 4.8.4, 4.9.5, 4.10.5, 4.11.5, 4.12.5, 4.13.3, 4.14.3 and 5 at the moment, and you’ll find them here:

http://taper.alienbase.nl/gitweb/?p=ktown.git

I think this will work best for future development on KDE. Currently, the patches in there are mostly gzipped, since this is what Slackware wants, but I am thinking about creating “unzipped” branches for recent releases, so that it will be easier to peek into the patches.

Any feedback, tips etc, let me know.

Eric

Edit – Tue Dec 23 12:26:25 UTC 2014:

I have added a cgit interface as well: http://taper.alienbase.nl/cgit/ktown/

« Older posts

© 2024 Alien Pastures

Theme by Anders NorenUp ↑