Tag Archives: git

Tracking development of slackware in git

Something had been nagging me for a long time, and I finally had enough of that itch and decided to deal with it.

As you know, there’s a private and a public side to Slackware’s development. The discussions and decisions are handled internally among the members of ‘the team’ and are not shared with the public at large until an update is done to the ‘slackware-current’ tree which can be found on every Internet mirror.
Thus you have access to the latest state of development always. But for some people it is a compelling idea to be able to access the development updates in a public repository like git – where you can track the changes over time.

A recent discussion on LinuxQuestions brought up the topic of SlackBuild scripts in Slackware-current. The scripts you can find in the -current directory tree on the Slackware mirrors are always the latest version. Sometimes there’s a good reason to want to go back in time and fetch an earlier version. In the thread post with the appropriate number “1337” it is ponce (Matteo Bernardini) who replies with a link to a git repository maintained by Adrien Nader which already has been tracking the development in -current for nearly 8 years!. So it’s quite a convenient way to retrieve a historical version of any script.

Me being me, it’s the existence of that repository which has been nagging me for a couple of years. Why? Because I wondered how it was done. And if I question an issue long enough, I will eventually create my own solution – as a learning exercise of course, but also to give back to the community.

And so, today arrived. I was pondering – if I were to create a git repository for tracking the developments in -current, what would I want in there? Exactly the same as Adrien’s? The answer has been “no” for a while. The most important capability that is missing from Adrien’s repo is that it contains a lot of compressed files that are impossible to read. Think of patches and doinst.sh scripts, and more. So I gave myself the task to implement a git repository with uncompressed files, as an improvement on the original effort. Also, it should track all relevant files in the complete tree, not just in the “./source/” subdirectory. In particular the documentation files (various .TXT files).

The result is a script, maintain_current_git.sh, and a repository, https://git.slackware.nl/current/ .
The repository just had its first commit. For those who want to check out a commit in order to compile a package from there, the maintain_current_git.sh script generates another script called ‘recompress.sh‘ and stores that script in the root directory of the repository. When you run this recompress script in the root directory of the repository, it will re-compress all the files that had been un-compressed before committing them to git. That way, a SlackBuild script will find the correct files and will function as intended. Note that you would still have to download the source tarballs from somewhere, because this repository of mine will only track the Slackware-specific files.

I decided that it is prudent and more respectful to not import Adrien’s work into my own repository. The two are similar but different and I think everyone of you can choose which repository suits your needs better.

I have scheduled the above script to run twice a day and update the git repository when new updates become available.
As with all my scripts, this one has a “-h” parameter to explain its usage. Let me know if it – and the git repository – are useful to you.
This particular script may be a bit messy because I have not spent a lot of time polishing it. I hope that’s OK 😉

Have fun!

Using git while working on KDE 5

Even though I caught a bad cold (luckily, it was not the flu as I feared at first) I managed to do a lot of prepping for new KDE 5 packages (Frameworks, Plasma, Applications) since last week.

The way I tackled KDE updates in the past, was to update the sources, check the continued need of patches used for prior releases, and then start compiling and checking the build logs for errors – and responding to errors with fixes to the scripts, new patches or even new or updated dependencies. The core of the KDE.SlackBuild framework would remain largely unchanged, and it never took lots of going back and forth to recompile packages and get rid of errors or missing dependencies. KDE 4 has always been very easy to update (well, easy – experience helps a lot, plus the fact that I wrote the KDE.SlackBuild framework and know its ins and outs).

Working with the first test release of KDE 5, last summer, was a bit of a challenge, but I took it like I did with every change to a new release cycle of KDE 4 (4.11 -> 4.12 etc…): finding all available documentation on the available sources, their inter-dependencies as well as their other needs, and reading the comments and patch proposals on the KDE packagers mailing list. Going from KDE 4 to 5 was a heck of a lot more work than I initially anticipated (the hardest part was figuring out the tiering concept of the Frameworks, and of course there was a lot of cursing at the immaturity of the Plasma sources) but I considered that first release for Slackware as nothing more than a play-test. The preview packages were designed not to mess up your existing KDE 4 configuration, and easily un-installable. For me the important thing with this proof of concept was to convince myself that the updates to the KDE.SlackBuild framework were sound and I understood and was able to respond to the new reality of fragmented release cycles.

This last week, I have been targeting a more serious outcome. The upcoming KDE 5 packages for Slackware which I will be releasing sometime later this month, are meant to replace your good old KDE 4 installment, and therefore I need to deliver something that you can work with on a daily basis – a full set of applications in a functional Desktop Environment.

For that reason, I decided to call upon git to guide me through the process of making changes to the build framework. I needed to be able to reproduce my previous build environments in order to back-track after encountering errors or dead ends, and take a fork from there and try again. Putting the whole KDE.SlackBuild tree in a git repository was something I did in advance – see my previous post on KDE 5 – and since then, I have been pushing my changes to the repository continuously.

Definitely, this has saved me from finding myself in a dead end. What sometimes happened was that some application would compile into a package one time, and fail to compile the next iteration. Usually the cause would be an update of a dependency, the introduction of a patch to another package and more of that vague stuff. My git updates enabled me to inspect all updates I had done since a previous compilation attempt, and revert or adapt some of the changes I had made. In some other cases, the changes I could track graphically in the ktown cgit web interface showed cause and effect with more ease than back in the time when I would keep multiple directory trees of past attempts. A time saver!

I am currently compiling the last couple of packages that are part of the Applications. If they succeed, it means that the KDE.SlackBuild is ready and I have a full set of packages. If you ask whether they produce a wonderful Desktop Environment… I do not know yet 🙂 That will be the next step: when all packages are conceptually OK, I will fire up an X session in the virtual machine and see if the stuff I compiled works at all. If I get into a functional KDE 5 environment, that will be cause for celebration. But I expect that I will have issues in the VM (with sddm and/or kwin). If I see failure, I will try installing the packages on a real computer and see how it fares there.

I won’t release any packages until I am fully satisfied, but feel free to tinker with the KDE.SlackBuild in advance.

Eric

Ktown for Slackware – development history available in git

kde4 For a long time I have been keeping copies of the full source directories for every KDE 4 release I have made for Slackware. That is amounting to a lot of megabytes, since I am also keeping the source tarballs, not just the scripts and patches. Traditionally, I have kept one KDE version publicly available for all recent Slackware releases, in my ‘ktown’ package repository at http://alien.slackbook.org/ktown/ . This repository is also available through rsync, not just http (using my primary mirror at rsync://taper.alienbase.nl/mirrors/alien-kde/).

But what was missing (because I am lazy) is a git interface. Now, you could argue that accessing the ktown scripts and patches through git is somewhat pointless, since you would either want the packages (through direct download or using slackpkg+) or the full sources (inclusing the source tarballs, not just the scripts) and the git interface I intend would not be offering that.

However,  with the advance of KDE 5, it starts making sense to use git: primarily for version control and history keeping.

The old KDE 4 releases were self-contained: all sources would be released at the same time, so that I could simply create a toplevel directory of, say, “4.14.3” and put everything that I needed to compile the packages below that directory. Easy-peasy. KDE 5 on the other hand, has abandoned the big coördinated release effort. Instead, the releases are now done for subsets of the Software Collection (old name, no longer used). You’ll find separate uncoördinated releases for the KDE Frameworks, Plasma and the Applications. Of course these are all dependent on each other and require specific  versions to work together, but there is no longer something like a “KDE 5.2.0”.

You saw in my initial release of KDE 5 packages that I simply named the toplevel directory “5”, i.e. without minor numbers. My plan for the future is to stick to that number until Frameworks and Plasma move away from 5.x. Inside the “5” directory you will see future partial updates: whenever new Frameworks, Plasma and Applications tarballs are released, I will update one or more of these at the same time, but not necessarily all of them at the same time.

This poses a problem for my internal version control. Copying directory trees every time I refresh a subsection, will create a big mess with greater risk of not being able to go back to a previous point in time, in particular for the “deps” packages but I also see a risk of not being able to retrace working combinations of Frameworks, Plasma and Applications.

Therefore I have decided to move my ktown sources and scripts into a git repository. What I did, was take every final increment of every KDE release since KDE 4.6 for which I have ever created packages, and build a git version control history using those sources. The git repository has a master branch which will be the release which I consider most recent and stable. At the moment, that is 4.14.3. Every release will have its own branch too. There are branches for 4.6.5, 4.7.4, 4.8.4, 4.9.5, 4.10.5, 4.11.5, 4.12.5, 4.13.3, 4.14.3 and 5 at the moment, and you’ll find them here:

http://taper.alienbase.nl/gitweb/?p=ktown.git

I think this will work best for future development on KDE. Currently, the patches in there are mostly gzipped, since this is what Slackware wants, but I am thinking about creating “unzipped” branches for recent releases, so that it will be easier to peek into the patches.

Any feedback, tips etc, let me know.

Eric

Edit – Tue Dec 23 12:26:25 UTC 2014:

I have added a cgit interface as well: http://taper.alienbase.nl/cgit/ktown/

Using git for version control

I have always really hated the Git Version Control System (or VCS).

I am pretty comfortable working with other products like RCS, CVS and Subversion (SVN). I am not a developer, so my main use for a VCS is a to allow me to keep track of the changes in my SlackBuild scripts or my server’s log book. I use RCS for this, it is simple but effective.

Of course, in order to compile software it is often required to check out the source code for the program. This is how I got familiar with CVS, SVN, mercurial and git. Way back when I was “build manager” for a software company, I have used PVCS, Clearcase and Visual SourceSafe which are all commercial programs. So, I have been around.

Still, git still confused me after all that time as a end-user. Git uses some alien (to me) concepts which I did not grasp because I never took the trouble to dive into its documentation – I knew just enough to checkout source code from a git repository and did not need – did not want to – know more.

This changed recently. Because two of the projects I am involved with are preparing to use git as their Version Control System, I was forced to start learning how to approach the tool as a developer.

I quickly decided it was worth the while to run two parallel tracks: start reading documentation, and at the same time actually using git by setting up a server for hosting git repositories.

I found some places which host very good reading material.

  • The Pro Git book at http://progit.org/ . This book is hosted itself in a git repository and the language translations are coordinated using git. Well-written with lots of visual examples.
  • The Git Community Book at http://book.git-scm.com/ .

I’m halfway through the “Pro Git” book, retracing my steps a few times after the new concepts found a place in my brain. Indeed, git is starting to make sense now. Surely it has some very strong points which make it interesting not only for large projects with lots of developers who are scattered all over the place (like the Linux kernel developers) but also for small projects like those I am participating in.

Perhaps I will document more about my git activities in a future Wiki article – for instance how I am setting up an online git repository.

This made me think of something else: could it be beneficial to dump the history of my SlackBuild script repository (past and future) into a git repository? What do you think – will it help you if you are able to look at older revisions of my SlackBuild scripts? Some time ago I started copying the SlackBuild script into the documentation directory of every Slackware package I create, but I realize that once I release a newer package, the older scripts disappear from public view.

Leave your opinions in the comments section below please.

Eric