Main menu:

Sponsoring

Please consider a small donation:

 

 

Or you can donate bitcoin:

 

Thanks to TekLinks in Birmingham, AL, for providing colocation and bandwidth.

Page Rank

Fame

FOSS Force Best Blog--2013 Award

Recent posts

Recent comments

About this blog

I am Eric Hameleers, and this is where I think out loud.
More about me.

Search

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 395 other subscribers

My Favourites

Slackware

Calendar

September 2018
M T W T F S S
« Aug    
 12
3456789
10111213141516
17181920212223
24252627282930

RSS Alien's Slackware packages

RSS Alien's unofficial KDE Slackware packages

RSS Alien's multilib packages

RSS Slackware64-current

RSS SBo

Meta

liveslak-1.3.0 with speed improvements

blueSW-64pxThere was no August release of a Plasma5 Live ISO as you will probably have noticed. The reason was that around the time when I released the August update of Plasma5 for Slackware, I was working on new liveslak functionality and wanted to finish that before releasing new ISOs. The testing took some more time than I anticipated due to increased work load in my day job. But I finished what I wanted to have in a new liveslak release, and today I want to write a post about the new stuff.

To accompany the new liveslak-1.3.0 I have uploaded fresh ISO images for the Slackware Live Edition. They are based on the latest Slackware-current dated “Fri Sep 7 23:00:06 UTC 2018″.
The available ISO variants on https://slackware.nl/slackware-live/latest/ are:

  • Full unmodified Slackware (64bit).
  • Stripped-down XFCE (32bit as well as 64bit), this ISO will fit on a CDROM medium.
  • Slackware with MATE 1.20 instead of KDE4 (64bit). Thanks to Willy Sudiarto Raharjo for the packaging.
  • Slackware with Plasma 5 instead of KDE4 (64bit) to showcase the KDE Plasma 5_18.09 desktop. This ISO also contains Calibre 3.30.0, Chromium 69, LibreOffice 6.1.0, and VLC 3.0.4 among many others.

The new liveslak version 1.3.0 has several memorable updates:

  • Support for a new compression tool ‘zstd’ that will increase the speed of extracting squashfs modules greatly (and ‘paying’ for the increased decompression speed with an increase of compressed size with around 10%). The “make_slackware_live.sh” script was enhanced with a new commandline parameter “-c” with which you can indicate a non-default compressor (xz being the default and zstd, gzip, lzo as the alternatives).
    Raw decompression speed is up to 5 times faster using zstd compared with an xz-compressed squashfs modules, but due to the nature of the storage medium, OS kernel and program execution times, the observed speed gains for the actual Slackware Live Edition vary from 20% to 80%. Largest speed gains are found when you boot a Live ISO in a virtual machine; the smallest speed gains will be found when you boot Slackware Live from a USB medium where the medium’s read speed is the limiting factor.
  • During ISO creation you can now specify your own custom default country/language. The script default is still “us” but you can select any of the other languages that are supported on boot, for instance to have a Live OS that boots into German localization and language settings without any input.
    A new commandline parameter “-l” to the “make_slackware_live.sh” script enables you to specify the ISO default language.
  • Due to changes in package lists (mostly adding new packages introduced in slackware-current) make it hard to keep the XFCE ISO below 700 MB. That was not different this time. Continuous pruning in the filesystem is unavoidable. But I think I have reached the limit of what I can cut away in relation to unneeded libraries and stuff. The ever growing footprint of Slackware-current’s applications demands that eventually I may have to start removing complete packages from the XFCE live ISO. Any thoughts as to what you find least relevant in a small ISO? Is it the GCC compiler? Is it the Asian TrueType fonts? Is it Firefox, ImageMagick, …? To me all of those are equally important and yet I may have to decide on their removal eventually.

Compression of the ISOs

I have used ‘zstd’ compression for the SLACKWARE, PLASMA5, MATE ISO images. You will notice substantially reduced boot-up times.
The XFCE images are still compressed with ‘xz’ but as a curious test, I have re-compressed the “min” module of the 64bit XFCE ISO with zstd. That increased its size with 21 MB but it’s still below CDROM size. There is a noticeable speed increase even by just using zstd on the “min” module – I get a 10% faster bootup of the XFCE Live OS in a virtual machine.

In order to keep the PLASMA5 ISO fitting on a DVD, I had to take the multilib module out. If you need multilib in a Slackware Live Edition and you are running it off a USB stick, you can simply download the module from the ‘bonus‘ directory on the mirrror server, and copy it to the “/liveslak/addons/” directory on the Linux partition of the USB stick so that it will load automatically when the Live OS boots.

Wayland?

The ‘testing’ branch in my ‘kown’ repository is currently identical to the ‘latest’ branch, so there is no Wayland support in it now. For a future ‘testing’ release I’ll most likely re-visit Wayland but I want Patrick to add Plasma 5 to Slackware first so I can do my own stuff in just the ‘latest’ branch again and use ‘testing’ for actual tests.

Where to get the ISOs

Some download locations (mirrors may need 24 hours to catch up) for the Live ISOs are:

A copy of the liveslak sources and scripts can be found here:

You can follow the liveslak development in git: https://git.slackware.nl/liveslak/

The use of zstd

The ISO images which I created with zstd compression are all using Slackware-current. Because zstd support for squashfs was added to the Linux 4.14 kernel and Slackware is using these kernels. no modifications were required for the Live ISOs to work with this new compression type. A package for ‘zstd’ or a recompilation of ‘squashfs-tools’ to add zstd support is only needed when creating the ISO. When the Live OS boots, the Linux kernel  takes care of the compressed squashfs filesystem transparently.

In order to extract data from a zstd-compressed squashfs module you will of course need a squashfs-tools package with support for zstd. Therefore I have both a ‘zstd‘ and a ‘squashfs-tools‘ package for Slackware-current in my repository. I am not providing these for Slackware 14.2 because its older kernel (4.4.x) is not supporting zstd anyway.

Refreshing your USB stick instead of re-formatting

If you already use a Slackware Live USB stick that you do not want to re-format, you should use the “-r” parameter to the “iso2usb.sh” script. The “-r” or refresh parameter allows you to refresh the liveslak files on your USB stick without touching your custom content. If you want to modify other parameters of your USB stick, use the script “upslak.sh“. It’s main feature is that it can update the kernel on the USB stick, but it also can replace the Live init script. As with most (if not all) of my scripts, use the “-h” parameter to get help on its functionality.

Historical info on liveslak

More detail about the features of Slackware Live Edition can be found in previous posts here on the blog.

Have fun!

Comments

Comment from Marcus_777
Posted: September 9, 2018 at 20:40

Glory, prayers have been dispatched. I do not have to spam the forum and write how xz compression is slow.

Comment from alienbob
Posted: September 9, 2018 at 20:55

Marcus_777 since liveslak was not conceived as a Live OS for your daily work, speed of decompression has never been more important to me than compression factor. To squeeze as much as possible into an ISO image you have to make a compromise and sometimes it means sacrificing speed.
But in any case, from now on the liveslak scripts allow you to choose what works best for you.

Comment from Antoine
Posted: September 9, 2018 at 21:09

Thank you for your good work Eric.

“In order to keep the PLASMA5 ISO fitting on a DVD, I had to take the multilib module out.”
Does the straight Slackware Current liveslack ISO include multilib?

What exactly is preserved by the -r parameter of iso2usb.sh? If I install multilib and say, a 32-bit acroread, would they be preserved next time I update my USB stick with your next release of liveslack?

Comment from alienbob
Posted: September 9, 2018 at 21:33

Antoine the meaning of “Full unmodified Slackware” eludes you? There is no multilib in that ISO.
In fact, the only ISO that ever contained a multilib module was the PLASMA5 ISO.
And as of today, the multilib module is external to the ISOs and has to be downloaded separately. Needless to say (hopefully) that this multilib module works just as well with the SLACKWARE, MATE or any other Live OS as it does with PLASMA5,

Comment from alienbob
Posted: September 9, 2018 at 21:44

Antoine the “help” text of the “iso2usb.sh” script reads as follows:

# -r|–refresh Refresh the USB stick with the ISO content.
# No formatting, do not touch user content.

This means in a more bloated textual form: whatever you added to the USB stick will not be deleted. The script will replace the content of /liveslak/system, /EFI and /boot with the content found on the ISO.

The script will also examine the initrd of the USB stick and extract some information from it which then gets applied to the new initrd file which is copied from the new ISO to your USB stick (which LUKS volumes you are using, what the name is of your persistence and liveslak directories, what the USB wait time is).

This means for instance, if you have a (possibly encrypted) persistent homedirectory, a persistence file for the OS, and perhaps added some extra squashfs modules to /liveslak/addons and have several files in /liveslak/rootcopy that are applied on top of the Live OS, then *nothing* of that gets touched when using the “-r” parameter.

I have been ‘refreshing’ my own Plasma5 USB stick for many releases now, since ever I added that “-r” parameter I have not re-formatted. That USB stick is on my key chain, I carry it around everywhere and show people the latest Slackware with all the bells & whistles if they are curious.

Comment from Marcus_777
Posted: September 9, 2018 at 21:44

I understand. For me it’s great improvements. I use LiveSlack as installation medium. Installation takes too long with xz. Now I expect a lot of improvement. Image size is not important to me. Especially at today’s USB flash drive prices. Thanks for addition this feature, Eric 🙂

Comment from alienbob
Posted: September 9, 2018 at 21:47

Marcus_777 I have not tested “setup2hd” but I expect a big speed improvement with the new ISO images and their zstd compression.
Like I said, initially liveslak was not conceived as a daily driver but the project has evolved a lot since those humble beginnings. Getting better speeds is more important now than it was way back, so the 10% size increase of the ISOS is something I gladly trade for the speed boost.

Comment from Antoine
Posted: September 9, 2018 at 21:51

Thanks Eric for your patience and for your detailed answers to my dumb questions.

Comment from alienbob
Posted: September 9, 2018 at 21:55

Antoine, not “dumb”. When there’s questions about my scripts then the scripts are obviously not documented well enough. I hope it makes more sense to you now?

Comment from Antoine
Posted: September 9, 2018 at 22:07

Yes, it makes sense, I am just not familiar with the structure and inner workings of liveslack. I will now go off and read the setup2hd help before asking any more questions. 🙂

Comment from USUARIONUEVO
Posted: September 10, 2018 at 01:49

And , why no test xz with blocksize 256k ?

mksquashfs -Xbcj ia64 -comp xz -b 256k

I use this in 64bits , and memory usage goes down , and the size is the same as stdz 1M 19 , but stdz ,use double of memory for same size of xz.

Test using blocksizes 256k or 128k , and see.

Comment from Emmanuel Roque
Posted: September 10, 2018 at 03:36

Hi Eric, thank you for all your work!

On Slackware 14.2 I used the new “iso2usb.sh” script to write the ISO to my USB stick, but I’m unable to boot, the BIOS does not recognize the USB as a bootable device, tested on 2 different laptops.
I understood that slackware-current was just needed to create the ISO, or is it mandatory for the correct functionality of the new scripts?

Thanks in advance.

Comment from Marcus_777
Posted: September 10, 2018 at 11:03

USUARIONUEVO : Memory usage is not critical for me. It is speed of decomression. I am testing all block sizes. From 4096 to 1048576. As you say, memory usage was double, but speed of decomp is up to 5 times faster. I also found out, that images with larger block sizes uncompress faster as a whole, they may introduce more latency on live media since a whole block will need to be uncompressed even if you’re just reading just 1 byte from a file.

Comment from alienbob
Posted: September 10, 2018 at 11:10

Emmanuel you are partly correct.
The iso2usb.sh script can be used on Slackware 14.2 to copy a Live ISO to a USB stick. That has not changed.

However with the latest set of ISOs there is one caveat: the script requires ‘unsquashfs’ if you use the “-c” or “–crypt” parameter to create a LUKS-encrypted container for your user’s homedirectory. The ‘unsquashfs’ tool on Slackware 14.2 is not able to read the new zstd-compressed squashfs modules. Therefore the script will fail if you use “-c” to create an encrypted /home.

I think it is good to add zstd and a rebuilt squashfs-tools to my 14.2 repository, just for this requirement (creation of a Live ISO with zstd compression will still be inpossible on Slackware 14.2).

Returning to your question: the ‘iso2usb.sh’ script should have produced a working bootable USB stick for you.
Did the script produce any errors while it was running?
Can you provide the *exact* commandline you have used to copy the ISO contents to your USB stick?
What does “blkid” show you for the partitions of the USB stick?
What does “fdisk -l” show about that USB stick?

Pingback from Links 10/9/2018: Nano 3.0, Nextcloud 14, Elive 3.0 and Git 2.19 Released | Techrights
Posted: September 11, 2018 at 01:27

[…] liveslak-1.3.0 with speed improvements […]

Comment from Ricardo J. Barberis
Posted: September 11, 2018 at 05:47

Nice additions to liveslak, and very interesting discussions.

I’m commenting just to subscribe and not to miss more comments 🙂

Comment from Alex
Posted: September 11, 2018 at 13:42

Hi Eric,
first of all: Thanks again!
I ran into the problem with -c on 14.2, that you describe.
Command line:

sh iso2usb.sh -i slackware64-live-plasma5-current.iso -o /dev/sdc -c 70%

// Last few lines of output:
Writing superblocks and filesystem accounting information: done

tune2fs 1.43.1 (08-Jun-2016)
Setting maximal mount count to -1
Setting interval between checks to 0 seconds
Setting reserved blocks percentage to 0% (0 blocks)
— Copying ‘/home’ from LiveOS to container…
Filesystem uses unknown compression, this is unsupported by this version
Decompressors available:
gzip
lzma
lzo
xz

Result: USB stick is not bootable.
Everything works just fine without -c or when I use dd to create the stick, except that I don’t have an encrypted /home, then, of course. 😉

Comment from Alex
Posted: September 11, 2018 at 13:46

Addition to my previous post:
zstd and a rebuilt squashfs-tools package for 14.2 would be much appreciated!

Comment from alienbob
Posted: September 11, 2018 at 14:23

Alex, those 14.2 packages are already available in my repository.
After installing/upgrading those two packages, the “-c” parameter will work as expected for you.

A liveslak ISO based on Slackware 14.2 on the other hand, will still *not* work because the older kernel does not support sztd in squashfs and therefore is unable to mount the squashfs modules into a filesystem.

Comment from Alex
Posted: September 12, 2018 at 03:04

Thanks, Eric: With the two updated packages it works great and I am writing this from Slackware Live Edition with LUKS encrypted /home.
As you write, the boot time is noticeably shorter than with previous versions.

Comment from Emmanuel Roque
Posted: September 12, 2018 at 03:08

Hi Eric.
Thank you for your answer, I misunderstood what you wrote on your post and I was trying to use the -c option on Slackware 14.2, that’s why it didn’t work, sorry, my mistake. I downloaded the zstd and squashfs-tools packages from your 14.2 repository and I made a bootable USB Stick, however, it does not work properly on every computer. I first used the USB on an older Dell Desktop computer, and after selecting “Start Slackware-Live …” on the boot menu, the computer restarts to the same screen, the second time it loads the system properly and everything works flawlessly , but on my laptop (Dell Inspiron 7567), after selecting “start Slackware-Live …” the computer just reboots, and this cycle continues, so I’m unable to boot on my laptop. Secure boot is disabled on the BIOS. Any ideas about how can I fix this?

Thanks again in advance.

Comment from Luke
Posted: September 12, 2018 at 16:28

Thanks so much for all your work! I can’t wait to test this. Did I see something on the LQ forums about Patrick potentially removing KDE from Slackware 15 in favor of XFCE, similar to what happened with GNOME after Slackware 10? Do you think this is likely to happen?

Comment from alienbob
Posted: September 12, 2018 at 17:15

Luke, KDE4 has been out of support for two years now, except for the kdelibs package but that too has not received any bugfix update since December 2017.
It is safe to say that we will say good-bye to KDE4.

XFCE is already part of Slackware, so this is not a question of replacing KDE4 with XFCE. Plasma5 is the logical candidate to replace KDE4.

Write a comment