Setting up Jack Audio in Slackware

Note: this article has been superseded by the (much less complex) instructions in a newer article “Configuring Slackware for use as a DAW“.

If you are using your computer as a Digital Audio Workstation (DAW) then the ALSA sound subsystem is just not up for the task. Musicians and audio professionals prefer to use Jack Audio on Linux. Let me quote from the Jack Audio web site:

What is JACK?

Have you ever wanted to take the audio output of one piece of software and send it to another? How about taking the output of that same program and send it to two others, then record the result in the first program? Or maybe you’re a programmer who writes real-time audio and music applications and who is looking for a cross-platform API that enables not only device sharing but also inter-application audio routing, and is incredibly easy to learn and use? If so, JACK may be what you’ve been looking for.

JACK is system for handling real-time, low latency audio and MIDI.

We “ordinary” users of Slackware don’t usually have a need for Jack. It makes your computer’s sound subsystem more complex, meaning that more things can go wrong (where you end up with distorted or even no sound) and that fixing things requires more knowledge.

However there are cases even for non-musicians to want to install Jack Audio. I am one of them. As I explained in my previous post, I want to record videos of programs running on my desktop, along with the comments I may record through a microphone. When I selected SimpleScreenRecorder as my tool for doing this, I found out that it needs Jack in order to record the audio (but come to think of it… after reading bradpit’s comment in the previous post I realize that there may be a way around Jack – something I will check out soon and report if I find anything worth mentioning).

This article is meant to show you how to install and configure Jack Audio, and how to configure ALSA so that applications will still have sound even if they are unaware of Jack (Jack grabs the computer’s sound hardware and won’t allow ALSA applications to use it).

The article will center around you, being the one behind the physical computer. No system file needs to be changed, all configuration is done in your own home directory, for your use only. If someone else logs in, he or she will not be bothered by Jack and everything will work as before.

 

Installing Jack Audio

In order to install Jack Audio, you need the following packages: jack and qjackctl.

The qjackctl package contains the “de facto” configuration utility for Jack Audio, also called qjackctl. It is a Qt-based graphical program which allows you to configure “several JACK daemon parameters, which are properly saved between sessions, and a way control of the status of the audio server daemon” according to the program description. It also allows you to configure and autoload the patchbay and offers full connection control.

qjackctl_main

Configuring ALSA

The goal is to have a configuration where ALSA applications can access a “audio hardware” device even when the real device is locked by Jack Audio. That way, ALSA applications will not complain about unavailable audio hardware. I will show you how to provide ALSA with such a virtual hardware, and ensure that all sound which goes into that virtual hardware will be picked up by Jack and played through your speakers (hence the phrase “bridged”).

This “virtual hardware” is provided by the ALSA loop driver. When loaded into the kernel, this driver provides a pair of cross-connected devices, forming a full-duplex loopback soundcard.

First: load the kernel module (as root)

# /sbin/modprobe snd-aloop

You should add the above command-line to the file “/etc/rc.d/rc.modules” or to “/etc/rc.d/rc.local” so that the module will be loaded automatically on every boot.

The driver creates 8 independent substreams by default, but we need only two. Therefore you can add the following line to a (new) file called “/etc/modprobe.d/alsaloop.conf”:

options snd-aloop pcm_substreams=2

Next: write an ALSA configuration file which uses the new loopback devices

Create your ~/.asoundrc file as follows – if this file exists in your home directory, please back it up first!

# ------------------------------------------------------
# hardware 0,0 : used for ALSA playback
pcm.loophw00 {
  type hw
  card Loopback
  device 0
  subdevice 0
  format S32_LE
  rate 44100
}

# ------------------------------------------------------
# hardware 0,1 : used for ALSA capture
pcm.loophw01 {
  type hw
  card Loopback
  device 0
  subdevice 1
  format S32_LE
  rate 44100
}

# ------------------------------------------------------
# playback PCM device: using loopback subdevice 0,0
pcm.amix {
  type dmix
  ipc_key 196101
  slave {
    pcm "loophw00"
    buffer_size 8192
    period_size 4096
    periods 2
  }
}

# capture PCM device: using loopback subdevice 0,1
pcm.asnoop {
  type dsnoop
  ipc_key 196102
  slave {
   pcm loophw01
   period_size 4096
   periods 2
  }
}

# ------------------------------------------------------
# software volume
pcm.asoftvol {
  type softvol
  slave.pcm "amix"
  control { name PCM }
  min_dB -51.0
  max_dB   0.0
}

# ======================================================

# ------------------------------------------------------
# duplex device combining our PCM devices defined above
pcm.aduplex {
  type asym
  playback.pcm "asoftvol"
  capture.pcm "loophw01"
  hint {
    description "ALSA->JACK Loop Bridge"
  }
}

# ======================================================

# ------------------------------------------------------
# Mixer control definitions to keep JACK and some other apps happy
ctl.amix {
    type hw
    card Loopback
}

ctl.asnoop {
    type hw
    card Loopback
}

ctl.aduplex {
    type hw
    card Loopback
}

# ======================================================

# ------------------------------------------------------
# for jack alsa_out: looped-back signal at other end
pcm.ploop {
  type hw
  card Loopback
  device 1
  subdevice 1
  format S32_LE
  rate 44100
}

# ------------------------------------------------------
# for jack alsa_in: looped-back signal at other end
pcm.cloop {
  type hw
  card Loopback
  device 1
  subdevice 0
  format S32_LE
  rate 44100
}

# ======================================================

# ------------------------------------------------------
# default device

pcm.!default {
  type plug
  slave.pcm "aduplex"
}

When you save that file, its content will re-define your ALSA configuration with immediate effect. KDE may complain about hardware that was added or went missing, you can ignore that for now.

What these definitions do for ALSA, is to create a new full-duplex PCM device called “pcm.aduplex” with a description (which you will see mentioned in your programs’ ALSA device selectors) of “ALSA->JACK Loop Bridge”. What these definitions also do, is to create additional PCM devices for capture (pcm.cloop) and playback (pcm.ploop) which we will connect Jack to. That way, your ALSA applications are going to pipe their audio into one end of the loopback device and Jack will see this as incoming audio and play it on your speakers.

 You can test your new ~/.asoundrc file even though you will not hear a thing because the new virtual device is not yet connected to a real audio device (we will come to that in the next section). If you run the following command (under your own account – not as root) you should not see any error message:

$ aplay /usr/share/sounds/alsa/Front_Center.wav

Playing WAVE ‘/usr/share/sounds/alsa/Front_Center.wav’ : Signed 16 bit Little Endian, Rate 48000 Hz, Mono

If you see errors instead of the above text, then there is something wrong with the ~/.asoundrc file you just created.

Configuring Jack Audio

You should use qjackctl to configure the jack daemon. Jack installs a D-Bus service which qjackctl will connect to. Qjackctl can launch the daemon by itself or attach to an already running jack daemon (the qjackctl tray icon will be green if it had to start jackd and orange if it connected to an already running jackd).

qjackctl_setup_misc

Qjackctl will write the jack daemon configuration to a file in your homedirectory: ~/.jackdrc .It will write its own configuration to a different file: ~/.config/rncbc.org/QjackCtl.conf

I found out that I needed to have a sampling frequency of 44100 (the Jack default) instead of what musicians usually use (48000) in order to prevent distorted sounds coming from ALSA applications (youtube flash videos!). Whatever frequency you choose, you will need to use the same sampling frequency in ~/.asoundrc (see above) and for jackd.

qjackctl_setup

Using qjackctl, you can easily configure jack to use the soundcard hardware to which your speakers are connected: it shows a list of available devices in a dropdown menu. If you want to construct the jackd command-line manually, the names of all hardware devices can be obtained by running the following command:

$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: NVidia [HDA NVidia], device 0: VT1708S Analog [VT1708S Analog]
  Subdevices: 2/2
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
card 0: NVidia [HDA NVidia], device 1: VT1708S Digital [VT1708S Digital]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

The content of my ~/.jackdrc file looks like this:

/usr/bin/jackd -r -m -dalsa -r44100 -p1024 -n2 -m -H -D -Chw:NVidia -Phw:NVidia

The various jackd parameters all correspond to selections in the qjackctl dialog window.

Now when jack starts, it will still not know what to do with our modified ALSA configuration. A few commands need to be executed to form a bridge between ALSA and Jack using  the loopback driver. This is done most easily by creating a script file (for instance as user root create the file “/usr/local/bin/loop2jack” so that other users of your PC can use it as well) and in that file, add the following:

#!/bin/sh
#
# script loop2jack, located in /usr/local/bin
#
# Start jack if it is not already running:
/usr/bin/jack_control start
# loop client creation
/usr/bin/alsa_out -j ploop -dploop -q 1 2>&1 1> /dev/null &
/usr/bin/alsa_in -j  cloop -dcloop -q 1 2>&1 1> /dev/null &
# give it some time before connecting to system ports
sleep 1
# cloop ports -> jack output ports
/usr/bin/jack_connect cloop:capture_1 system:playback_1
/usr/bin/jack_connect cloop:capture_2 system:playback_2
# system microphone to "ploop" ports
/usr/bin/jack_connect system:capture_1 ploop:playback_1
/usr/bin/jack_connect system:capture_2 ploop:playback_2
# done
exit 0

Don’t forget to make the script executable:

chmod +x /usr/local/bin/loop2jack

This script should to be started after jackd has been started, but if jackd is not yet running, the script will start it for you. You can run it manually to see if there are any errors in your ~/.asoundrc file that you have to fix first. You can use qjackctl to run this script automatically when it starts:

qjackctl_setup_options

In the above screenshot you will also notice how the patchbay definitions are made persistent – I will show why this is important when I get to the section on SimpleScreenRecorder.

 

Starting Jack Audio

You will want to start the  jack daemon before any of your ALSA applications start! The ALSA driver grabs the audio hardware as soon as an application tries to make a sound using ALSA. Desktop Environments like KDE and XFCE have a feature called “session restore” which means that programs which were running when you logged out, will be started again – automatically – when you login again. A program like Skype, or even the startup sounds of your KDE desktop, will prevent Jack from launching properly.

If you are in runlevel 4 (graphical login) then you can start jackd by adding an appropriate command line in the following file: ~/.xprofile

If you are in runlevel 3 then you can start jackd from your normal login, using the file: ~/.profile

For instance, I have added this text to my ~/.xprofile :

# Create the bridge between ALSA applications and JACK output:
/usr/local/bin/loop2jack

… which will solve all timing issues: the loop2jack script will start the Jack daemon if it was not yet running, and it will bridge the ALSA and Jack in- and output channels using our virtual loopback device. This way, qjackctl does not even have to start anything (but we will not change the qjackctl configuration, in case you need to stop and restart Jack during a desktop login session).

 

Caveats

You have to keep in mind that running Jack is now required after making these modifications to your ALSA setup! When Jack is stopped (or not started) your ALSA sound system is no longer bridged to the physical sound card and you’ll hear nothing.

Also note that you may have a system-wide ALSA configuration file “/etc/asound.conf” which may interfere with your setup. Your own ~/.asoundrc definitions are added on top of the definitions in /etc/asound.conf and do not replace them.

 

Reverting this bridged ALSA-Jack setup

Reverting to the original Jack-less configuration is easy.

  1. Stop the Jack daemon using the qjackctl menu, and then stop qjackctl

  2. Remove or rename the file ~/.asoundrc (and restore your backup if you had an earlier version of this file)

  3. Remove or rename the file ~/.jackdrc

  4. Remove the invocation of “/usr/local/bin/loop2jack” from your ~/.xprofile and/or ~/.profile scripts.

The steps (1) and (2) are sufficient to restore the default ALSA behaviour and steps (3) and (4) ensure that Jack will not interfere at next login.

Getting audio into SimpleScreenRecorder (SSR)

In the “connections” window of qjackctl, my computer’s audio layout looks like this when I have started SimpleScreenRecorder (read my earlier post about this video recording tool):

qjackctl_connections_ssr

It’s a fairly simple layout with just a single Jack input client (being SSR) and a single output client (the entry “alsa-jack.jackP5161.0” which is the VLC program playing music).

The connections you see here between Jack’s “monitor” outputs (which were activated by checking the “Monitor” box in qjackctl’s setup dialog) and SSR’s inputs are automatically created when SSR connects to the Jack server. This happens because I defined these connections in qjackctl’s “patchbay” and let them be automatically activated. It seemed to be necessary this way because I could not get the monitor’s output to start inputting into SSR except by creating manual connections everytime.

qjackctl_patchbay_ssr

This is how I was finally able to record videos of (programs running on) my Slackware Linux desktop. Feedback highly appreciated!

Eric

 

A simpler solution (but not as robust)

There is another way to route the audio to and from ALSA-using applications when Jack is the sound server. In http://jackaudio.org/routing_alsa you can see how to use the “ALSA Jack PCM plugin” which creates a new PCM type called “jack”. This is a lot simpler to setup than the above article but it is not as robust and flexible. I used this solution for a little while and was happy with it – until I found out that it will not enable me to record audio with the SimpleScreenRecorder. So I had to abandon it.

You will need my alsa-plugins package for this. This package has two dependencies: jack (naturally) and ffmpeg. Note that if you want to use alsa-plugins and you are running 32-bit software which plays or records audio (think of Skype, Steam games) and you are on a 64-bit Slackware system, then you will additionally need multilib (of course) as well as “compat32” versions of the 32-bit ffmpeg and jack packages.

 

References

These pages have been helpful (most important info in the first link):

Slackware Cloud Server Series Episode 6: Etherpad with Whiteboard

Hi all!
This is the 6th episode in a series I am writing about using Slackware as your private/personal ‘cloud server’. It is an unscheduled break-out topic to discuss an Etherpad server specifically.
Check out the list below which shows past, present and future episodes in the series, if the article has already been written you’ll be able to click on the subject.
The first episode also contains an introduction with some more detail about what you can expect.
These articles are living documents, i.e. based on readers’ feedback I may add, update or modify their content.

Etherpad with Whiteboard

In Episode 3 (Video Conferencing) we setup a Jitsi Meet server in a Docker container stack which includes an Etherpad server for real-time document collaboration during a video meeting.
That Etherpad instance as configured by the Docker-Jitsi-Meet project is really only a demo setup. It uses a “dirtydb” JSON backend wich is not meant for anything else but testing. It really needs a proper SQL database like MariaDB to power it. And you can’t export your documents from this demo Etherpad in any meaningful format.
Furthermore, this Etherpad container is not using our Keycloak IAM for authentication; everyone who knows the public URL can create a document, invite others and start writing. Even shared documents created in Jitsi meetings are not secure and anyone who guesses the room name has access to the Etherpad document.

This article means to set things right and configure Etherpad correctly, adding Whiteboard functionality as we go. I will also discuss the differences between our Jitsi integrated Etherpad and a running a standalone Etherpad server in case you are not interested in Video meetings and only want the text collaboration.

Preamble

This article assumes you have already setup an Etherpad in a Docker container as part of a Dockerized Jitsi Meet server (see Episode 3 in this series), and this Etherpad is running at a publicly accessible URL:

  • https://meet.darkstar.lan/pad/

We want to make it delegate user authentication to our OpenID Provider: Keycloak. That Keycloak service is available at:

  • https://sso.darkstar.lan/auth

If you are not interested in Jitsi Meet and only want to know how to run an Etherpad server, this article still contains everything you need but keep in mind that my examples are all assuming the above URL for the Etherpad. Adapt that URL to your own real-life situation. You may still have to setup an Apache webserver first, which serves an empty page at “https://meet.darkstar.lan/” but I will leave that to you.

Configuring MariaDB

By default, Etherpad will use a ‘DirtyDB’ JSON file-based backend. It is straight-forward to make it switch to for instance a MariaDB database server backend, we only need to provide the connection details for a pre-existing database.
Like with the previous articles we are using the Slackware MariaDB database server which is running on the host. First, we will create a database (etherpad_db), a database user (etherpad) and grant this user sufficient access to the database. Then we will use these database configuration values when editing the Docker-Jitsi-Meet files in order to change the Etherpad container properties.
This is how we create the database and the user (using a secure password string for ‘EPPPASSWD‘ of course):

$ mysql -uroot -p
> CREATE DATABASE IF NOT EXISTS `etherpad_db` CHARACTER SET utf8 COLLATE utf8_unicode_ci;
> CREATE USER 'etherpad'@'localhost' identified by 'EPPASSWD';
> CREATE USER 'etherpad'@'%' identified by 'EPPPASSWD';
> GRANT CREATE,ALTER,SELECT,INSERT,UPDATE,DELETE on `etherpad_db`.* to 'etherpad'@'localhost';
> GRANT CREATE,ALTER,SELECT,INSERT,UPDATE,DELETE on `etherpad_db`.* to 'etherpad'@'%';
> FLUSH PRIVILEGES;
> exit;

Note from the above SQL statements that we are allowing the ‘etherpad‘ user remote access to the database. This is needed because Etherpad in the Docker container contacts MariaDB via the network, using the IP address of the Docker network bridge in the Jitsi Meet container stack.

Reconfiguring Docker-Jitsi-Meet

My advise is to start with briefly re-visiting Episode 3 of the series and read back how we customized the ‘docker-compose.yml‘ and ‘.env‘ files in order to startup the Docker-Jitsi-Meet stack properly. Because we are going to update these two files again.
This is what we need to change to make Etherpad connect to the external MariaDB database:

Relevant .env additions:

# MariaDB parameters for mysql DB instead of dirtydb
ETHERPAD_DB_TYPE=mysql
ETHERPAD_DB_HOST=172.20.0.1
ETHERPAD_DB_PORT=3306
ETHERPAD_DB_NAME=etherpad_db
ETHERPAD_DB_USER=etherpad
ETHERPAD_DB_PASS=EPPASSWD
ETHERPAD_DB_CHARSET=utf8

Relevant docker-compose additions:

In the ‘.env‘ file we defined the IP address for the database server (172.20.0.1). Etherpad is running inside a container, and its way out is through the default gateway of its Docker network. In order to have 172.20.0.1 as the gateway address, we need to configure the internal ‘meet.jitsi‘ network a deterministic IP range so that we always know its gateway address. if we are going to give that network the IP range “172.20.0.0/16“, the “networks” statement all the way at the bottom needs to be changed from:

# Custom network so all services can communicate using a FQDN
networks:
  meet.jitsi:

to:

# Custom network so all services can communicate using a FQDN 
networks: 
  meet.jitsi: 
    ipam: 
      config: 
        - subnet: 172.20.0.0/16

Use the variables we added to ‘.env’ to create an updated Etherpad container definition. Right underneath this line:

            - SKIN_VARIANTS=${ETHERPAD_SKIN_VARIANTS}

Add the following lines:

            - DB_TYPE=${ETHERPAD_DB_TYPE} 
            - DB_HOST=${ETHERPAD_DB_HOST} 
            - DB_PORT=${ETHERPAD_DB_PORT} 
            - DB_NAME=${ETHERPAD_DB_NAME} 
            - DB_USER=${ETHERPAD_DB_USER} 
            - DB_PASS=${ETHERPAD_DB_PASS} 
            - DB_CHARSET=${ETHERPAD_DB_CHARSET} 

Accessing the admin console

Etherpad has an admin console where you can manage its plugin configuration and other things too. It will only be enabled if you configure an admin password. So let’s do that too.
This is what we need to change to enable the admin console for Etherpad:

Relevant .env additions:

# The password for Etherpad admin page
ETHERPAD_ADMIN_PASSWORD="my_secret_admin_pass"

Relevant docker-compose additions:

Right underneath this line:

            - SKIN_VARIANTS=${ETHERPAD_SKIN_VARIANTS}

Add the following lines:

            - ADMIN_PASSWORD=${ETHERPAD_ADMIN_PASSWORD}

Relevant Apache httpd additions:

If you would now access the URL for the admin console, https://meet.darkstar.lan/pad/admin/ you would only see the message “Unauthorized“. The Etherpad expects you to provide the Basic Authentication hook in front of that page which passes the admin credentials on to the backend. So, we will add a ‘AuthType Basic‘ block to our Apache httpd configuration to add Basic Authentication which will pop up a login dialog, and then add the admin user and its password “my_secret_admin_pass” to a htaccess file.

Remember, in Episode 4 we configured the Etherpad to be available at “https://meet.darkstar.lan/pad/” which means the admin console URL is “https://meet.darkstar.lan/pad/admin/
This is the block to add to your VirtualHost configuration for the Etherpad:

<Location /pad/admin>
    AuthType Basic
    AuthBasicAuthoritative off
    AuthName "Welcome to the Etherpad"
    AuthUserFile /etc/httpd/passwords/htaccess.epl
    Require valid-user
    Order Deny,Allow
    Deny from all
    Satisfy Any
</Location>

And then we still need to create that htaccess file using the ‘htpasswd‘ tool, like this:

# mkdir /etc/httpd/passwords
# htpasswd -B -c /etc/httpd/passwords/htaccess.epl admin

The “-B” parameter enforces the use of bcrypt encryption for passwords. This is currently considered to be very secure.
The above command will prompt for the password, and there you enter that “my_secret_admin_pass” string. The content of that file will look like this:

# cat /etc/httpd/passwords/htaccess.epl
admin:$2y$05$JpvucTlKIQEJViCynem.JelENHpv/maJStsPM4iU9d/sg4cMU.UfW

The Docker Jitsi Meet container stack needs to be refreshed and restarted since we edited ‘.env’. I don’t want to repeat the detailed instructions here, so refer you to the section “Considerations about the “.env” file” in Episode 3 of this article series. Do that now, and when the updated container stack is up and running again, continue here.

And after also restarting the Apache httpd and refreshing the URL “https://meet.darkstar.lan/pad/admin/” you will be asked to enter your admin credentials and you will end up in the Etherpad admin console.
The screenshot below does not reflect the status of the barebones Etherpad by the way; you see a lot of installed plugins mentioned on the admin page. We will be installing those into the Etherpad image in one of the next sections:

At this stage, we have accomplished a well-performing Etherpad installation with a SQL database back-end and an administrative web-interface. The next step is to add authentication through an OpenID provider like our Keycloak IAM server.

Integrating with Keycloak IAM

The out-of-the-box Etherpad Docker container is not very functional. The above sections already showed how to replace the “DirtyDB” with a proper SQL database server like MariaDB. But the default image misses a few useful plugins and a real desktop editor program which allows Etherpad users to export their collaborative work to a proper document format instead of pure HTML.

Whatever plugins we add, at the very least we need to add a plugin which allows us to let the Etherpad authenticate against our Keycloak IAM server. This plugin needs to be inside the Docker image, we cannot use it outside the running container. There’s no other option than to create a custom Docker image for Etherpad. We use this as an opportunity to add some more plugins, as well as Abiword (to enable document export in Etherpad).

I’ll show how to announce Etherpad to Keycloak (we create a Client profile in Keycloak); then I’ll share the required configuration to be added to the Etherpad Docker files; and then I’ll show how to create a custom Docker image enriched with additional plugins which we will use instead of the basic image from Docker Hub.

Keycloak

First of all, let’s create a Client profile for Etherpad in Keycloak.

  • Login to the Keycloak Admin Console (https://sso.darkstar.lan/auth/admin/)
    • Select our ‘Foundation‘ realm from the dropdown at the left.
    • Under ‘Clients‘, create a new client:
      ‘Client ID’ = “etherpad
      ‘Root URL’ = “https://meet.darkstar.lan/pad/ep_openid_connect/callback
      Note that for the Etherpad OIDC plugin ‘ep_openid_connect’  – see below – to work, the ‘Valid Redirect URIs’  (a.k.a. callback URL) must be the concatenation of the Etherpad base URL (https://meet.darkstar.lan/pad/) plus “/ep_openid_connect/callback“. When setting the ‘Root URL‘ to the above value, the ‘Redirect URIs‘ will automatically also be set correctly to “https://meet.darkstar.lan/pad/ep_openid_connect/callback/*
    • Save.
    • In the ‘Settings‘ tab, change:
      ‘Access Type’ = “confidential” (default is “public”)
    • Save.
    • Go to the ‘Credentials‘ tab
    • Make sure that ‘Client Authenticator‘ is set to “Client Id and Secret
    • Copy the value of the ‘Secret‘, which we will use later in the Etherpad connector; the Secret will look somewhat like this:
      2jnc8H6RH9jIYMXExUHA7XF7uD8YKIRs“.

Keycloak configuration being completed, we can turn our attention to the connector between Etherpad and Keycloak.

EP_openid_connect

We will use the Etherpad plugin ep_openid_connect which I already briefly mentioned earlier. This plugin provides the needed OpenID client functionality to Etherpad.
When we add this plugin to the Etherpad Docker image we need to be able to configure it via the ‘docker-compose.yml‘ and ‘.env‘ files of Docker-Jitsi-Meet. The existing configuration files in the repository for the docker-jitsi-meet stack are just meant to make the basic Etherpad work, so we need to add more parameters to configure our custom Etherpad properly.
I will show you what you need to add, and where.

The ‘ep_openid_connect’ plugin expects an ‘ep_openid_connect’ block in the ‘settings.json’ file (we will get to that file in the next section). Since that file is JSON-formatted, we arrive at the following structure:

"ep_openid_connect": {
"issuer": "https://sso.darkstar.lan/auth/realms/foundation",
"client_id": "etherpad",
"client_secret": "2jnc8H6RH9jIYMXExUHA7XF7uD8YKIRs",
"base_url": "https://meet.darkstar.lan/pad"
},

The text string values in green highlight are of course the relevant ones. What is the meaning of the parameters:

  • issuer: this is the string you obtain through Keycloak’s OpenID Discovery URL. Make sure you have ‘jq‘ installed and then run this command to obtain the value for ‘issuer‘:
    $ curl https://sso.darkstar.lan/auth/realms/foundation/.well-known/openid-configuration | jq .issuer
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100  5749  100  5749    0     0   6594      0 --:--:-- --:--:-- --:--:--  6600
    "https://sso.darkstar.lan/auth/realms/foundation"

    In this case, you could easily have guessed the ‘issuer’ value, but using the above ‘well-known’ query URL will always get you the correct value.

  • client_id, client_secret: those are the same OAuth2 values obtained from Keycloak when creating the Etherpad Client profile as seen above.
  • base_url: this is the URL where Etherpad is externally accessible (https://meet.darkstar.lan/pad/ – see Episode 3).

Additionally, since we are now enforcing login, Etherpad’s ‘requireAuthentication‘ setting must be set to “true”. Note that the default setting is “false”; this is how that setting is defined in the Etherpad configuration:

"requireAuthentication": "${REQUIRE_AUTHENTICATION:false}",

We’ll just have to define a “true” value for that variable later on.

Note: Each configuration parameter can also be set via an environment variable, using the syntax "${ENV_VAR}" or "${ENV_VAR:default_value}". This ability is what we will use when updating the Docker Compose file for Jitsi Meet. We will not use the literal JSON block above, instead we will fill it with variable names and use our Docker Compose files to provide values for these variables. That way I am able to create a generic Docker image that I can upload to the Docker Hub and share with other people.
The file ‘settings.json.template‘ in the Etherpad repository has lots of examples.

Hold on to that thought for a minute while we proceed with creating our custom Etherpad Docker image, since we have all the data available to do this now. Once we have that image, we will once again return to the re-configuration of Docker Jitsi Meet and integrate our Etherpad with the Jitsi container stack.

Custom Etherpad Docker image

How to create a custom Docker image?

  • First we clone the “etherpad-lite” git repository. That contains a Dockerfile plus all the context that is needed by ‘docker build‘ to generate an image.
    $ mkdir ~/docker-etherpad-slack
    $ cd ~/docker-etherpad-slack
    $ git clone https://github.com/ether/etherpad-lite .
  • There is one relevant configuration file in the root directory of the checked-out repository:  ‘settings.json.docker‘. This file will be copied into the Etherpad Docker image and renamed to ‘settings.json‘ when we run ‘docker build‘ command. Any plugin configuration we want to enable via environment variables needs to be present in this file.
    Now the standard configurable parameters for the Etherpad are contained in that file, but our custom settings for the “ep_openid_connect” plugin are not. I already showed you how that block of configurable parameters looks in the previous section, and I promised to parametrize it. This is how the parameters look, and we will give them values in the next section where we update the Docker Jitsi Meet configuration.
  • Relevant ‘settings.json.docker’ additions:
    • Support for OpenID Connect in Etherpad – add this JSON code:
      "ep_openid_connect": {
      "issuer": "${OIDC_ISSUER:undefined}",
      "client_id": "${OIDC_CLIENT_ID:undefined}",
      "client_secret": "${OIDC_CLIENT_SECRET:undefined}",
      "base_url": "${OIDC_BASE_URL:undefined}"
      },
    • We also add a connector for the WBO Whiteboard server (its setup is described in the next section below) to the Docker image: the plugin is called ‘ep_whiteboard‘ and needs the following JSON configuration block to be added:
      "ep_draw": {
      "host": "${WBO_HOST:undefined}"
      },
    • Enable AbiWord in the configuration, since we are going to add it to the image. The full path to the ‘abiword‘ binary needs to be configured in ‘settings.json.docker‘.
      Look up this line in the file:
      "abiword": "${ABIWORD:null}",
      and change it to:
      "abiword": "${ABIWORD:/usr/bin/abiword}",
  • Then we build a new image, adding several useful (according to the developers) plugins, as well as the Abiword word processor.
    I tag the resulting image as “liveslak/etherpad” so that I can upload (push) it to the Docker Hub later on:

    $ docker build \
    --build-arg ETHERPAD_PLUGINS="ep_openid_connect ep_whiteboard ep_author_neat ep_headings2 ep_markdown ep_comments_page ep_align ep_font_color ep_webrtc ep_embedded_hyperlinks2" \
    --build-arg INSTALL_ABIWORD="yes" \
    --tag liveslak/etherpad .

This leads to the following output and results in an image which is quite a bit larger (786 MB uncompressed) as the standard Etherpad image (474 MB uncompressed) because of the added functionality:

Step 24/26 : HEALTHCHECK --interval=20s --timeout=3s CMD ["etherpad-healthcheck"]
---> Running in 2c241a795e46
Removing intermediate container 2c241a795e46
---> 5ca0246c1e61
Step 25/26 : EXPOSE 9001
---> Running in 21fcdf511d46
Removing intermediate container 21fcdf511d46
---> 1c288502f632
Step 26/26 : CMD ["etherpad"]
---> Running in 08a25b585280
Removing intermediate container 08a25b585280
---> 912c54fb6c0a
Successfully built 912c54fb6c0a
Successfully tagged liveslak/etherpad:latest

If you create the image on another computer and need to transfer it to your Slackware Cloud Server in order to use it there, you can save the image to a compressed tarball on the build machine, using docker commands:
$ docker save liveslak/etherpad | xz > etherpad-slack.tar.xz

You can use ‘rsync’ or ‘scp’ to transfer that tarball to your Cloud Server and then load it into the Docker environment there, also using docker commands so that you don’t need to know the intimate details on how Docker works with images:
$ cat etherpad-slack.tar.xz | docker load

I pushed this image to my own Docker repository https://hub.docker.com/repository/docker/liveslak/etherpad but I first added a tag to reflect the latest Etherpad release (1.8.16 at the moment):

$ docker login
$ docker tag liveslak/etherpad liveslak/etherpad:1.8.16
$ docker push liveslak/etherpad:1.8.16
$ docker push liveslak/etherpad:latest

This means that you can use the Hub version of ‘liveslak/etherpad’. But you can just as well use your own locally generated etherpad image in the ‘docker run‘ commands that launch your Etherpad container.
When you have a local image called “liveslak/etherpad”, then Docker will not check for an online image called “liveslak/etherpad”. If you did not generate your own image, Docker will look for (and find) my image at the Hub (or at the private Registry you may have configured), so it will download and use that.

Setting up WBO Whiteboard

Etherpad will be even more attractive if it offers users a collaborative Whiteboard and not just a collaborative text editor.
Enter WBO, which is an actual drawing board with infinite canvas and real-time refresh for all users.
Its boards are persistent; if you re-visit a board later on, all your content will still be there. Look at the WBO demo site… amazing.

We will run WBO in its own Docker container and re-configure our Etherpad webserver with a reverse proxy so that WBO can be integrated into Etherpad through the ‘ep_whiteboard’ connector.
It’s not so complex actually.

Docker container

First, launch a Docker container running WBO. We ensure that the data of the whiteboards you will be creating are going to be stored persistently outside of the container, so let’s create that data directory first and ensure that the internal WBO user is able to write there (you may have a different preference for directory location):

# mkdir -p /opt/dockerfiles/wbo-boards
# chown 1000:10 /opt/dockerfiles/wbo-boards

Then launch the container as a background process, and make it listen at port “5001” of your host’s loopback address:

$  docker run -d -p localhost:5001:80 -v "/opt/dockerfiles/wbo-boards:/opt/app/server-data" --restart unless-stopped --name whiteboard lovasoa/wbo:latest

Reverse proxy

We make WBO available behind an apache httpd reverse proxy which takes care of the encryption (https) using a Let’s Encrypt certificate.

Add the following block to your <VirtualHost></VirtualHost> definition of the server which also defines the reverse proxy for your Etherpad (which is https://meet.darkstar.lan/pad/ remember?):

# Reverse proxy for the WBO whiteboard Docker container:
<Location /whitepad/>
    ProxyPass http://127.0.0.1:5001/
    ProxyPassReverse http://127.0.0.1:5001/
</Location>

After restarting Apache httpd, your WBO whiteboard will be accessible via https://meet.darkstar.lan/whitepad/ . We will use that green highlighted text down below as the value for the ETHERPAD_WBO_HOST variable. Etherpad will prefix that text with “https://” and that prefix cannot be changed… hence the requirement for a reverse proxy that can handle the data encryption.

One caveat when you do this on your real-life internet-facing cloud server…
The Whiteboard server is accessible without authentication. It may be advisable to just come up with a different path component than “/whitepad/“, you can think of something like a UUID-like string: “/8cd77cbe-a694-4390-800a-638c7cc05f49/” as long as you use the same string in both places (reverse proxy and ETHERPAD_WBO_HOST definitions). Also, your board names are not visible anywhere unless you share their URLS with other people. So, a relatively safe environment.

Using the custom Etherpad with Jitsi Meet

If you followed Episode 3, you will have a directory “/usr/local/docker-jitsi-meet-stable-6826“, your version number may differ from my “6826“. Inside you will have your modified ‘docker-compose.yml‘ file.

We are going to edit two files: ‘.env‘ and ‘docker-compose.yml‘.

  • Relevant ‘.env’ additions:
    In the ‘.env‘ file we define correct values for the variables we introduced earlier. You can add the following lines basically anywhere, but it is of course most readable if you copy them immediately after the other ETHERPAD_* variables you added earlier on for the MySQL database backend:
    ETHERPAD_OIDC_ISSUER="https://sso.darkstar.lan/auth/realms/foundation"
    ETHERPAD_OIDC_BASE_URL="https://meet.darkstar.lan/pad/"
    ETHERPAD_OIDC_CLIENT_ID="etherpad"
    ETHERPAD_OIDC_CLIENT_SECRET="2jnc8H6RH9jIYMXExUHA7XF7uD8YKIRs"
    ETHERPAD_REQUIRE_AUTHENTICATION="true"
    ETHERPAD_WBO_HOST="meet.darkstar.lan/whitepad"
  • Relevant ‘docker-compose.yml’ additions:
    Add the following lines to the “etherpad:” section immediately below the MySQL database variable definitions you added earlier on in this Episode. You notice the variable names we defined in the previous section when dealing with ‘ep_openid_connect‘:
    - OIDC_ISSUER=${ETHERPAD_OIDC_ISSUER}
    - OIDC_CLIENT_ID=${ETHERPAD_OIDC_CLIENT_ID}
    - OIDC_CLIENT_SECRET=${ETHERPAD_OIDC_CLIENT_SECRET}
    - OIDC_BASE_URL=${ETHERPAD_OIDC_BASE_URL}
    - REQUIRE_AUTHENTICATION=${ETHERPAD_REQUIRE_AUTHENTICATION}
    - WBO_HOST=${ETHERPAD_WBO_HOST}
  • More ‘docker-compose.yml’ updates:
    The “etherpad:” service definition in that YAML file contains the following reference to the Etherpad Docker image:

image: etherpad/etherpad:1.8.16

You need to change that line to:

image: liveslak/etherpad:1.8.16

…in order to use our custom Etherpad image instead of the default one.

The re-configuration is complete and since we modified the ‘.env‘ file again, we  need to refresh and restart our Docker Jitsi Meet container stack again.
Note however, that at this point we have to perform this restart differently than mentioned earlier in this article. Since we are switching to a new Etherpad image, the container based on the old image needs to be removed also. For this scenario, please consult the detailed instructions in both sections “Considerations about the “.env” file” and “Upgrading Docker-Jitsi-Meet” in Episode 3 of this article series.
The complete set of steps to follow is a mix of both sections, and I share it with you for completeness’ sake:

# cd /usr/local/docker-jitsi-meet-stable-*
# docker-compose down
# rm -rf /usr/share/docker/data/jitsi-meet-cfg/
# mkdir -p /usr/share/docker/data/jitsi-meet-cfg/{web/letsencrypt,transcripts,prosody/config,prosody/prosody-plugins-custom,jicofo,jvb,jigasi,jibri}
# docker-compose pull
# docker-compose up -d

Don’t forget to remove the old, unused, Etherpad image because it is now wasting 474 MB uncompressed disk space.

Summarizing

Even though we set it up as part of the Jitsi stack, we now have a standalone Etherpad running which requires you to login when you visit “https://meet.darkstar.lan/pad/”.
On the other hand, you can also access Etherpad via Jitsi Meet. What’s different?
When you start a Jitsi meeting, via “https://meet.darkstar.lan/” and then click on “Open shared document“, you are already authenticated against Keycloak and the Etherpad document will open for you right away, no second login required.

After login, you will be met with a much more powerful editor than the basic one that comes with Docker Jitsi Meet. You’ll notice the extended document export capability thanks to Abiword and the small video widget at the top for face-to-face communication thanks to the WebRTC plugin.

Happy collaborating!

Running the custom Etherpad standalone

If you are not interested in Jitsi Meet, this is the command to start the customized Etherpad container and make it listen at port 9001 of the loopback address:

# docker run -d -p 127.0.0.1:9001:9001 liveslak/etherpad

The Etherpad container is now accessible only on your computer by pointing your browser at http://localhost:9001/ . You still need to add an Apache reverse proxy definition to the VirtualHost site definition to make your Etherpad available for other users at https://meet.darkstar.lan/pad/ .
If you want to change the container’s behavior using the available variables as documented before, you can pass these to the ‘docker run‘ command using one or more “-e” parameters, like so (this example just enables the admin console):

# docker run -d -p 127.0.0.1:9001:9001 \
  -e ADMIN_PASSWORD="my_secret_admin_pass" \
  --name etherpad \
  liveslak/etherpad

With additional environment variables you can enable more of the latent functionality. See the earlier sections of this article for all the relevant variables: those that enable the MySQL database backend; the one that enables the Whiteboard; those that enable the Keycloak authentication, etc.

Thanks

Etherpad with integrated Whiteboard can be a compelling solution for some user groups. Even without Jitsi Meet, you can jointly write and draw, save your work to your local harddrive and you have voice & video in a small overlay if you need to discuss the proceedings.
I encourage you to try it out. With or without integration into Jitsi Meet or even without Keycloak authentication if you want to create this as a completely free and low-treshold service to your local community.

Let me know what you think of this Episode in the comments section below. The final Episode, how to setup your own private Docker image repository, will take some time to write… I have not yet started doing in-depth research on that topic. But the six available Episodes will hopefully keep you occupied for a while 🙂
Thanks for reading until the end.

Eric

Slackware Cloud Server Series, Episode 4: Productivity Platform

Hi all!
Welcome to the fourth episode in a series of articles that I am writing about using Slackware as your private/personal ‘cloud server’.
Check out the list below which shows past, present and future episodes in the series, if the article has already been written you’ll be able to click on the subject.
The first episode also contains an introduction with some more detail about what you can expect.
These articles are living documents, i.e. based on readers’ feedback I may add, update or modify their content.

Nextcloud Productivity Platform

So far, I have explored Docker container infrastructure management, shown you how to build an Identity and Access Management service in Docker and used that to make a Video Conferencing platform available to the users of your Slackware Cloud server. Today we are going to explore other means of collaboration and personal productivity.
Some of the topics in the current Episode will cover cloud storage: how do you share your data (documents, music, photos, videos) across your computers with a safe backup copy on your Cloud server? How do you keep track of your activities, read your emails, plan your work, chat with your friends and setup video calls with them? Moreover, how can I achieve all of this without having to depend on the Cloud plaforms that Google, Dropbox, Amazon, Microsoft offer and which seem free but are most certainly not?

The Nextcloud platform which is the main topic of this article, offers a solution to this dilemma. Nextcloud is a spin-off from ownCloud. It was announced in 2016 by the original founder of ownCloud after he left the company because he did not agree to the course ownCloud was going to take. OwnCloud now offers part of its features exclusively as a paid-for subscription model, whereas Nextcloud is fully open source and all features are available to community users as well as paying customers.

The core of Nextcloud is called the Nextcloud Hub, the server product which integrates the three main components Files, Talk and Groupware and which you can host and manage on a server which you own and control. Client programs are available for Linux, Windows and Android platforms that allow you to sync your local files to the server, participate in videoconferences directly from your phone etc.

The goal of setting up Nextcloud on our Slackware Cloud Server is to achieve total control over your data, share these data securely with other people and eliminate dependencies on similar commercial offerings while enjoying the same functionality.

But first a slight detour before we dive into the setup of our Collaboration Platform.
Nextcloud allows for document collaboration in similar fashion to Microsoft Office 365. This is a topic which has its own Episode in the article series: the next one called “Collaborative Document Editing” will center around the integration of Collabora Online Development Edition (CODE) into our Nextcloud Hub.
Collabora is a major contributor to the LibreOffice suite. An online web-based front end to LibreOffice programs was initiated and is being developed by Collabora since 2014, and in essence, Collabora Online is the continued development of the original ‘LibreOffice Online’ code. Collabora moved the repository to github and it is the source used to build CODE, but is also used to create an “enterprise” version which has a paid subscription model. The pre-compiled CODE binaries have a limitation of 10 simultaneously edited documents, but compiling the source yourself will circumvent this artificial limitation.

Preamble

For the scope of this article I will assume that you have a Slackware 15.0 server with IP address “10.10.10.10“. Your own IP address is of course different but your (web)server needs to be accessible online.
Our example server (you need to apply real-life values here of course) has the following configuration:

  • a working Docker environment;
  • Apache httpd configured and running and reachable as “https://darkstar.lan/” with the DocumentRoot directory “/var/www/htdocs/” which is the default location for Apache httpd’s content in Slackware.
  • Keycloak configured and running at “https://sso.darkstar.lan/auth/”;
  • Jitsi Meet configured to use Keycloak for authentication and running at “https://meet.darkstar.lan/“; with the keycloak-jitsi bridge at “https://sso.meet.darkstar.lan/“.
  • PHP version 7.3, 7.4 or 8.0, which is the reason for the Slackware 15.0 requirement – the PHP of Slackware 14.2 is just too old.

Nextcloud Hub II

The Nextcloud Hub II is the new name of the Nextcloud server product which you can download as a single ZIP file and install into your webserver. Nextcloud Hub can be extended with plugins and for that, there is an active App Store. The Nextcloud administrator can search for, install, enable, disable, update, configure and remove these applications from the web-based management console or even from the command-line. This article will discuss some of these applications, in particular the ‘featured apps‘ which are the ones that are supported or developed by Nextcloud GmbH directly.
The core of Nextcloud Hub II consists of the following components:

Files:

Nextcloud Files allows easy access to your files, photos and documents. You can share files with other Nextcloud users, even with remote Nextcloud server instances. When CODE is installed (see the next Episode in this Series) you can even collaborate on documents in real-time with friends, family, team members, customers etc. We have inherent security through data encryption (in transit and on the local file storage) and access control mechanisms, so that only those people with whom you explicitly share your file(s) will be able to access them.

Users store files in their account and this translates to a per-user directory in the host’s filesystem. By default this storage area is not encrypted. If you don’t feel at ease with the idea that unauthorized access to the server potentially exposes userdata to the intruder, the Nextcloud administrator can enable on-disk encryption of all the users’ files. Once encrypted, the userdata can not be decrypted by anyone else than the owner of the files (aka the user) and those with whom files are shared. An encryption key is created per user and the user’s password unlocks that key which is always and only stored on the server. If the user loses their password, they will permanently lose access to their data!
The administrator can configure a ‘recovery key‘ however. You may pose that this allows that administrator to gain access to the userdata, but that’s the other side of the coin of being able to decrypt userdata in the event of loss of the password…

Talk:

Nextcloud Talk offers screensharing, online meetings and web conferencing. The video conferencing capabilities are not yet as strong as Jitsi Meet (in particular it does not scale well to a lot of simultaneous users) but the Talk component is being improved continuously.

Groupware:

Nextcloud Groupware consists of easy to use applications for Webmail, Calendaring and Contacts and is integrated with Nextcloud Files. In webmail you can configure multiple IMAP server accounts, PGP encryption is supported. Calendars can be shared and have Nextcloud Talk integration for video conferences. You can create appointments and define and book resources. Contacts can be grouped, shared with others, and synchronized to and from your phone.
In addition, you can install the featured app ‘Deck‘ which enables you to manage your work with others through private or shared Kanban-style task boards.

Setting up MariaDB SQL server

Nextcloud Hub uses a MariaDB server as its configuration back-end. The users’ files are stored in your host file-system.
You could use SQLite instead… but that is really going to affect the performance of your server negatively if the load increases. In case you have never setup a MariaDB SQL server, I will give you a brief rundown.

MariaDB is part of a full installation of Slackware, and since ‘full install‘ is the recommended install, I will assume you have a mariadb package installed. If not, because you decided to do only a partial installation on your server, be sure to start with an installation of mariadb and its dependencies lz4 and liburing.

  • Create the system tables for the SQL server:
    # mysql_install_db --user=mysql
    Installing MariaDB/MySQL system tables in '/var/lib/mysql' ...
    OK
  • Make the MariaDB boot script executable:
    # chmod +x /etc/rc.d/rc.mysqld
  • Comment the ‘ SKIP="--skip-networking" ‘ line in ‘rc.mysqld‘ because we will use client connections from outside.
  • Execute the boot script (or reboot the machine):
    # /etc/rc.d/rc.mysqld start
  • Perform initial configuration of the database server:
    # /usr/bin/mysql_secure_installation
    Switch to unix_socket authentication [Y/n] n
    Set root password? [Y/n] y
    Remove anonymous users? [Y/n] y
    Disallow root login remotely? [Y/n] y
    Remove test database and access to it? [Y/n] y
    Reload privilege tables now? [Y/n] y

    This script locks down your SQL server properly. It will prompt you to setup the admin (root) password and gives you the option of removing the test databases and anonymous user which are created by default. This is strongly recommended for production servers. I answered “NO” to “Switch to unix_socket authentication” because the Docker containers on our Slackware Cloud server will need to access this MariaDB SQL server over the local network.

You now have MariaDB SQL server up and running, the root (admin) password is set and the root user can only connect to the database from the localhost. It’s ready to setup its first database.
We will manually create the required SQL database  for NextCloud (you might want to insert your own account/password combo instead of my example “nextcloud/YourSecretPassword“). Since we are installing NextCloud on the ‘bare metal‘, NextCloud can communicate with the SQL server via the ‘localhost‘ network address:

# mysql -uroot -p
> CREATE USER 'nextcloud'@'localhost' IDENTIFIED BY 'YourSecretPassword';
> CREATE DATABASE IF NOT EXISTS ncslack CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
> GRANT ALL PRIVILEGES ON ncslack.* TO 'nextcloud'@'localhost';
> FLUSH PRIVILEGES;
> quit;

Configuring Apache and PHP

First of all, we are going to increase the PHP memory limit from the default 128 MB to 512 MB (recommended value) in the file ‘/etc/php.ini‘ or else the more complex operations will fail:

memory_limit = 512M

Don’t forget to enable PHP support in “/etc/httpd/httpd.conf” by un-commenting the line:
Include /etc/httpd/mod_php.conf‘.

Your Apache server needs some directives inside the ‘<VirtualHost></VirtualHost>‘ block that are not enabled by default.
This section needs to be added:

  <Directory /var/www/htdocs/nextcloud/>
    Require all granted
    Satisfy Any
    AllowOverride All
    Options FollowSymLinks MultiViews

    <IfModule mod_dav.c>
      Dav off
    </IfModule>
  </Directory>
  Alias "/localapps" "/opt/netxtcloud/localapps"

Also, the Apache httpd modules mod_rewrite, mod_headers, mod_env, mod_dir and mod_mime need to be enabled. This is usually done in ‘/etc/http/httpd.conf‘.

Configuring local storage

We are going to maintain all of the Nextcloud users’ personal data separately from the server code and even outside of the webserver document root. Nobody should be able to access your files through a possible hole in your webserver.

Also, when the Nextcloud administrator (you) is going to extend the server’s functionality by installing additional apps, you will want these apps to be installed outside of the Nextcloud application directory. This allows for a clean upgrade process if we want to install a newer version in the future. The “/opt/nextcloud/” will be the root directory for all of this. The httpd user account “apache” needs to own these files.

Let’s start with creating the userdata directory:

#  mkdir -p /opt/nextcloud/data
#  chown apache:wheel /opt/nextcloud/data

And the directory where the add-ons will be installed:

# mkdir -p /opt/nextcloud/localapps
# chown apache:wheel /opt/nextcloud/localapps

To the Apache httpd VirtualHost configuration you should add an  ‘Alias‘ statement which points the webserver “/localapps” path component to the filesystem location “/opt/nextcloud/localapps“. This line was  already mentioned in the Apache configuration snippet further up:

Alias "/localapps" "/opt/netxtcloud/localapps"

Installing the Nextcloud Hub

We will download the latest stable release from their server installation page.
At the time of writing, 23.0.0 is the latest available so:

# cd ~
# wget https://download.nextcloud.com/server/releases/nextcloud-23.0.0.zip
# unzip -n nextcloud-23.0.0.zip -d /var/www/htdocs/
# chown -R apache:wheel /var/www/htdocs/nextcloud/

We have already created the SQL database, so we can immediately continue with the Web setup. Point your browser at http://darkstar.lan/nextcloud/ :

This is where you will define the account and password for the Nextcloud administrator. It will be the only account with local credentials (stored in the SQL backend). All  future Nextcloud user authentication & authorization will be configured and managed via the Keycloak IAM program. The green text snippets match with configuration settings documented higher up in this article.

  • I will use the name “admin” for the account and give it a secure password.
  • Storage & Database:
    • Data Folder: “/opt/nextcloud/data
    • Database: MySQL/MariaDB
    • Database user: “nextcloud
    • Database password: “YourSecretPassword
    • Database name: “ncslack
    • Localhost: add the MariaDB port number so that it reads “localhost:3306”
  • Install recommended apps: Calendar, Contacts, Talk, Mail & Collaborative editing: ensure that this box is checked (note that the Collaborative Editing program – CODE – download will probably fail but we will install it as an external application later):

TIP: I experienced that when the password fields are set to “show password”, the installation will refuse to continue.

Click “Finish” to kick off the Nextcloud setup.

This configuration process ends with some screens that will show off the capabilities of the programs, you can either close these immediately (the “X” top right) or click through them until the end. You will end up in your Nextcloud dashboard:

Post-configuration

After installation and initial configuration we are now going to tune the server.

Pretty URLs

A Nextcloud URL looks like “http://localhost/nextcloud/index.php/apps/dashboard/” and I want to remove the “index.php” string which makes the URLs look nicer. Check the official documentation.

  • Edit “./config/config.php” file in the nextcloud directory. Add the following two variables right before the last line which goes like “);“:
    ‘overwrite.cli.url’ => ‘https://darkstar.lan/nextcloud’,
    ‘htaccess.RewriteBase’ => ‘/nextcloud‘,
  • We are going to use the NextCloud command-line interface ‘occ‘ (here it shows its heritage; ‘occ‘ stands for ‘Own Cloud Console‘) in order to apply the changes in ‘config.php‘ to the ‘.htaccess‘ file in the NextCloud installation directory.
    The ‘occ’ command needs to be run as the httpd user (‘apache‘ in the case of Slackware):
    # sudo -u apache php -d memory_limit=512M /var/www/htdocs/nextcloud/occ \
    maintenance:update:htaccess
    .htaccess has been updated
    Note that I have not been able to get this to actually work! Pointers with a fix for pretty URLs are welcome!

Server scan

You can run a “server scan” as admin user via “Settings > Administration > Security & Setup Warnings” to see if Nextcloud has suggestions on improving performance and functionality of the server.
In my case, it reports that a number of database indexes are  missing, and that the following commando will fix this:

# sudo -u apache php /var/www/htdocs/nextcloud/occ db:add-missing-indices

This scan will also point out performance issues about which I will share more details, below in “Tuning server performance” section.

Cron maintenance

It is advised to install a cron job which runs as the “apache” user, and performs maintenance every 5 minutes. In Nextcloud as the admin user, you configure the “Cron” maintenance type via “Settings > Administration > Basic Settings > Background Jobs“.
This needs to be accompanied by an actual cron job definition on the Slackware server; so this is the command that I added to the crontab for “apache“:

# crontab -u apache -l
# Run maintenance for NextCloud every 5 minutes:
*/5 * * * * /usr/bin/php -f /var/www/htdocs/nextcloud/cron.php

Install add-on applications outside of Nextcloud directory

We have already created a ‘/opt/nextcloud/localapps‘ directory with the intention of using that location for the installation of all the apps that we want to add to our Hub.
In order to enable a separate application installation path, you need to add the following lines to ‘./config/config.php‘:

'appstoreenabled' => true,
'apps_paths' => array(
    array(
    'path'=> OC::$SERVERROOT . '/apps',
    'url' => '/apps',
    'writable' => false,
  ),
  array(
    'path'=> '/opt/nextcloud/localapps',
    'url' => '/localapps',
    'writable' => true,
  ),
),

Tuning server performance

Memory cache

The main optimizations which will boost your server’s performance and responsiveness are related to memory caching and transactional file locking.

You can address these both by installing redis (an in-memory key-value memory store) and php-redis (PHP extension for redis) as Slackware packages from my repository:

After installing redis, configure it correctly for Nextcloud.

  • Enable the php redis extension by editing ‘/etc/php.d/redis.ini‘ and removing the semicolon so that you get the following line:
    ; Enable redis extension module
    extension=redis.so

    Optionally add these lines to tune its performance:
    ; Should the locking be enabled? Defaults to: 0.
    redis.session.locking_enabled = 1
    ; How long should the lock live (in seconds)?
    ; Defaults to: value of max_execution_time.
    redis.session.lock_expire = 60
    ; How long to wait between attempts to acquire lock, in microseconds?.
    ; Defaults to: 2000
    redis.session.lock_wait_time = 10000
    ; Maximum number of times to retry (-1 means infinite).
    ; Defaults to: 10
    redis.session.lock_retries = -1
  • Make the redis daemon listen at a UNIX socket as well as a TCP port, uncomment these lines in the “/etc/redis/redis.conf” configuration file:
    unixsocket /var/run/redis/redis.sock
    unixsocketperm 660
  • Make the init script executable and start it manually (it gets added to rc.local so it will start on every boot from now on):
    # chmod +x /etc/rc.d/rc.redis
    # /etc/rc.d/rc.redis start

Enable the use of redis in Nextcloud’s main configuration file “/var/www/htdocs/nextcloud/config/config.php” by adding the following block:

'memcache.locking' => '\\OC\\Memcache\\Redis',
'memcache.local' => '\\OC\\Memcache\\Redis',
'memcache.distributed' => '\\OC\\Memcache\\Redis',
'redis' => 
array (
    'host' => '/var/run/redis/redis.sock',
    'port' => 0,
),

Finally, allow the “apache” user to be able to read the Redis socket by adding the user to the “redis” group:

# gpasswd -a apache redis

… and restart the Apache httpd service.

PHP OPcache

OPcache is a PHP extension for Apache httpd which stores (pre-)compiled PHP script bytecode in memory. This speeds up the performance of PHP-based applications like Nextcloud, because it reduces or eliminates the need to load PHP scripts from disk and parse them every time they are called.
Slackware’s PHP has opcache enabled in “/etc/php.ini“. Relevant settings are:

opcache.enable=1
opcache.memory_consumption=128
opcache.max_accelerated_files=10000
opcache.revalidate_freq=200

These are default values for Slackware and therefore they are commented-out in the “php.ini” file but you can validate that they are enabled via the command:

# php -i |grep opcache

If you find that OPcache is disabled, you should fix that ASAP.

Check your security

Nextcloud offers a neat online tool that connects to your Nextcloud instance and performs a series of security checks, and will advise you in case you need to improve on your server security: https://scan.nextcloud.com/ .

Commandline management

The web-based management interface for the “admin” user is ‘https://darkstar.lan/nextcloud/index.php/settings/user‘ but a lot of administration tasks can also be done directly from the Slackware server’s command prompt (as root) using the “Own Cloud Console” program ‘occ‘ I already briefly touched on earlier in this article.

Some examples:

Install an application, say “richdocumentscode” which is a built-in low-performance version of CODE (Collabora Online Development Edition):

# sudo -u apache php -d memory_limit=512M /var/www/htdocs/nextcloud/occ app:install richdocumentscode

App updates can be done on commandline too:

# sudo -u apache php -d memory_limit=512M /var/www/htdocs/nextcloud/occ app:update --all

Remove an app:

# sudo -u apache php -d memory_limit=512M /var/www/htdocs/nextcloud/occ app:remove richdocumentscode

Get the list of possible occ command options:

# sudo -u apache php -d memory_limit=512M /var/www/htdocs/nextcloud/occ app:list

Integration with Keycloak IAM

The server is running nicely, and it is time to invite users. All our users will be managed by our Keycloak IAM program.
To integrate Keycloak into Nextcloud we first create a “Client ID” for Nextcloud in our Keykloak admin console and copy the relevant secret bits from that ID into Nextcloud. Nextcloud will contact Keycloak with a URL that will ebable login for the users in our “foundation” realm, i.e. the realm which we created when we setup Keycloak.

Keyloak:

  • Login to the Keycloak Admin Console, https://sso.darkstar.lan/auth/admin/ .
    • Under ‘Clients‘, create a new client:
      Client ID‘ = “nextcloud
      Root URL‘ = “darkstar.lan/nextcloud
    • Save.
    • In the ‘Settings‘ tab, change:
      Access Type‘ = “confidential” (default is “public”)
    • Save.
    • In the ‘Credentials‘ tab, copy the value of the ‘Secret‘, which we will use later in Nextcloud: “61a3dfa1-656d-45eb-943a-f3579a062ccb” (your own value will of course be different).

Nextcloud:

  • Login to Nextcloud as the admin user: https://darkstar.lan/nextcloud
    • In ‘Profile > Apps‘, install the ‘Social Login‘ app.
  • Configure Nextcloud’s social login to use Keycloak as the identity provider
    • Make sure ‘Disable auto create new users‘ is UN-checked because every user logging in via Keycloak for the first time must automatically be created as a new user.
    • DO check ‘Prevent creating an account if the email address exists in another account‘ to ensure that everyone will have only one account
    • Add a new ‘Custom OpenID Connect‘ by clicking on the ‘+‘:
      ‘Title’ = “keycloak”
      ‘Authorize url’ = “https://sso.darkstar.lan/auth/realms/foundation/protocol/openid-connect/auth”
      ‘Token url’ = “https://sso.darkstar.lan/auth/realms/foundation/protocol/openid-connect/token”
      ‘Client id’ = “nextcloud
      ‘Client Secret’ = “61a3dfa1-656d-45eb-943a-f3579a062ccb” (this is the value you obtained earlier in the Keycloak Client definition)
      ‘Scope’ = “openid”
      ‘Default group’ = “none”
    • ‘Groups claim’ = “nextcloud” (optional setting, make sure you create a group “nextcloud” in Keycloak’s “foundation” realm and have added your future Nextcloud users as members of this group).
    • Also optional, ‘Add group mapping‘ can automatically arrange that the Keycloak group “nextcloud” is mapped to Nextcloud group, e.g. called “My social circle” with which you can limit access in Nextcloud to only those apps that you approved.
    • Save.

Your Nextcloud login page will now additionally offer a ‘Login with Keycloak Authentication” option. Clicking this will redirect you to a Keycloak authentication page. Screenshots of how this looks exactly can be found further down in the section “Nextcloud clients“.
After successful login and authorization with an existing Keycloak user account you will be returned to Nextcloud and logged in. If this was the user’s first login to Nextcloud using Keycloak credentials, Nextcloud will create an internal user account which is linked to the Keycloak credentials.

Integrating Jitsi Meet

The integration of an embedded Jitsi Meet window into Nextcloud requires some configuration in Apache httpd (for the embedding) and in Nextcloud itself (securing the intercommunication via a JSON Web Token).

Apache:

Add the following to your httpd global configuration, for instance to ‘/etc/httpd/httpd.conf‘ (the parts in green need to be replaced by your real-life hostnames of course):

# Mitigate the risk of "click-jacking" (other sites embedding your pages
# and adding other, possibly malicious, content)
# See https://developer.mozilla.org/en-US/docs/HTTP/X-Frame-Options
Header always append X-Frame-Options SAMEORIGIN
# And allow embedding of Jitsi Meet in Nextcloud (add any URL for which you would also allow embedding):
RequestHeader set X-HTTPS 1
Header always set Content-Security-Policy "frame-ancestors 'self';"
Header always set Content-Security-Policy "frame-ancestors https://darkstar.lan https://sso.darkstar.lan https://meet.darkstar.lan ;"

Followed by a restart of Apache httpd.

Nextcloud:

  • Install the “Jitsi Integration” app as the admin user via ‘Profile > Apps
  • Go to  ‘Profile >Settings > Jitsi’
    • ‘Server URL’: enter “https://meet.darkstar.lan
    • ‘JWT Secret’: enter “NmjPGpn+NjTe7oQUV9YqfaXiBULcsxYj” (this is the JWT token which you entered as the value for ‘JWT_APP_SECRET‘ variable in the ‘.env‘ file of  our ‘docker-jitsi-meet‘  instance, see the previous Episode on Video Conferencing for details)
    • JWT App Id‘: enter “jitsi” (which is the value of the  ‘JWT_APP_ID‘ variable in that same ‘.env‘ file)
    • Save

That’s it!
Users will now find a ‘Conferences‘ icon in the application bar at the top of their Nextcloud home. Clicking it will open an embedded Jitsi video conferencing window.

One obvious advantage to using this NextCloud embedded Jitsi Meet is that the resulting Meet URLs are impossible to guess, they will not mention the meeting room name but instead use a UUID like ‘https://meet.darkstar.lan/06135d8e-5365-4a6f-b31b-a654d92e8985‘.
If you compare this to the regular Jitsi Meet at ‘https://meet.darkstar.lan/‘, your meeting room with the name ‘roomname‘ would be accessible as ‘https://meet.darkstar.lan/roomname‘ which is a lot easier to guess… un-desirable if you want to hold private meetings.

Adding new apps

I encourage you to check out the Nextcloud App Store. There’s a wealth of cool, useful and fun applications to be found there.

Some of the apps that I installed are: GPXPod (visualize your GPX routes); Carnet (powerful note-taking app); Music (listen to your local collection or tune into online streams); PhoneTrack (keep track of your phone, needs the Android app to be installed too); Element (messenger app with bridges to Matrix and Libera.Chat for instance); Cookbook (maintain your own recipe database and easily import new recipes from external websites using only the URL to the recipe), etc…

Adding client push daemon

Extremely useful is Push update support for the desktop app. The 21 release of Nextcloud introduced a ‘high performance backend‘ for filetransfers. This increases the speed of filetransfers substantially, but it needs a couple of things to make it work.

  • The worker app is called “Client Push” and needs to be installed on your server from the Nextcloud App Store. Once the app is installed, the push binary still needs to be setup. The README on github contains detailed setup instructions. For the Slackware way, read on.
  • You need to have the redis package installed and the redis server needs to be started and running. We already took care of this in the section ‘Performance tuning‘.
  • Slackware, Apache and Nextcloud need to be configured for redis and the notify_push daemon.

Slackware:

The notify_push binary gets installed as ‘/opt/nextcloud/localapps/notify_push/bin/x86_64/notify_push‘. Hint: this is 64-bit only. This binary needs to run always, so the easiest way is to add a Slackware init script and call that in ‘/etc/rc.d/rc.local‘.

The init script can be downloaded from my repository, but it’s listed here in full as well. I highlighted the texts in green that are relevant. Save it as ‘/etc/rc.d/rc.ncpush‘ and make it executable:

#!/bin/bash
# Description: Push daemon for NextCloud clients.
# Needs: redis php-fpm mariadb
# Written by: Eric Hameleers <alien@slackware.com> 2021
daemon=/usr/bin/daemon
description="Push daemon for Nextcloud clients"
pidfile=${pidfile:-/var/run/nextcloud/notify_push.pid}
ncconfig=${ncconfig:-/var/www/htdocs/nextcloud/config/config.php}
command=${command:-/opt/nextcloud/localapps/notify_push/bin/x86_64/notify_push}
command_user=${command_user:-apache}
command_args="--bind 127.0.0.1 --port 7867 $ncconfig"

[ ! -x $command ] && exit 99
[ ! -f $ncconfig ] && exit 99

RETVAL=0

start() {
  if [ -e "$pidfile" ]; then
    echo "$description already started!"
  else
    echo -n "Starting $description: "
    mkdir -p $(dirname $pidfile)
    chown $command_user $(dirname $pidfile)
    chmod 0770 $(dirname $pidfile)
    $daemon -S -u $command_user -F $pidfile -- $command $command_args
    RETVAL=$?
    [ $RETVAL -eq 0 ] && touch /var/lock/subsys/$(basename $command)
    echo "- done."
  fi
}
stop(){
  echo -n "Stopping $description: "
  kill -TERM $(cat $pidfile)
  RETVAL=$?
  [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/$(basename $command)
  echo "- done."
}
restart(){
  stop
  start
}
condrestart(){
  [ -e /var/lock/subsys/$(basename $command) ] && restart
}
status() {
  pids=$(cat $pidfile 2>/dev/null)
  if test "$pids" ; then
    echo "$description is running."
    ps up $pids
  else
    echo "$description is stopped."
  fi
}
# See how we were called.
case "$1" in
start)
  start
  ;;
stop)
  stop
  ;;
status)
  status
  ;;
restart)
  restart
  ;;
condrestart)
  condrestart
  ;;
*)
  echo "Usage: $0 {start|stop|status|restart|condrestart}"
  RETVAL=1
esac
exit $RETVAL
# ---

The script is then called on boot by adding the following to your server’s ‘/etc/rc.d/rc.local‘:

if [ -x /etc/rc.d/rc.ncpush ]; then
  # Start Nextcloud Client Push Daemon
  echo "Starting Nextcloud Client Push Daemon: /etc/rc.d/rc.ncpush start"
  /etc/rc.d/rc.ncpush start
fi

Apache:

Setup the bits of Apache configuration for your webserver. The following block of configuration needs to be added:

ProxyPreserveHost On
ProxyTimeout 900
SSLProxyEngine on
RequestHeader set X-Forwarded-Proto "https"

# Setup a reverse proxy for the client push server
# https://github.com/nextcloud/notify_push/blob/main/README.md :
<Location /push/ws>
    ProxyPass ws://127.0.0.1:7867/ws
</Location>
<Location /push/>
    ProxyPass http://127.0.0.1:7867/
    ProxyPassReverse http://127.0.0.1:7867/
</Location>

# Do not forget WebSocket proxy:
RewriteEngine on
RewriteCond %{HTTP:Connection} Upgrade [NC]
RewriteCond %{HTTP:Upgrade} websocket [NC]

You know where to add the above.

Nextcloud:

Add these lines to “./config/config.php“. Relevant pieces are highlighted in green, you should of course change that into your own Internet IP address:

‘trusted_proxies’ => [‘10.10.10.10‘],
‘forwarded_for_headers’ => [‘HTTP_X_FORWARDED’, ‘HTTP_FORWARDED_FOR’, ‘HTTP_X_FORWARDED_FOR’],

Piecing it all together:

Preparations are complete after you have restarted Apache httpd and started the ‘rc.ncpush‘ script. Redis should also be running.
Now we can finally enable the app we installed in Nextcloud:

# sudo -u apache php -d memory_limit=512M /var/www/htdocs/nextcloud/occ app:enable notify_push

Setup the connection between the app and the daemon:

# sudo -u apache php -d memory_limit=512M /var/www/htdocs/nextcloud/occ notify_push:setup https://darkstar.lan/push/
> redis is configured
> push server is receiving redis messages
> push server can load mount info from database
> push server can connect to the Nextcloud server
> push server is a trusted proxy
> push server is running the same version as the app
configuration saved

Upgrading to newer release

Here is an example of the steps to take when you upgrade from one major release to the next (for instance you want to upgrade from 23 to 24).

First, download and extract the release tarball, and set the file ownership to the ‘apache‘ user:

# cd ~
# wget https://download.nextcloud.com/server/releases/nextcloud-24.0.0.tar.bz2
# tar xf nextcloud-24.0.0.tar.bz2
# chown -R apache:wheel /root/nextcloud

Disable the server-maintenance cronjob of the “apache” user before continuing, for instance by adding ‘#” in front of the cron commandline!

# crontab -u apache -e
# crontab -u apache -l
# Run maintenance for NextCloud every 5 minutes:
####*/5 * * * * /usr/bin/php -f /var/www/htdocs/nextcloud/cron.php

Stop the Apache httpd:

# /etc/rc.d/rc.httpd stop

Rename the old “/var/www/htdocs/nextcloud/” installation directory and move the new ‘nextcloud‘ directory in its place:

# mv -i /var/www/htdocs/nextcloud /root/nextcloud.orig
# mv -i /root/nextcloud /var/www/htdocs/nextcloud

Copy your Nextcloud configuration into the new directory (this should only overwrite the ‘config.sample.php‘ file):

# cp -ia /root/nextcloud.orig/config/* /var/www/htdocs/nextcloud/config/

Start the Apache httpd again:

# /etc/rc.d/rc.httpd start

Start the commandline updater. This MUST be done from within the Nextcloud installation directory:

# cd /var/www/htdocs/nextcloud/
# sudo -u apache php -d memory_limit=512M occ upgrade

Now, re-enable the cronjob for user “apache“.
Then re-create the now again missing database indices:

# sudo -u apache php /var/www/htdocs/nextcloud/occ db:add-missing-indices

Stable versus beta channels

Nextcloud offers “Stable” and “Beta” channels of its software. From time to time, new functionality becomes available in the Beta channel that you really want to use.
As the admin user, you can configure your preferred release channel in “Settings > Administration > Overview > Version > Update Channel” if you have installed Nextcloud from a release tarball like we did.

NOTE that you can only upgrade to a newer versionSkipping major versions when upgrading and downgrading to older versions is not supported by Nextcloud.
For instance, if you went via ‘Beta‘ to 24.0.0rc4 and ‘Stable‘ is still on 23.0.5, you have to wait with further upgrades until 24.0.0 or later becomes available in the ‘Stable‘ channel.

Nextcloud makes new versions incrementally available to user installations in the Stable channel which means it can take a while before your server alerts you that a new release is available.

Nextcloud clients

Nextcloud mobile and desktop clients are available from https://nextcloud.com/clients/ .

Client applications can keep your files synchronized between Nextcloud server and your desktop or phone; can manage your passwords that you store securely on the server; can manage your calendar, setup chat and video connections to other users of your server; and more.

The desktop file-sync client (nextcloud-client) is available in my own repository as a package and on SlackBuilds.org as a build script (but the SBo version is actually still the old OwnCloud mirall client). Its GUI is Qt5-based. This is a free alternative for Dropbox and the likes!

When you start the nextcloud-client desktop application for the first time, you will go through an initial setup. The client needs to get configured to connect to your Nextcloud server:

After clicking “Log in to your Nextcloud” you can enter the URL at which your server is available:

The client will open your default browser (in case of my KDE Plasma5, this was Konqueror) and ask you to switch to that browser window to logon to your Nextcloud:

You can click “Copy Link” so that you  can use another browser if you are not happy with the default browser choice. The next steps take place inside that browser window:

You need to login with your Nextcloud user account. In this dialog we click on “Log in with Keycloak Authentication” instead of entering an account/password into the top entry fields – remember, only the admin user has an account with local credentials:

You will probably recognize the familiar Keycloak Single Sign On dialogs from the previous Episode (Video Conferencing):

This account has 2-Factor Authentication (2FA) enabled:

After succesful login, the Nextcloud server wants to get your explicit permission to grant access for the desktop client to your server data:

After you click “Grant access” the client will finally be able to connect to your data and start syncing:

You get a choice of the local desktop directory to synchronize with your server data – by default this is ‘~/Nextcloud/‘:

The client docks into the system tray from where you can access your files and manage the sync. The desktop client integrates nicely with KDE’s Dolphin and Konqueror, and shows the sync status of the ‘~/Nextcloud/‘ directory (and the files in it) with a badge – similar to what Dropbox does.
All files inside ‘~/Nextcloud/‘ will have a right-mouseclick context menu offering some filesharing options and the possibility to open the serverlocation of the file in your webbrowser.

Fixes

To end this Episode, I will collect the fixes that were necessary to make Nextcloud do what I wanted it to do.

Enable geolocation in the NextCloud phonetrack app by applying this patch:

# diff -u lib/public/AppFramework/Http/FeaturePolicy.php{.orig,}
--- lib/public/AppFramework/Http/FeaturePolicy.php.orig 2021-10-02 12:57:56.1506
83402 +0200
+++ lib/public/AppFramework/Http/FeaturePolicy.php      2021-10-11 15:38:01.2146
23284 +0200
@@ -49,7 +49,9 @@
        ];

        /** @var string[] of allowed domains that can use the geolocation of the
 device */
-       protected $geolocationDomains = [];
+       protected $geolocationDomains = [
+               '\'self\'',
+       ];

        /** @var string[] of allowed domains that can use the microphone */
        protected $microphoneDomains = [];

Thanks

Thanks for reading all the way to the end of this Episode. I hope you learnt from it and are eager to try this Nextcloud platform on your own private Slackware Cloud Server!
As always, leave constructive and helpful feedback in the comments section below.

Cheers, Eric

Slackware 15.0 has been released on 2022-02-02

Pick your favorite quote for today!

What would you do if you were stuck in one place,
and every day was exactly the same and nothing that you did mattered?

… or rather:

Welcome to Adventure!!  Would you like instructions?

Wed Feb 2 22:22:22 UTC 2022
Slackware 15.0 x86_64 stable is released!

Another too-long development cycle is behind us after we bit off more than
we could chew and then had to shine it up to a high-gloss finish. Hopefully
we've managed to get the tricky parts out of the way so that we'll be able
to see a 15.1 incremental update after a far shorter development cycle.

I honestly kept my breath and have had some difficulty believing that this would eventually really happen, but yet here it is. Slackware 15.0 stable! Released yesterday and available on mirrors across the globe today.
I am fiercely proud of Patrick Volkerding who has again showed all of us that the oldest actively maintained Linux distribution is more than a rolling release for Linux enthusiasts.
The latest stable release 15.0 is packed with applications for both desktop and server usage. It is truly a modern Linux platform that power users can and should embrace as their daily driver.

I will share with you some of the highlights and particulars of the development process between the release of Slackware 14.2 (the previous stable) and the new 15.0.

  • With five years and seven months in the making, this has been the largest interval between stable releases.
  • We said ‘farewell’ to the Slackware Store and goodbye to printed DVD and CDROM sets.
    A major reason for this dragged-out development cycle is the realization that for a long time, Pat and family have been deprived of substantial amounts of revenue which the people of The Store reaped through the sales of Slackware installation media, clothing and other paraphernalia. The truth was shared by Pat in a post on LinuxQuestions and resulted in a long-running thread discussing how the community could help with the funding: https://www.linuxquestions.org/questions/slackware-14/donating-to-slackware-4175634729/ .
    Luckily a new means of securing an income for Pat was established through Patreon, which allows Pat to focus again on his Slackware distro. After all, Slackware is his only source of income; the rest of the coreteam have regular day jobs and we don’t get (nor want) any part of the revenue.
  • A ChangeLog.txt of roughly 40,000 lines makes the process of “slackpkg install-new” (churning through all that text) to find all those added packages a long and painful one.
  • The Slackware 15.0 core distribution consists of 1590 packages, a 19% increase from the 1332 packages in Slackware 14.2 core (i.e. everything not in ./extra or in ./testing).
    A lot of packages have been removed (276), many of those belong to KDE4 and 82 of them are the old language packs. You won’t miss a thing there because language support in Plasma5 is now included in all the application packages.
    If you look at the number of packages that have been added to the distro (i.e. did not exist in Slackware 14.2), that’s an impressive 534.
  • The compressed size of these packages went up from 2.4 GB to 3.0 GB. Uncompressed and installed onto your computer, the difference will be significantly larger. The recommended disk space needed for a full installation, increases from 9+ GB (Slackware 14.2) to 16+ GB.

So what about that exciting content you can find in the new 15.0 release?

  • Slackware 15.0 installs Linux kernel 5.15.19.
  • The kernels all have preemption enabled and you can configure your bootloader for full, dynamic or no preemption.
  • The installer properly supports installing to NVMe disks and has improved support for installing on UEFI platform.
  • Slackware still supports 32bit hardware platforms and is one of the few relevant Linux distros that do so today.
    Both the 64bit and the 32bit distro are built from a single source tree.
  • Collaboration with smart people on the LinuxQuestions.org forum resulted in the ability for Slackware to be reliably built from scratch these days. Thanks in particular to worsel and nobodino. The latter maintains his efforts in a git repository from which Slackware has borrowed several patches. Slackware itself has a ‘master build script’ called make_world.sh and Patrick keeps a log of everything that “fails to build from source” aka the FTBFSlog files which you can find in the source-directory root of every package series. Nothing in the distro fails to build from source right now, which is awesome and a definite change from previous stable  releases.
  • In ./source/installer you find the complete sources to easily rebuild the Slackware installer from scratch.
  • ConsoleKit2 was replaced with elogind as the session/seat manager. Elogind is a standalone version of the logind code in systemd and independently maintained at https://github.com/elogind/elogind .
  • FFmpeg and Lame have been added to the distro, yay! Also, it includes SDL2, speex, opus, fluidsynth.
  • A new command-line ISO creation tool to supplement cdrtools was added: xorriso.
  • Python3 was introduced but Python2 stuff remains available and is still the default python. Its most relevant modules have been bundled in a compatibility package.
  • Postfix has been added and Sendmail moved to ./extra where it still receives upgrades. Lucky me, I still prefer and use Sendmail on my boxes.
  • Dovecot is the powerful new IMAP and POP3 server, replacing the old imapd and ipop3d.
  • The Qt4 graphical toolkit has been removed and Qt5 came in its place.
  • KDE 4 (which was already abandoned at the release of Slackware 14.2) has been replaced with KDE Plasma5. More specifically, with Frameworks 5.90, Plasma 5.23.5 and Applications 21.12.1, meaning the very latest versions; all released this January 2022. This is topped with some cool additional Plasma5 applications like the Calligra office suite, Krita vector drawing program, Digikam photo management, KMymoney personal finance management, the Chromium- and Qt5-based webbrowser Falkon and Kyle LaTeX editor (and then some). And of course KDevelop.
  • The latest XFCE 4.16 is included and it’s a very nice step-up from the version of Slackware 14.2.
  • Wayland 1.20.0 is available as alternative to the venerable X.Org 1.20.14 which is still the default graphical display system.
  • The latest Mesa and Vulkan libraries are installed and GLvnd (the GL Vendor-Neutral Dispatch library) ensures that for instance NVIDIA’s binary driver is a breeze ti install and will not overwrite any system files.
  • Pulseaudio has competition from Pipewire which has been added to the distro and ships with a configuration that can mostly replace pulseaudio with this new low-latency audio server.
  • The whole build toolchain has seen a thorough refresher, with glibc 2.33, GCC 11.2.0 and LLVM 13.0.0 and the addition of Rust which is now at 1.58.1. Also essential tools that are part of modern-day software compilation like meson, ninja, scons and icecream are new in Slackware 15.0.
  • … and there’s a lot of other stuff that you have to find out for yourself.

Using Slackware

All in all, when you install Slackware 15.0 on your computer you will be able to work in graphical desktop environments and using tools that are on par with all the big distros. When looking for software that is not part of the core distro you can turn to slackbuilds.org (SBo) which is a curated platform for Slackware package build scripts. Tools like sbopkg, sbotools and slpkg will assist you in automating the build- and dependency resolving process when using these scripts from SBo.
Third-party package repositories are also available to quickly install binary packages if you do not trust yourself when compiling from source. You can think of my own alien and restricted packages but also Robby Workman’s package repository or the SlackOnly collection of packages that have been pre-compiled for you out of all the SBo scripts.
Slackware’s own slackpkg package manager which only deals with official Slackware packages can be extended with the plugin slackpkg+ if you want to be able to easily manage a mix of official and 3rd-party packages using a single tool.

Slackpkg with the slackpkg+ plugin also supports managing a multilib installation (i.e. a 64bit Slackware OS which is capable of running and compiling 32bit software). With multilib, running the Steam gaming platform is fully supported on your 64bit Slackware, and Steam applications run as fast on Slackware (or faster) than on Windows.

I hope you are all as excited about this new release of Slackware Linux as I am, and I hope you will have a lot of fun using it. Of course, those of you who have been running the ‘bleeding edge’ all the time, aka Slackware-current, can keep doing that. The Slackware-current tree is still there and it will now focus on post 15.0 development towards the next stable release.

Upgrading from 14.2

People upgrading from Slackware 14.2 have multiple options. You should definitely read the CHANGES_AND_HINTS file to get a good picture of the many package additions and removals. The UPGRADE.TXT file is an invaluable source if you want to perform a manual upgrade.
The Slackware Documentation Project contains an article called “upgrading Slackware to a new release“. It describes how to use the slackpkg tool (which is part of Slackware) to perform the upgrade with a large degree of automation.

Still, it is good to be aware that Slackware moved from ConsoleKit2 to elogind and one thing will be immediately obvious if you start an X session after upgrade and reboot:

With elogind as the session/seat manager instead of ConsoleKit2, if you run ‘startx’ or ‘startkwayland’ at the console, you won’t see a VT (virtual terminal) switch. In the past, your console TTY would usually be tty1 but your graphical session would start on tty7 and you would automatically be switched from tty1 to tty7. This is no longer true – the graphical session will re-use your console TTY.

You can read more details about the implications of the switch to elogind in my 2020 blog post here on Alien Pastures.

Get Slack

Download your official ISO images from any of the Slackware mirrors. We use mirrorbrain to ensure that a visit to mirrors.slackware.com will lead you to a server that is near to you.

Support Slackware

Please consider supporting Patrick and his family by visiting his Patreon Page, choosing a membership category and becoming a Patron. If you can spare a dollar or so per month, it will help keeping this unique distribution alive. All that money goes only to Pat.
If you are longing for the stuff that The Store used to sell, you can have a look at this CafePress URL which is Pat’s. Anything you buy there generates some money for him: so on you go to Patrick’s CafePress Shop.
Don’t fall for the fakes (several others on Cafepress sell clothing and gear with Slackware’s logo on them)!

Give Slackware a test-drive

Has the article made you curious or interested but you don’t yet feel comfortable installing Slackware on your computer?
We’ve got you covered!
There’s a Slackware Live Edition waiting for you. It is the same Slackware 15.0 but it will run entirely from a USB stick, won’t touch your harddisks and you don’t have to install anything.
Unlike Slackware itself, the Slackware Live Edition supports Secure Boot but you’ll have to enroll the liveslak certificate into your computer’s MOK first.
And if you do want to install Slackware after seeing the Live Edition, there’s a “Install Slackware” icon right there on the graphical desktop, which you can click (and use the password “live“!) to run through a slightly modified but still very familiar Slackware setup program (called ‘setup2hd‘ in the Live Edition).

Get the ISO for Slackware Live Edition 15.0 here: https://slackware.nl/slackware/slackware-live/slackware64-15.0-live/ (rsync via rsync://slackware.nl/mirrors/slackware/slackware-live/slackware64-15.0-live/). Instructions about installing the ISO to a USB stick and using Slackware Live can be found at https://liveslak.org/ or https://docs.slackware.com/slackware:liveslak#enduser_documentation .

Choosing development or stable

Slackware comes in two flavors:

  1. The stable releases (14.2 and 15.0 being the most recent ones) which get regular security updates but no changes in functionality;
  2. And the development release called ‘slackware-current’ which has a lot of activity with sometimes daily updates, new functionality and basically being there for all of us to test and find the bugs.

Note that the designation ‘-current‘ confuses some people who interpret “current” as “latest stable“. It is not!

Are you running ‘-current’ but want to switch to Slackware 15.0 and enjoy a stable experience with regular security updates but without the hassle of daily upgrades and occasional breakage? It’s as easy as opening the mirror list (/etc/slackpkg/mirrors) for your package manager slackpkg, and selecting a Slackware 15.0 mirror instead of a Slackware-current mirror.
To give an example: I myself would change the single un-commented line in that file  “http://slackware.nl/slackware/slackware64-current/” to  “http://slackware.nl/slackware/slackware64-15.0/“, and then run “slackpkg update ; slackpkg install-new ; slackpkg upgrade-all“. Now my computer is fully up-to-date with Slackware 15.0 and ready to receive future security upgrades.

Note that if you are not only interested in security updates but actually want to be able to run new software all the time, then the development release  “Slackware-current” is still the better option for you; but you need to be able to fix the occasional breakage which is inherent to walking the bleeding edge. If you want to switch from a stable release to -current, the process is simular to the example above: change the mirror line in “/etc/slackpkg/mirrors” to the URL of a slackware-current repository and be sure to read the documentation.

Looking forward

Things I hope to see in that next release post 15.0:

  • Out-of-the-box support for SecureBoot
  • The switch from Elilo to Grub as the default bootloader for UEFI platforms
  • Getting rid of the huge kernel and automatically generate and use initrd.gz images with generic kernels in Lilo and Grub
  • Full integration of Pipewire as an audio/video server replacing Pulseaudio and making Jack2 superfluous
  • A basic firewall as part of the initial installation

Not all of that will probably be achievable, but we’ll see…

In memoriam

A long-standing member of the Slackware core team passed away during the development process towards Slackware 15.0. Erik Jan Tromp (alphageek) would have loved to have been able to witness the release of Slackware 15.0 but circumstances decided otherwise. We said goodbye to him in March 2020.
He will be remembered fondly.

Steam client update, also fresh Wine, QEMU, MinGW-w64

It was already a while ago that I refreshed my ‘steamclient‘ package for Slackware.

The steamclient package is meant to bootstrap the installation of Valve’s Steam gaming platform on your Slackware computer. The package installs a couple of scripts and a 32-bit Linux runtime based on Ubuntu. When you first start ‘steam’ from the menu or from the X terminal commandline, the client will download a larger set of runtime libraries, including 64-bit support. Onwards, the client will keep its runtime libraries up-to-date automatically, every time it starts up and connects to the Steam servers.
The Slackware package has a couple of tweaks because we obviously do not have Ubuntu tools on board. As a result, on Slackware-current (32bit and 64-bit with multilib) Steam works out of the box.

The reason for a package refresh is a recent bug report on Valve’s github, about an ALSA related crash on Slackware. The root cause was eventually found and it was part of the customization I added to the steam launcher 6 years ago when we were still on release 14.1 and we did not have pulseaudio as part of the Operating System.

So I removed (actually, commented-out) these lines, and that should fix the root cause for that bug. If you do not use Pulseaudio or want to enforce ALSA sound regardless, just un-comment the relevant lines at the top of the ‘/usr/bin/steam’ script again – it’s self-explanatory.
I have also refreshed the READMEs for Slackware and additionally removed support for all Slackware versions older than 14.2. To be realistic, I assume that gamers are all on the -current platform already.

NOTE:
If you have an older ‘steamclient’ package installed on 64bit Slackware and use the slackpkg+ extension to manage 3rd-party repositories, you need to un-install the old steamclient package first. The old packages have a ‘i386’ architecture tag whereas the new one has a ‘i386’ architecture tag for the 32bit Slackware, and a x86_64 tag for use on 64bit Slackware.
They are the same package actually but I was asked to make ‘steamclient’ installable via slackpkg/slackpkg+ also on 64bit Slackware. So:

# removepkg steamclient
# slackpkg update
# slackpkg install steamclient

Have fun playing games on Steam!

Some other recent package updates in my Slackware-current repository are:

Wine: we’re up to version 6.23 now. The 32bit wine package is just that – 32bit Wine. My 64bit wine package contains both 64bit Wine and the 32bit WoW64 (Wine on Wine64). Both have the ‘staging‘ patches applied.
The external dependencies for this package remain the same: FAudio and vkd3d are required. On 64bit Slackware you need to have multilib installed. In addition to multilib, you need to convert the 32bit versions of the FAudio and vkd3d packages to ‘compat32’ packages and install those.

MinGW-w64: I have updated this package to v9.0.0_gcc11.2.0. Mingw-w64 is based on the original mingw.org project (which was created to support the GCC compiler on Windows systems), particularly adding 64-bit support.

The MinGW compiler suite is used to generate the native Windows DLLs in the wine package.

The QEMU machine emulator and virtualizer got a refresh because I am preparing (fingers crossed!) for a release of Slackware 15.0 which according to an online statement could happen soon after New Year 2022.
I run Slackware Virtual Machines (VMs) in QEMU and I wanted to take advantage of the newest QEMU features. At the moment my VM host is Slackware 14.2 and that has a fairly old QEMU package installed. The VM host will get an upgrade to Slackware 15.0 as soon as the new release is made available.
Dependencies for my qemu package are: vde (historically for non-root network bridge support) and as of now also: device-tree-compiler, jack2 (note that jack2 has its own sub-dependencies!) and virglrenderer.


Have fun with these during the holiday break, and I wish you all a Happy New Year.