My thoughts on Slackware, life and everything

Month: October 2019 (Page 1 of 2)

Using Let’s Encrypt to Secure your Slackware webserver with HTTPS

In the ‘good old days‘ where everyone was a hippy and everyone trusted the other person to do the right thing, encryption was not on the table. We used telnet to login to remote servers, we transferred files from and to FTP servers in the clear, we surfed the nascent WWW using http:// links; there were no pay-walls; and user credentials, well who’d ever heard of those, right.
Now we live in a time where every government spies on you, fake news is the new news, presidents lead their country as if it were a mobster organisation and you’ll go to jail – or worse – if your opinion does not agree with the ruling class or the verbal minority.
So naturally everybody wants – no, needs – to encrypt their communication on the public Internet nowadays.

Lucky for us, Linux is a good platform for the security minded person. All the tools you can wish for are available, for free, with ample documentation and support on how to use them. SSH secure logins, PGP encrypted emails, SSL-encrypted instant messaging, TOR clients for the darkweb, HTTPS connections to remote servers, nothing new. Bob’s your uncle. If you are a consumer.

It’s just that until not too long ago, if you wanted to provide content on a web-server and wanted to make your users’ communications secure with HTTPS, you’d have to pay a lot of money for a SSL certificate that would be accepted by all browsers. Companies like VeriSign, DigiCert, Komodo, Symantec, GeoTrust are Certificate Authorities whose root certificates ended up in all certificate bundles of Operating Systems, browsers and other tools, but these big boys want you to pay them a lot of money for their services.
You can of course use free tools (openssl) to generate SSL vertificates yourself, but these self-signed certificates are difficult to understand and accept for your users if they are primarily non-technical (“hello supportline, my browser tells me that my connection is insecure and your certificate is not trusted“).

SSL certificates for the masses

Since long I have been a supporter of CACert, an organization whose goal is to democratize the use of SSL certificates. Similar to the PGP web-of-trust, the CACert organization has created a group of ‘assurers‘ – these are the people who can create free SSL certificates. These ‘assurers’ are trusted because their identities are being verified face-to-face by showing passports and faces. Getting your assurer status means that your credentials need to be signed by people who agree that you are who you say you are. CACert organizes regular events where you can connect with assurers, and/or become one yourself.
Unfortunately, this grass-roots approach is something the big players (think Google, Mozilla) can not accept, since they do not have control over who becomes an assurer and who is able to issue certificates. Their browsers are therefore still not accepting the CACert root certificate. This is why my web site still needs to display a link to “fix the certificate warning“.
This is not manageable in the long term, even though I still hope the CACert root certificate will ultimately end up being trusted by all browsers.

So I looked at Let’s Encrypt again.
Let’s Encrypt is an organization which has been founded in 2016 by a group of institutions (Electronic Frontier Foundation, Mozilla Foundation, Michigan University, Akamai Technologies and Cisco Systems) who wanted to promote the use of encrypted web traffic by allowing everyone to create the required SSL certificates in an automated way, for free. These institutions have worked with web-browser providers to get them to accept and trust the Let’s Encrypt root certificates. And that was successful.
The result is that nowadays, Let’s Encrypt acts as a free, automated, and open Certificate Authority. You can download and use one of many client programs that are able to create and renew the necessary SSL certificates for your web servers. And all modern browsers accept and trust these certificates.

Let’s Encrypt SSL certificates have a expiration of 3 months after creation, which makes it mandatory to use some mechanism that does regular expiration checks on your server and renews the certificate in time.

I will dedicate the rest of this article to explain how you can use ‘dehydrated‘, a 3rd-party and free Let’s Encrypt client which is fully compatible with the official ‘CertBot’ client of Let’s Encrypt.
Why a 3rd-party tool and not the official client? Well, dehydrated is a simple Bash shell script, easy to read and yet fully functional. On the other hand, please have a look at the list of dependencies you’ll have to install before you can use CertBot on Slackware! That’s 17 other packages! The choice was easily made, and dehydrated is actively developed and supported.

I will show you how to download, install and configure dehydrated, how to configure your Apache web server to use a Let’s Encrypt certificate, and how to automate the renewal of your certificates. After reading the below instructions, you should be able to let people connect to your web-server using HTTPS.


Install dehydrated

The easiest way to install  dehydrated is to use the SlackBuilds.org script for it. It will install the script, create a default configuration, install a man-page and documentation.

# wget https://slackbuilds.org/slackbuilds/14.2/system/dehydrated.tar.gz
# tar xvf dehydrated.tar.gz
# cd dehydrated
# . dehydrated.info && wget $DOWNLOAD
# chmod +x dehydrated.SlackBuild
# ./dehydrated.SlackBuild
# installpkg /tmp/dehydrated-0.6.5-noarch-1_SBo.tgz

Installing the package will also create a cron job “/etc/cron.d/dehydrated” which makes dehydrated run once a day at midnight. I want that file to have some comments about what it does and I do not want to run it at midnight, so I overwrite it with a line that makes it run once a week at 21:00 instead. It will also log its activity to a logfile, “/var/log/dehydrated” in the example below:

cat <<EOT > /etc/cron.d/dehydrated
# Check for renewal of Let's Encrypt certificates once per week on Monday:
0 21 * * Mon /usr/bin/dehydrated -c >> /var/log/dehydrated 2>&1
EOT

Dehydrated uses a directory structure below “/etc/dehydrated/”.
The main configuration file you’ll find there is called “config”.
The file “domains.txt” contains the host- and domain names you want to manage SSL certificates for.
The directory “accounts” will contain your Let’s Encrypt user account and private key, once you’ve registered with them.
And a new directory “certs” will be created to store the SSL certificates you are going to create and maintain.

How to deal with these files is going to be addressed in the next paragraphs.

The dehydrated configuration files

config

The main configuration file “/etc/dehydrated/config” is well-commented, so I just show the lines that I used:

DEHYDRATED_USER=alien
DEHYDRATED_GROUP=wheel
CA="https://acme-staging-v02.api.letsencrypt.org/directory"
#CA="https://acme-v02.api.letsencrypt.org/directory"
CHALLENGETYPE="http-01"
WELLKNOWN="/usr/local/dehydrated"
PRIVATE_KEY_RENEW="no"
CONTACT_EMAIL=eric.hameleers@gmail.com
LOCKFILE="${BASEDIR}/var/lock"
HOOK=/etc/dehydrated/hook.sh

Let’s go through these parameters:

  • We are starting the ‘dehydrated script as root, via a cron job or at the commandline. The values for DEHYDRATED_USER and DEHYDRATED_GROUP are the user and group the script will switch to at startup. All activities will be done as user ‘alien’ and group ‘wheel’ and not as the user ‘root’. This is a safety measure.
  • CA: this contains the Let’s Encrypt URL for dehydrated to connect to. You’ll notice that I actually list two values for “CA” but one is commented out. The idea is that you use the ‘staging’ URL for all your tests and trials, and once you are satisfied with your setup, you switch to the URL for production usage.
    Also note that Let’s Encrypt expects clients to use the ACMEv2 protocol. The older ACMEv1 protocol will still work, but you can not register a new account using the old protocol. Its only use nowadays is to assist in migrating old setups to ACMEv2. The “CA” URL contains the protocol version number, and I highlighted that part in red.
  • CHALLENGETYPE : we will be using HTTP challenge type because that’s easiest to configure. Alternatively if you manage your own DNS domain you could let dehydrate update your DNS zone table to provide the challenge that Let’s Encrypt demands.
    What is this challenge? Let’s Encrypt’s ACME-protocol wants to verify that you are in control of your domain and/or hostname. It will try to access a verification file via a HTTP request to your webserver.
  • WELLKNOWN: this defines the local directory  where dehydrated creates the ‘challenge-tokens’ which are then served by your webserver. The Let’s Encrypt ‘ACME server’ will connect to your server as part of the ‘http-01’ challenge and expects to find a specific file there with specific content (created by dehydrated). In the case of a webserver running on our example domain “foo.net”, that URL would be  http://foo.net/.well-known/acme-challenge/m4g1C-t0k3n . The dehydrate client must provide that “m4g1c-t0k3n” file which it will create during a certificate creation or renewal. Below I will explain how to create this URL location “.well-known/acme-challenge” and make it readable for an external server like Let’s Encrypt.
    If your “domains.txt” file contains more than one hostname or domain, the ACME server will repeat this challenge for every one of them. Usually, multiple hostnames or (sub-)domains means that you have defined multiple VirtualHost in your Apache webserver configuration. For every VirtualHost you need to enable access to this ‘http-01’ challenge location (I will show you how, below).
    Note: The first connect from the ACME server will always be over HTTP on port 80, but if your site does a redirect to HTTPS, that will work.
  • PRIVATE_KEY_RENEW: whether you want the certificate’s private key to be renewed along with the certificate itself. I chose “no” but the default is “yes”.
  • CONTACT_EMAIL: the email address which will be associated with your Let’s Encrypt account. This is where warning emails will be sent if your certificate about to expire but has not been renewed.
  • LOCK: the directory (which must be writable by our non-root user) where dehydrated will place a lock file during operation.
  • HOOK: the path to an optional script that will be invoked at various parts of dehydrate’s activities and which allows you to perform all kinds of related administrative tasks – such as restarting httpd after you have renewed its SSL certificate.
    NOTE: do not enable this “HOOK” line – i.e. put a ‘#” comment character in front of the line – until you actually have created a working and executable shell script with that name! You’ll get errors otherwise about the non-existing script.

domains.txt

The file “/etc/dehydrated/domains.txt” contains the list hosts and domain names you want to associate with your SSL certificates. You need to realize that a SSL certificate contains the hostname(s) or the domain name(s) that it is going to be used for. That is why you will sometimes see a “hostname does not match server certificate” warning if you open a URL in your browser, it means that the remote server’s SSL certificate was originally meant to be used with a different hostname.

In our case, the “domains.txt” file contains just one hostname on a single line:

www.foo.net

… but that line can contain any amount of different space-separated hosts under the same domain. For instance the line could be “foo.net www.foo.net” which would tell Let’s Encrypt that the certificate is going to be used on two separate web servers: one with hostname “foo.net” and the other with the hostname “www.foo.net“. Both names will be incorporated into the certificate.

Your “/etc/dehydrated/domains.txt” file can be used to manage the certificates of multiple domains, each domain on its own line (e.g. domain foo.org on one line, and domain foo.net on another line). Each line corresponds to a different SSL certificate – e.g. for different domains. Every line can contain multiple hosts in a single domain (for instance: foo.org www.foo.org ftp.foo.org).

Directory configuration

Two directories are important for dehydrated, and we need to create and/or configure them properly.

/etc/dehydrated

First, the dehydrated configuration directory. We have configured dehydrated to run as user ‘alien’ instead of user ‘root’ so we need to ensure that the directory is writable by this user. Or better (since we installed this as a Slackware package and a package upgrade would undo an ownership change of /etc/dehydrated) let’s manually create the subdirectories “accounts” “certs”, “chains” and “var” where our user actually needs to write, and make ‘alien’ the owner:

# mkdir -p /etc/dehydrated/accounts
# chown alien:wheel /etc/dehydrated/accounts
# mkdir -p /etc/dehydrated/certs
# chown alien:wheel /etc/dehydrated/certs
# mkdir -p /etc/dehydrated/chains
# chown alien:wheel /etc/dehydrated/chains
# mkdir -p /etc/dehydrated/var
# chown alien:wheel /etc/dehydrated/var

/usr/local/dehydrated

The directory “/usr/local/dehydrated” is the location where dehydrated to will generate the Let’s Encrypt challenge files. These files provide the proof that we actually own the domain(s) we are requesting a certificate for.
So let’s create that directory and allow our non-root user to write there:

# mkdir -p /usr/local/dehydrated
# chown alien:wheel /usr/local/dehydrated

SUDO considerations

We configured the dehydrated script to drop its root privileges at startup and continue as user ‘alien’, group ‘wheel’. Because we also change the group iit is important that the sudo line for root in the file “/etc/sudoers” is changed from the default:

#root ALL=(ALL) ALL

to

root ALL=(ALL:ALL) ALL

Else you’ll get the error “Sorry, user root is not allowed to execute ‘/usr/bin/dehydrated -c’ as alien:wheel on localhost.“.

Apache configuration

I expect that you have already setup your Apache for un-encrypted connections and already have a web site. If you still need to figure out how to setup a web site using Apache, I suggest you look for a good tutorial before you proceed with my article, like https://docs.slackware.com/howtos:network_services:setup_apache .

Before we register an account with Let’s Encrypt and start generating certificates, let’s first update our existing Apache configuration so that it works with dehydrated. We need to make the ‘http-01’ challenge location (http://foo.net/.well-known/acme-challenge/) accessible to external web clients, else the certificate generation will fail.

Note that the above example mentions the “foo.net” hostname. If your “/etc/dehydrated/domains.txt” contains lines with multiple hosts under a domain, you’ll have to make the URL path component “/.well-known/acme-challenge” accessible through every domain host you configured in Apache. The complete certificate generation process will fail in case any of these challenge URLs cannot be validated.
To make life more simple if you run multiple web servers, we created “/usr/local/dehydrated/” to store the challenge file. It’s a single file location.  With the help of the Apache “Alias” directive we can use that single file location in all our web servers.

Use this snippet of text in the <VirtualHost></VirtualHost> configuration block for every webserver host:

# We store the dehydrated info under /usr/local and use an Apache 'Alias'
# to be able to use it for multiple domains. You'd use this snippet:
Alias /.well-known/acme-challenge /usr/local/dehydrated
<Directory /usr/local/dehydrated>
    Options None
    AllowOverride None
     Require all granted
</Directory>

You can use “lynx” on the command-line to test whether a URL is valid:

$ lynx -dump http://www.foo.net/.well-known/acme-challenge/
Forbidden: You don't have permission to access /.well-known/acme-challenge/ on this server.

Despite that error, this message actually shows that the URL works (otherwise the return message would have been “Not Found: The requested URL /.well-known/acme-challenge was not found on this server.“).

This completes the required Let’s Encrypt modifications to your Apache web server configuration.
Next, and before we restart ‘httpd‘, our Apache server must be enabled to accept SSL connections. This is achieved by un-commenting the following line in “/etc/httpd/httpd.conf”:

# Secure (SSL/TLS) connections
Include /etc/httpd/extra/httpd-ssl.conf

You can now restart Apache httpd to activate our modifications (but always test the syntax of your configuration first:

# apachectl configtest
# /etc/rc.d/rc.httpd restart

To end the Apache configuration instructions, here are the bits that define the SSL parameters for your host. Note that you should not add them yet! You do not have a SSL certificate yet. Only after you have executed “dehydrated -c” and obtained the certificates, you can add the following lines to every <VirtualHost</VirtualHost> block where where you previously added the ‘Alias’ related stuff above:

SSLEngine on
SSLCertificateFile /etc/dehydrated/certs/foo.net/cert.pem
SSLCertificateKeyFile /etc/dehydrated/certs/foo.net/privkey.pem
SSLCertificateChainFile /etc/dehydrated/certs/foo.net/chain.pem
SSLCACertificatePath /etc/ssl/certs
SSLCACertificateFile /etc/ssl/certs/ca-certificates.crt

Note the hostname “foo.net” in these SSL lines above? This is an example of course and you need to change that to your own hostname.
What you need to realize is that this name corresponds to the first name of the line in your “/etc/dehydrated/domains.txt” file. Earlier in the article I used an example line for this “domains.txt” file which looks like this: “foo.net www.foo.net“. Even more hosts are possible, they should be space-separated. A single certificate will be generated which is valid for all of these hosts, and the directory where they are stored in is “/etc/dehydrated/certs/” followed by “./foo.net” which the name of that first entry of the line.

Running dehydrated for the first time, using the Let’s Encrypt staging server:

With all the preliminaries taken care of, we can now proceed and run ‘dehydrated’ for the first time. Remember to make it connect to the Let’s Encrypt ‘staging’ server during all your tests, to prevent their production server from getting swamped with bogus test requests!

Examining the manual page (run “man dehydrated“) we find that we need the parameter ‘–cron’, or ‘-c’, to sign/renew non-existent/changed/expiring certificates:

# /usr/bin/dehydrated -c
# INFO: Using main config file /etc/dehydrated/config
# INFO: Running /usr/bin/dehydrated as alien/wheel
# INFO: Using main config file /etc/dehydrated/config

To use dehydrated with this certificate authority you have to agree to their terms of service which you can find here: https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf

To accept these terms of service run `/usr/bin/dehydrated --register --accept-terms`.

What did we learn here?
In order to use dehydrated, you’ll have to register first. Let’s create your account and generate your private key!

Do not forget to set the “CA” value in /etc/dehydrated/config to a URL supporting ACMEv2. If you use the old staging server URL you’ll see this error: “Account creation on ACMEv1 is disabled. Please upgrade your ACME client to a version that supports ACMEv2 / RFC 8555. See https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430 for details.

With the proper CA value configured (you’ll have to do this both for the staging and for the production server URL) , you’ll see this if you run “/usr/bin/dehydrated –register –accept-terms”:

# /usr/bin/dehydrated --register --accept-terms
# INFO: Using main config file /etc/dehydrated/config
# INFO: Running /usr/bin/dehydrated as alien/wheel
# INFO: Using main config file /etc/dehydrated/config
+ Generating account key...
+ Registering account key with ACME server...
+ Fetching account ID...
+ Done!

Generate a test certificate

We’re  ready to roll. As said before, it is proper etiquette to run all your tests against the Let’s Encrypt ‘staging’ server and use their production server only for the real certificates you’re going to deploy.
Let’s run the command which is also being used in our weekly cron job, “/usr/bin/dehydrated -c”:

# /usr/bin/dehydrated -c
# INFO: Using main config file /etc/dehydrated/config
# INFO: Running /usr/bin/dehydrated as alien/wheel
# INFO: Using main config file /etc/dehydrated/config
+ Creating chain cache directory /etc/dehydrated/chains
Processing www.foo.net
+ Creating new directory /etc/dehydrated/certs/www.foo.net ...
+ Signing domains...
+ Generating private key...
+ Generating signing request...
+ Requesting new certificate order from CA...
+ Received 1 authorizations URLs from the CA
+ Handling authorization for www.foo.net
+ Found valid authorization for www.foo.net
+ 0 pending challenge(s)
+ Requesting certificate...
+ Checking certificate...
+ Done!
+ Creating fullchain.pem...
+ Done!

This works! You can check your web site now if you did not forget to add the SSL lines to your VirtualHost block; your browser will complain that it is getting served an un-trusted SSL certificate issued by “Fake LE Intermediate X1“.

Generate a production certificate

First, change the “CA” variable in “/etc/dehydrated/config” to the production CA URL “https://acme-v02.api.letsencrypt.org/directory”.
Remove the fake certificates that were created in the previous testing step so that we can create real certificates next:

# rm -r /etc/dehydrated/certs/www.foo.net

Now that we’ve cleaned out the fake certificates, we’ll generate real ones:

# /usr/bin/dehydrated -c
# INFO: Using main config file /etc/dehydrated/config
# INFO: Running /usr/bin/dehydrated as alien/wheel
# INFO: Using main config file /etc/dehydrated/config
Processing www.foo.net
+ Creating new directory /etc/dehydrated/certs/www.foo.net ...
+ Signing domains...
+ Generating private key...
+ Generating signing request...
+ Requesting new certificate order from CA...
+ Received 1 authorizations URLs from the CA
+ Handling authorization for www.foo.net
+ 1 pending challenge(s)
+ Deploying challenge tokens...
+ Responding to challenge for www.foo.net authorization...
+ Challenge is valid!
+ Cleaning challenge tokens...
+ Requesting certificate...
+ Checking certificate...
+ Done!
+ Creating fullchain.pem...
+ Done!

If you reload the Apache server configuration (using the command “apachectl -k graceful”) you’ll now see that your SSL certificate has been signed by “Let’s Encrypt Authority X3” and it is trusted by your browser. We did it!

Automatically reloading Apache config after cert renewal

When your weekly cron job decides that it is time to renew your certificate, we want the dehydrated script (which runs as a non-root account) to reload the Apache configuration. And of course, only root is allowed to do so.

We’ll need a bit of sudo magic to make it possible for the non-root account to run the “apachectl” program. Instead of editing the main file “/etc/sudoers” with the command “visudo” we create a new file “httpd_reload” especially for this occasion, in sub-directory “/etc/sudoers.d/” as follows:

# cat <<EOT > /etc/sudoers.d/httpd_reload
alien ALL=NOPASSWD: /usr/sbin/apachectl -k graceful
EOT

This sudo configuration allows user ‘alien’ to run the exact command “sudo /usr/sbin/apachectl -k graceful” with root privileges.

Next, we need to instruct the dehydrated  script to automatically run “sudo /usr/bin/apachectl -k graceful” after it has renewed any of our certificates. That is where the “HOOK” parameter in “/etc/dehydrated/config” comes to play.

As the hook script, we are going to use dehydrated’s own sample “hook.sh” script that can be downloaded from https://raw.githubusercontent.com/lukas2511/dehydrated/master/docs/examples/hook.sh or (if you used the SlackBuilds.org script to create a package) use “/usr/doc/dehydrated-*/examples/hook.sh”.

# cp -i /usr/doc/dehydrated-*/examples/hook.sh /etc/dehydrated/
# chmod +x /etc/dehydrated/hook.sh

This shell script contains a number of functions, each is relevant and will be called at a certain stage of the certificate renewal process. The dehydrated script will provide several environment variables to allow a high degree of customization, and all of that is properly documented in the sample script, but we do not need any of that. Just at the end of the “deploy_cert()” function we need to add a few lines:

deploy_cert() {
# ...
# After successfully renewing our Apache certs, the non-root user 'alien'
# uses 'sudo' to reload the Apache configuration:
sudo /usr/sbin/apachectl -k graceful
}

That’s all. Next time dehydrated renews a certificate, the hook script will be called and that will reload the Apache configuration at the appropriate moment, making the new certificate available to visitors of your web site.

Summarizing

I am glad you made it all the way down here! In my usual writing style, the article is quite verbose and gives all kinds of contextual information. Sometimes that makes it difficult for the “don’t bother me with knowledge, just show me the text I should copy/paste ” user but I do not care for that.

I do hope you found this article interesting, and useful. If you spotted any falsehoods,let me know in the comments section below. If some part needs more clarification, just tell me.

Have fun with a secure web!

Eric

Continue reading

How to access your e-books online

In a previous article about the Calibre e-book library management program, I wrote that I did away with USB cable connections to transfer e-books from my Calibre library to my E-reader. Instead, I made my e-book library accessible on-line and now I am able to download my books securely over the Internet – no matter where I am, as long as I have a network connection.

In this article today, I am going to explain how I make my Calibre library accessible on-line.

Understandably, the article will be Slackware-centric, because that is where I have my Calibre library and where I run all the programs that I need. But, the technology and the configuration is generic enough that readers of this article running another distro or even employing an embedded (ARM) device will be able to make it work.

Which software is used?

Apache and PHP are standard part of Slackware Linux. I assume that you know how to setup an Apache web server. The good news is that Slackware’s default httpd configuration works out of the box. However, it does not enable a secure (https) web site. That part is on you.
If your content server is going to be used in your home network only, then HTTPS is not even needed.
I have a package for Calibre in my repository which of course you will definitely need if you want to build an e-book library.
You’ll have to download the ZIP file of the latest COPS release from https://github.com/seblucas/cops/releases – the ZIP contains the extra stuff that is not contained in the source .tar.gz file.
The remainder is just configuration.

Now let me explain my setup.

The Calibre library

It all starts with Calibre of course. It is a desktop program, available for Linux, Windows and MacOS. It stores your e-book library as a directory tree.  The directories immediately below the root of the tree are the book authors. In the root of the directory tree, Calibre creates a SQLite database file containing all the library metadata. The simple directory structure and the SQLite database file make that a Calibre library is independent of the Operating System that Calibre runs on. Its database scheme can be updated between major releases though. From within the Calibre GUI, it is however trivial to re-create this database file from scratch if it becomes corrupted.
This simple library setup has the advantage that if you store the library on a Cloud service like Dropbox, you can have access to your library from multiple computers running different Operating Systems (but preferably all the same version of Calibre). As long as one person accesses the library at any time, the Calibre database file will not get corrupted. This is how I have setup Calibre for my wife so that she can manage her e-books from every computer in the house.

Where to create the library

Myself, I use Calibre on a Linux machine which runs 24/7 (my package-build server in fact). I run a XFCE graphical desktop session inside a VNC server and that allows me to work in a desktop environment that’s always there for me. I use NoVNC to access this desktop using a web browser and that is perfect for corporate envronments which put a firewall up for everything except HTTPS traffic. Perhaps I’ll write another article sometime, explaining how to set that up.

What matters is that the Calibre library is stored on a machine with Internet access, and that this computer is running a web server like Apache. In Slackware, Apache httpd is the default webserver program.
I made this web server accessible from the Internet by configuring my ISP’s Internet router to forward ports 80 (http) and 443 (https) to the internal IP address of my Slackware machine.
For the sake of this article, let’s assume that two separate Calibre libraries exist on this computer, mine is located in /data/Calibre/EricLibrary and my wife’s books are stored in /data/Calibre/WifeLibrary . In Calibre itself, you can easily switch between libraries so that me and my wife are both able to maintain a separate collection of books. Our book tastes have no overlap…

How to configure the web server

I want both libraries to be accessible online.
I do not want to use Calibre’s own content server but instead use COPS. COPS is a collection of PHP scripts, and it does not require the Calibre program to function. It just needs access to the physical directory containing your Calibre library and it understands the Calibre SQLite database format.
If you want to get creative, you can install COPS on an Internet server – e.g. if you do not have a webserver at home. In that case, use a cloud storage service like Dropbox to keep your library in sync between your computer at home and that remote server running Apache.

My example setup is a Slackware computer at home with Apache enabled:

# chmod +x /etc/rc.d/rc.httpd
# /etc/rc.d/rc.httpd start

Apache uses the concept of a DocumentRoot, which is a directory accessible by the httpd server. Everything stored below that DirectoryRoot will be accessible via the URL of the webserver. Never store files below the DocumentRoot that you do not want to share!
I downloaded and extracted the COPS zip file directly into the Apache DocumentRoot directory (the default in Slackware is configured in the file ‘/etc/httpd/httpd.conf’ to be “/srv/httpd/htdocs/” but you can change that default location of course):

# cd /tmp
# links https://github.com/seblucas/cops/releases/download/1.1.3/cops-1.1.3.zip
# cd /srv/httpd/htdocs/
# unzip /tmp/cops-1.1.3.zip
# ln -s cops-1.1.3 mybooks
# cd mybooks
# cat <<EOT > config_local.php
$config = array();
$config['calibre_directory'] = array ("Eric" => "/data/Calibre/EricLibrary/", "Wife => "/data/Calibre/WifeLibrary/");
$config['cops_title_default'] = "Aliens Library";
$config['cops_use_url_rewriting'] = "1";
$config['cops_recentbooks_limit'] = '100';
$config['cops_update_epub-metadata'] = "1";
$config['cops_books_filter'] = array ("Books" => "!News");
EOT

The lines from the “cat << EOT” until the “EOT” on its own line is called a “here document“. You can copy/paste these lines into a terminal and it will create the file “config_local.php” with the content as shown between the “EOT” lines.
This is a configuration which allows access to two separate libraries. If you access the “recently added” section of a library it will show the latest 100 books instead of the default 20 books. It does some URL re-writing to generate cleaner URLs and it will not show any News sources (in Calibre, you can add various paid-for and free newspaper subscriptions and those will then be dpwnloaded daily, but I do not want to see those online).
Compare this configuration of mine to the ‘config_local.php.example’ file in the COPS directory. The above is not all you can configure, although it is sufficient. All configurable parameters are documented in ‘config_default.php’ which also defines their default values.

You’ll have noticed that I created a symlink to the extracted cops files, which allows me to use an easy to remember URL to access my books. Suppose my apache server is accessible as http://foo.net/ – then my COPS installation will be accessible at http://foo.net/mybooks/ .

That’s all!

Under the above ‘mybooks’ URL you will now find a COPS installation with two Calibre libraries with separate names “Eric” and “Wife”. Pick a nice template and theme, and try out the ebook-reader which is part of COPS (you  can read EPUBs in a web browser, no E-reader required).

Accessing COPS through the OPDS protocol

You want to configure your E-reader to access the library. An E-reader which contains a webbrowser is easy, just open http://foo.net/mybooks/ and click a book to download it.
Other E-readers, like FBReader for Android, are able to connect to an OPDS (Open Publication Distribution System) catalog. The COPS program exposes your library as a OPDS catalog if you just add ‘feed.php’ to the URL.
I.e. enter http://foo.net/mybooks/feed.php into the Network Library menu of FBReader and you’ll get instant access to “Alien’s Library” where you can browse Authors, Series, Categories, Recent Additions and read book descriptions, and then download the books you want to read.
If you want to always start with a list of recent library additions, you  can add ‘?page=10’ at the end of the URL so that it becomes http://foo.net/mybooks/feed.php?page=10

Hiding the cops files from the Apache DocumentRoot

Some people feel a bit anxious having these cops files accessible in their DocumentRoot – suppose someone gains access to them trough other means. You can use Apache’s “Alias” directive to install COPS in e.g. /usr/local instead. In that case, add the following lines to your /etc/httpd/httpd.conf or whatever file holds your website definition:

    <Directory "/usr/local/cops-1.1.3/">
        AllowOverride All
        Options +ExecCGI +FollowSymLinks
        Require all granted
    </Directory>
    Alias /mybooks/ /usr/local/cops-1.1.3/

You should check the validity of your httpd configuration before restarting the webserver:

# apachectl configtest
# /etc/rc.d/rc.httpd restart

And the URL to access your library will remain the same, i.e. http://foo.net/mybooks/feed.php?page=10 , only in this case your actual DocumentRoot directory will be empty.

Adding some sense of security

After following the above instructions, you now have an e-book library online whose access is not restricted in any way. If some search engine passes by it will neatly catalog all your books for other people to find. Oops!

You’ll need a basic access restriction at least. Apache offers a ‘Basic Authentication’ setup which will prompt anyone who tries to access your library for a valid account and password.
To achieve this, let’s first create a password file containing the account/password combinations for the people you want to grant access. For the sake of this example, I will grant myself and my wife access by creating account/password combinations for us in a file which we will then use in  our Apache configuration.
The first ‘htpasswd’ command below has an additional ‘-c’ parameter to create the file “/etc/httpd/passwords/htaccess.opds”:

# mkdir -p /etc/httpd/passwords/
# htpasswd -b -c /etc/httpd/passwords/htaccess.opds eric ericspassword
# htpasswd -b /etc/httpd/passwords/htaccess.opds wife wifespassword

The file will then have these contents, you see that the passwords are now MD5 encrypted:

eric:$apr1$OtKDA27W$l2ac4DAhGCG53igy6jT5A/
wife:$apr1$Zql/HEUC$wrNckoe57YPC0u2w8mL/M0

Read “man htpasswd” to find out more about this command and its parameters.

Next you need to change the Apache <Directory></Directory> block which I provided above. It will become like this:

    <Directory "/usr/local/cops-1.1.3/">
        AllowOverride All
        Options +ExecCGI +FollowSymLinks
        AuthBasicAuthoritative off
        AuthUserFile /etc/httpd/passwords/htaccess.opds
        AuthType Basic
        AuthName "OPDS Server"
        Require valid-user
        # Require all granted
    </Directory>
    Alias /mybooks/ /usr/local/cops-1.1.3/

Once you made these changes, validate the configuration and restart the webserver:

# apachectl configtest
# /etc/rc.d/rc.httpd restart

Now you will be greeted by an authentication request, next time you access your library. Only those users whose username/password combinations are stored in “/etc/httpd/passwords/htaccess.opds” will be able to get access.

Using encrypted HTTP

You should realize that these passwords will still be transmitted in cleartext if your webserver is not using HTTPS. It is possible for people to sniff the network connection and find your credentials.

I think however that instructions about enabling HTTPS for your Apache belongs in another blog post. Let me know in the comments section below if you have a need for such an article.
Also, let me know if parts of the above instructions are too cryptic and I will update the text where needed.

Good luck! Eric

Chromium 78 for Slackware

This week, Google released the first 78 version in the “stable” channel of their Chromium sources – the basis of Chrome, Opera, Vivaldi and even the Edge browsers, and of course the Chromium open-source browser itself. The release notes contain a fairly long list of security issues (CVE’s) which were taken care of.

I built packages for you today, so that you can enjoy the latest&greatest Chromium browser on Slackware 14.2 or -current.

 

What’s new in Chromium 78?

  • Tab hover cards. If you have a multitude of browser tabs open, it becomes difficult to recognize which tab has what page open. From now on, if you move your mouse over a tab you will see a small ‘hover card’ showing the title and the hostname of the page you have open in there.
  • Dark mode. You can now force ‘dark mode’ on every web site, whether the web site supports it or not. The web site code is not changed at all, and Chromium will perform a smart color inversion. You can choose between several algorithms. This is an experimental feature still, so you will have to enable it via chrome://flags like so:
    chrome://flags/#enable-force-dark
  • Chrome Password Safety tool which was available as an extension since February of this year has now been folded into the core of the browser. This feature will inform you of weak, leaked or re-used passwords in your list of saved passwords. Later on, Google is going to add functionality that will warn you when you try to use a password which is leaked online – even if you do not save your passwords in the browser.
    Still an experimental feature in this version of Chromium, you will have to enable it first via chrome://flags:

    chrome://flags/#password-leak-detection
  • A new extended menu which appears if you click “customize” in the lower right corner of the ‘new tab‘ page. Decorate the ‘new tab’ with your own background image or a different theme. This is still an experimental functionality so you have to enable it explicitly via chrome://flags :
    chrome://flags/#ntp-customization-menu-v2
    chrome://flags/#chrome-colors
  • Just missed the deadline for Chrome 78:
    DNS-over-HTTPS (DoH): Google has started a field test of its newly developed technology to tunnel your DNS queries through regular HTTPs requests. This security feature will be invaluable to those of you who do not want to risk 3rd parties to sniff your DNS requests for instance on a public Wi-Fi network. It will also prevent potential DNS spoofing attacks.
    This  Google blog article about DoH explains the caveats of using their implementation and the difference with the Mozilla Firefox implementation of DoH. Firefox enables DoH by ecapsulating your DNS requests in HTTPs and sending those to a CloudFlare server, while Chrome honours your existing DNS configuration (like paternal controls and safe browsing). The Chrome browser will check if your DNS provider is among a list of DoH-compatible providers, and swiches to your DNS provider’s equivalent DoH service. If the DNS provider isn’t in the list, Chrome will keep using regular DNS requests. Therefore, you’ll have to actively check whether this feature is going to work for you.
    As said, that same blog page mentions that this feature did not make Chrome 78 at the last moment.
    Also good to know is that the field trial is only going to be enabled for a small percentage of Chrome users (and not on Linux or iOS at all) – once it gets enabled. Not sure if this is going to be available to all of us Linux Chromium users any time soon.

Enjoy the new browser release!

Eric

Calibre 4.2.0 for Slackware with no external dependencies

One of the programs I use a lot on all my computers is Calibre, the E-book library management system. My wife and I both have Kobo e-readers, and on my Android phone I use FBReader to read my books on the road whenever I have some free time.

Having Calibre on a computer still requires a USB cable to transfer e-books from the computer to your ereader/phone, but networked alternatives exist. Calibre comes with a content server which can make your e-book library accessible online (on your LAN or on the Internet). Earlier versions of the Calibre content server were too resource-intensive and therefore I have chosen another solution for online access to my books. That’s COPS. which is short for “Calibre OPDS PHP Server“. OPDS (Open Publication Distribution System) Is the protocol through which E-readers can access online libraries. COPS allows me to download new books to read from my Calibre library over the wireless network to both my Kobo e-reader and to FBReader on my phone. No more cables needed!

FBReader on Android

FBReader accesses e-book library via COPS

Calibre is of course more than a library manager and a content server – the software comes bundled with its own e-book reader, a format converter, and an e-book editor!
Calibre is written in Python. For its graphical interface it heavily uses PyQt5 which in turn is based on Qt5 libraries.
I have been offering calibre packages in my repository for over eight years, starting with the 0.7.20 release as far as I could see (then still Qt4 based). This roughly coincided with my wife buying her first E-reader. The program has evolved a lot over the years and is much better than the proprietary library management software that ships with the major E-reader hardware.

Kovid Goyal is the primary developer, and he released a new major version of Calibre a few weeks ago. The new 4.x series delivers a completely re-written reader, a new content server with basically the same management and editing capabilities as the calibre desktop program, and a stark increase in amount and version numbers of the dependent software. For instance, Calibre 4 now depends on features in Qt 5.13.0 and PyQt 5.13.0. Several new pieces of sofware are listed in the dependencies list now, which were not required for Calibre 3.

The challenge was to create a solid Slackware package for Calibre 4. I know, you can simply download pre-built binaries for a generic Linux platform, and SlackBuilds.org even offers a build script to package these binaries for you, but it is much more fun, and very educational, to compile all of it yourself. In addition you’ll get binaries which are native to your Slackware distro instead of having to resort to binaries that were compiled on Ubuntu or Debian and do unpeakable things to your system when you’re not looking.

Luckily I have taken a week off from work, so I had time to spend on a rewrite of calibre.SlackBuild.

I also decided to do away with external dependencies. Until now, you would have to install podofo and unrar in order to fully use Calibre’s functionality. WIth my new calibre-4 package all dependencies are bundled inside. This includes Python 2.7.17 and Qt 5.13.1 as well as podofo and unrar libraries. This bundling makes it possible to build a fully functional Calibre on Slackware 14.2 (I had to add a newer version of the hunspell spell-checker because Slackware 14.2 has a version that is too old).

You can still compile a calibre package without all these internal libraries. For instance if you already have Qt 5.13.1 installed you can reduce the size of the package significantly by recompiling it. The calibre.SlackBuild will find your Qt5 libraries and won’t compile an internal version then. Same for Python 2.7. But I would recommend building the package always with the python interpreter and the slew of python modules included. It does not take that much time and you’ll be independent of updates in Slackware. Also, I do not offer packages for all those required Python modules anyway.

Here’s a couple of example command lines in case you would want to recompile. If you want create a package without any external dependency:

# BUILD_QT=YES BUILD_PYTHON=YES BUILD_MTP=YES  ./calibre.SlackBuild

In case you already have Qt 5.13.0 or newer installed and want to shrink the size of the package:

# BUILD_PYTHON=YES ./calibre.SlackBuild

Note:
On Slackware 14.2 you will have to install meson, python3 and python3-setuptools from my repository or else the compilation will fail.  Slackware-current contains all of them already.

Note:
If you have issues with Calibre 4 and think you can convince me to re-add a Calibre 3 package to my repository, let me know in the comments section below.

Note:
If you use XFCE or some other desktop environment which does not support StatusNotifier (and only supports the old X11 based XEmbed specification like the systray), then Calibre will not be able to display its system tray icon. You will have to install ‘xfce4-statusnotifier-plugin’ and its dependency ‘libdbusmenu-gtk’. Both are available as packages in my repository. After installing, right-click on the XFCE status bar, select ‘Panel > Add new items’ and add ‘Status Notifier Plugin’.

Get the new ‘calibre-4.2.0‘ package from my repository (or any mirror) and enjoy your e-book library!

If you want to know how to properly setup COPS and serve your e-book library on the internet (with or without authentication) let me know and I will devote an article to the topic. Perhaps in more generic terms, I should address the topic of reverse proxying.

Eric

KDE Plasma 5 – Slackware October release

I had already finished compiling KDE-5_19.10 and was waiting for the Plasma 5.17 public release announcement, when Pat upgraded libdvdread in slackware-current. That could mean trouble because of the dreaded ‘Shared library .so-version bump‘ message.
But he added the older libdvdread.so.4 library to aaa_elflibs so that the k3b program in Plasma5 does not break, and hopefully it remains in there until after I recompile k3b (which ultimately happens for the Plasma5 November release).

Unfortunately the earlier update of the ‘icu4c’ package broke some other stuff in Plasma5 as well. Be sure to install my ‘icu4c-compat‘ package, which contains the libraries from several older icu4c packages. Read my older article on ‘shared library .so version bumps‘ if you have not already done so, to understand the causes for this breakage.

The packages for KDE-5_19.10 are available for download from my ‘ktown‘ repository. As always, these packages are meant to be installed on a full installation of Slackware-current which has had its KDE4 removed first. These packages will not work on Slackware 14.2.

What’s new in the October 2019 release

This month’s KDE Plasma5 for Slackware contains the KDE Frameworks 5.63.0, Plasma 5.17.0 and Applications 19.08.2. All this on top of Qt 5.13.1.

Deps:
The ‘cracklib’ package got a version bump, and the latest ‘phonon’ and ‘phonon-vlc’ releases have been packaged.
The telepathy dependencies have been removed completely. Indeed, the feedback on my question in the README for last month’s ‘ktown’ release made clear that no one uses KDE Telepathy. For me it never worked anyway, so this month we say good-bye to KDE Telepathy and its dependencies.
Note that ‘qt5’ and ‘qt5-webkit’ should really be recompiled to fix the icu4c broken dependency, but I do not have the time right now, and the icu4c-compat package will take care of this anyway. Soon, though.

Frameworks:
Frameworks 5.63.0 is a regular update release. See: https://www.kde.org/announcements/kde-frameworks-5.63.0.php, but there is something worth mentioning still: the packages ‘kcalcore’ and ‘kcontacts’ which were part of KDE Applications and which you would find in the kde/kdepim section of my ‘ktown’ repository, have moved to the KDE Frameworks. As part of this move, ‘kcalcore’ was also renamed to ‘kcalendarcore’.

Plasma:
Plasma 5.17.0 is the start of a new release cycle of the Desktop part of KDE. See https://www.kde.org/announcements/plasma-5.17.0.php. Some take-aways from the release notes: the Plasma startup script (/usr/bin/startkde) which was traditionally a bash script has been replaced with a C++ program which is faster than the interpreted shell script code, and also starts the various services in parallel. The devs claim that Plasma5 desktop starts up a lot faster as a result. Do you feel the same?
Chrome/Chromium should blend in more with the Breeze theme and GTK applications should have the KDE color scheme applied. There’s more to read, just follow the above link.

Plasma-extra;
I updated ‘latte-dock’ which is my default application launcher here on the laptop for a couple of months now.
Note that ‘sddm-qt5’ should really be recompiled against the new icu4c in slackware-current, but like with qt5, my ‘icu4c-compat’ package will fix the breakage for now. This one is on my TODO list for next week.

Applications;
Applications 19.08.2 is a stability and bug-fix update for the 19.08 cycle. For more information, see https://www.kde.org/announcements/announce-applications-19.08.2.php and you may still want to visit the original release notes for 19.08.0 as well.

Applications-extra:
I upgraded ‘digikam’, ‘libktorrent’, ‘ktorrent’, ‘alkimia’, ‘kmymoney’, ‘kpmcore’, ‘krita’, ‘okteta’, and the development suite ‘kdevelop’, ‘kdev-php’ and and ‘kdev-python’ to their latest releases.

Telepathy:
KDE Telepathy is no longer part of my ‘ktown’ distribution of KDE Plasma5.

Where to get it

Download the KDE-5_19.10 from the usual location at https://slackware.nl/alien-kde/current/latest/ . Check out the README file in the root of the repository for detailed installation or upgrade instructions.

Development of Plasma5 is tracked in git: https://git.slackware.nl/ktown/ .
A new Plasma5 Live ISO has been uploaded and you will find it at https://slackware.nl/slackware-live/latest/ (rsync://slackware.nl/mirrors/slackware-live/latest/)

Have fun! Eric

« Older posts

© 2024 Alien Pastures

Theme by Anders NorenUp ↑