My thoughts on Slackware, life and everything

Tag: epub (Page 1 of 3)

Calibre 6.14.1 – I finally made it past 6.11…

News for people who no longer read paper books!
Well 😉 I guess everyone needs paper from time to time, but in all honesty, my eyes are getting worse and an E-reader is the optimal device for me when I want to read books – I can enlarge the font and it has a night light.

Calibre 6.x is a really cool-looking and versatile graphical Python3 application, using PyQt to build the graphical interface based on Qt6 widgets. It is the application to use if you have a collection of E-books and an E-reader, and want a decent library management program to move these E-books onto your E-reader. In addition, it offers a good ebook-viewer application for your computer and even an impressively powerful E-book editor.

As you may know, I have been creating native Slackware packages for Calibre for many years. However I had some issues with the compilation of any Calibre release past version 6.11.0. Kovid Goyal, the developer of this e-book management software, removed some of the content from his release tarballs and since version 6.12.0 requires that the build process generates these required files instead.

This meant that I had to study the changes in the Python files which are used to build Calibre to discover how the missing pieces where getting downloaded and compiled during a ‘bootstrap’ build. That took a while but I found a way to get these sources in place before starting the build.
So now again, Calibre packages for Slackware 15.0 and -current will be getting refreshes in my repository.

Download the new Calibre 6.14.1 packages from my repository or any mirror (like my own US mirror). No external dependencies, works out of the box on Slackware 15.0 or -current.

 

Have fun! Eric

First package for Calibre6 in my repository

Not so very long after I was finally able to produce my first packages for Calibre 5.x, Kovid Goyal ended that development cycle and bumped his e-book management application’s major version number to “6” in order to make a switch from Qt5 to Qt6 as its graphical engine.

The main hurdle for me when the upgrade from Calibre 4.x to 5.x happened was that internally, Calibre switched from Python2 to Python3. Essentially the whole of Calibre is written in Python and it uses PyQt to build the graphical interface using Qt widgets.

It took me a lot of work to re-write the calibre.SlackBuild to also make that Python switch. After all, my single calibre package is actually getting built from many sources (44 tarballs for Calibre 4, 55 tarballs for Calibre 5) and a lot of those had to be replaced to work with Python3. Moving my calibre.SlackBuild to Python3 took so much effort that I decided to apply some simplification as well: I removed the script’s ability to build its own Qt5 libraries from source, instead I let my calibre-5.x packages depend on the qt5 package which is already present in the Slackware OS since release 15.0.
Naturally I was not looking forward to doing the same cumbersome and time-consuming exercise again, now having to figure out the intricacies of Qt6, a graphical toolkit I had never built or used before.

But I was bothered by the consequence that I had to stick with Calibre 5.44 being the final release using Qt5, and so I decided to take my time, and inbetween other activities I started gradually re-writing the calibre.SlackBuild script to encompass the new sources and get these compiled and tested.

Now then finally, you can install Calibre 6 on Slackware using a 170 MB large natively-built package which needs 64 source tarballs to compile You can get the new Calibre 6 packages from my repository or any mirror (like my own US mirror). No external dependencies, works out of the box on Slackware 15.0 or -current.

To refresh your memory in case you are still thinking “what the heck is he talking about” – Calibre is an e-book library management program which I started using after buying my first E-reader (a Sony TRS-T1) and discovering that the accompanying e-book management software was proprietary and not user-friendly. The functionality and ease-of-use of Calibre were so much better than the commercial software that I never looked back.
I now read my e-books on a Kobo Aura2 H2O . Adding books to the reader is dead-simple: Calibre exports my book library on the internet as a OPDS server that works with any e-reader that has Internet connectivity, and I can scroll through my library even when on vacation and pick a new book to read. The power of Open Source.

A note for those of you who are on a 32bit Slackware and are currently using my Calibre 5.x package. With Calibre 6 it is no longer needed to set the “QTWEBENGINE_CHROMIUM_FLAGS” variable to disable seccomp. Qt6 does not appear to have the issues that the embedded Chromium browser of Qt5 had.

I also checked that the application’s screenreader still works. Right-click the current page in your open e-book text and then click “Read Aloud“. The text-to-speech is provided by an embedded speech-dispatcher program.
This is great, but the downside of it is, that the default espeak voice sounds like Steve Hawking came to visit you. If you find out how to improve the text-to-speech capability of Calibre by adding voices (especially non-english) and playing with the various engines, please let me know in the comments section below. I really would like to improve the out-of-the-box TTS experience for people with visual disabilities or dyslexia.

Have fun! Eric

Calibre 5.x available for Slackware 15.0 and -current (finally)

Finally! I have a working package for Calibre 5.38.0, targeting Slackware 15.0 and -current.

As you surely know by now, Calibre is an e-book library management program, probably the best you can get and it surpasses its commercial rivals in terms of feature set and ease of use.

Calibre is not only a library manager, it can act as a content server to make your book library accessible online (on your phone and in web browsers for instance), and it also contains a Qt5-based e-book reader application, as well as a full-fledged e-book editor. If you have online magazine or newspaper subscriptions, Calibre can download these magazines automatically for you and add them to your library.

It is also quite the complex piece of software. It is written in Python, using several modules to enable its features. Calibre creates its graphical user interface using PyQt5 widget libraries. My calibre package for Slackware embeds all these modules, so that the package does not have any external dependencies. It does expect a full Slackware installation however, because that includes Qt5, PyQt5 and related packages. You could slim down your Slackware as long as you keep Qt5 related packages installed.

It took a long time to upgrade my Calibre 4.x package to 5.x, the first release in the 5.x series was on 25 September 2020. The reason is that the developer, Kovid Goyal, switched Calibre from Python 2 to Python3 and that influenced many of the Python modules that are used by the program. I had decided to wait for a Slackware 15.0 release to start working on the calibre.SlackBuild… but then that Slackware 15.0 release got delayed, and delayed, and… I could finally free up some of my time to actually do this, last week.

So here it is: Calibre 5.38.0, get it from my repository or any mirror (like my own US mirror)!

Note that you should either install my calibre4 package, or calibre (now at 5.x) but do not install both at the same time! Their files overlap.

Another note: on 32bit Slackware 15.0 and -current, all Chromium based programs will crash with a seccomp error. This is caused by the changes in glibc with regard to secure computing (seccomp), and the affected versions of glibc can be found in Slackware 15.0 and newer. The Chromium developers have been unable to update their sourcecode to make this work on 32bit Operating Systems. As a result, for instance Falkon on 32bit Slackware 15.0 and newer will crash immediately on startup.
The workaround is to disable the seccomp filter sandbox for your 32bit OS. This is achieved without much effort, you have to make an environment variable available after login: QTWEBENGINE_CHROMIUM_FLAGS needs to be set to “--disable-seccomp-filter-sandbox“.

For bash-compatible shells you would do as follows:

# echo "export QTWEBENGINE_CHROMIUM_FLAGS='--disable-seccomp-filter-sandbox'" > /etc/profile.d/chromium_seccomp.sh
# chmod +x /etc/profile.d/chromium_seccomp.sh

And after logging in again, you should find that calibre works also on 32bit Slackware.

Addendum: even the screenreader works. Right-click the current page in your open e-book text and then click “Read Aloud“. The text-to-speech is provided by an embedded speech-dispatcher program. Unfortunately the configuration button does not work there, but if you don’t like the default espeak voice you can manually pick one of the available alternatives by editing the file “/usr/lib64/calibre/etc/speech-dispatcher/speechd.conf” (on 32bit Slackware the libdir is ‘lib‘ of course).

Have fun! Eric

Calibre 5.x – I can’t create a working package

Calibre is my favorite e-book library management program. My repository contains a package for Calibre 4.x which works well, but it is using Python2 and has been superseded since September 2020 by the new 5.x releases which are based on Python3.
I have been postponing the Slackware package migration from 4.x to 5.x until Slackware 15.0 would be released, since the move from Python 2 to 3 promised to be a significant effort in terms of changing the calibre.SlackBuild script.

Now that Slackware 15.0 is available, I decided to pick up this chore and re-write the build script to support Calibre 5.x. The re-write took several days because of all the new Python modules that are now required by Calibre 5.
I have dropped support for embedding a copy of the Qt5 libraries and compiling an embedded Python interpreter is no longer optional. It simplified the script and reduced the compile-time a lot (no more Qt5 compilation).

Now the bad part. I have been trying to compile Calibre 5.38 on Slackware 15.0 now for the past week. Well, compiling is OK, and the ebook viewer and editor work properly. But the main calibre program crashes with an error “free(): invalid pointer” as soon as it tries to create a new (or read an existing) Calibre database (metadata.db). As a consequence, the Calibre library maintenance program is completely useless.
I have not found a cause for that, and unfortunately the Calibre developer refuses to discuss any bug that is not in his own pre-built binaries.

So I am between a rock and a hard place. I have no idea how to proceed from here, and I am not willing to add a non-functional package to my repository. So I have uploaded my work here:

If anyone wants to try this out and finds the cause of the crash bug, please share your thoughts here.

Thanks, Eric

How to access your e-books online

In a previous article about the Calibre e-book library management program, I wrote that I did away with USB cable connections to transfer e-books from my Calibre library to my E-reader. Instead, I made my e-book library accessible on-line and now I am able to download my books securely over the Internet – no matter where I am, as long as I have a network connection.

In this article today, I am going to explain how I make my Calibre library accessible on-line.

Understandably, the article will be Slackware-centric, because that is where I have my Calibre library and where I run all the programs that I need. But, the technology and the configuration is generic enough that readers of this article running another distro or even employing an embedded (ARM) device will be able to make it work.

Which software is used?

Apache and PHP are standard part of Slackware Linux. I assume that you know how to setup an Apache web server. The good news is that Slackware’s default httpd configuration works out of the box. However, it does not enable a secure (https) web site. That part is on you.
If your content server is going to be used in your home network only, then HTTPS is not even needed.
I have a package for Calibre in my repository which of course you will definitely need if you want to build an e-book library.
You’ll have to download the ZIP file of the latest COPS release from https://github.com/seblucas/cops/releases – the ZIP contains the extra stuff that is not contained in the source .tar.gz file.
The remainder is just configuration.

Now let me explain my setup.

The Calibre library

It all starts with Calibre of course. It is a desktop program, available for Linux, Windows and MacOS. It stores your e-book library as a directory tree.  The directories immediately below the root of the tree are the book authors. In the root of the directory tree, Calibre creates a SQLite database file containing all the library metadata. The simple directory structure and the SQLite database file make that a Calibre library is independent of the Operating System that Calibre runs on. Its database scheme can be updated between major releases though. From within the Calibre GUI, it is however trivial to re-create this database file from scratch if it becomes corrupted.
This simple library setup has the advantage that if you store the library on a Cloud service like Dropbox, you can have access to your library from multiple computers running different Operating Systems (but preferably all the same version of Calibre). As long as one person accesses the library at any time, the Calibre database file will not get corrupted. This is how I have setup Calibre for my wife so that she can manage her e-books from every computer in the house.

Where to create the library

Myself, I use Calibre on a Linux machine which runs 24/7 (my package-build server in fact). I run a XFCE graphical desktop session inside a VNC server and that allows me to work in a desktop environment that’s always there for me. I use NoVNC to access this desktop using a web browser and that is perfect for corporate envronments which put a firewall up for everything except HTTPS traffic. Perhaps I’ll write another article sometime, explaining how to set that up.

What matters is that the Calibre library is stored on a machine with Internet access, and that this computer is running a web server like Apache. In Slackware, Apache httpd is the default webserver program.
I made this web server accessible from the Internet by configuring my ISP’s Internet router to forward ports 80 (http) and 443 (https) to the internal IP address of my Slackware machine.
For the sake of this article, let’s assume that two separate Calibre libraries exist on this computer, mine is located in /data/Calibre/EricLibrary and my wife’s books are stored in /data/Calibre/WifeLibrary . In Calibre itself, you can easily switch between libraries so that me and my wife are both able to maintain a separate collection of books. Our book tastes have no overlap…

How to configure the web server

I want both libraries to be accessible online.
I do not want to use Calibre’s own content server but instead use COPS. COPS is a collection of PHP scripts, and it does not require the Calibre program to function. It just needs access to the physical directory containing your Calibre library and it understands the Calibre SQLite database format.
If you want to get creative, you can install COPS on an Internet server – e.g. if you do not have a webserver at home. In that case, use a cloud storage service like Dropbox to keep your library in sync between your computer at home and that remote server running Apache.

My example setup is a Slackware computer at home with Apache enabled:

# chmod +x /etc/rc.d/rc.httpd
# /etc/rc.d/rc.httpd start

Apache uses the concept of a DocumentRoot, which is a directory accessible by the httpd server. Everything stored below that DirectoryRoot will be accessible via the URL of the webserver. Never store files below the DocumentRoot that you do not want to share!
I downloaded and extracted the COPS zip file directly into the Apache DocumentRoot directory (the default in Slackware is configured in the file ‘/etc/httpd/httpd.conf’ to be “/srv/httpd/htdocs/” but you can change that default location of course):

# cd /tmp
# links https://github.com/seblucas/cops/releases/download/1.1.3/cops-1.1.3.zip
# cd /srv/httpd/htdocs/
# unzip /tmp/cops-1.1.3.zip
# ln -s cops-1.1.3 mybooks
# cd mybooks
# cat <<EOT > config_local.php
$config = array();
$config['calibre_directory'] = array ("Eric" => "/data/Calibre/EricLibrary/", "Wife => "/data/Calibre/WifeLibrary/");
$config['cops_title_default'] = "Aliens Library";
$config['cops_use_url_rewriting'] = "1";
$config['cops_recentbooks_limit'] = '100';
$config['cops_update_epub-metadata'] = "1";
$config['cops_books_filter'] = array ("Books" => "!News");
EOT

The lines from the “cat << EOT” until the “EOT” on its own line is called a “here document“. You can copy/paste these lines into a terminal and it will create the file “config_local.php” with the content as shown between the “EOT” lines.
This is a configuration which allows access to two separate libraries. If you access the “recently added” section of a library it will show the latest 100 books instead of the default 20 books. It does some URL re-writing to generate cleaner URLs and it will not show any News sources (in Calibre, you can add various paid-for and free newspaper subscriptions and those will then be dpwnloaded daily, but I do not want to see those online).
Compare this configuration of mine to the ‘config_local.php.example’ file in the COPS directory. The above is not all you can configure, although it is sufficient. All configurable parameters are documented in ‘config_default.php’ which also defines their default values.

You’ll have noticed that I created a symlink to the extracted cops files, which allows me to use an easy to remember URL to access my books. Suppose my apache server is accessible as http://foo.net/ – then my COPS installation will be accessible at http://foo.net/mybooks/ .

That’s all!

Under the above ‘mybooks’ URL you will now find a COPS installation with two Calibre libraries with separate names “Eric” and “Wife”. Pick a nice template and theme, and try out the ebook-reader which is part of COPS (you  can read EPUBs in a web browser, no E-reader required).

Accessing COPS through the OPDS protocol

You want to configure your E-reader to access the library. An E-reader which contains a webbrowser is easy, just open http://foo.net/mybooks/ and click a book to download it.
Other E-readers, like FBReader for Android, are able to connect to an OPDS (Open Publication Distribution System) catalog. The COPS program exposes your library as a OPDS catalog if you just add ‘feed.php’ to the URL.
I.e. enter http://foo.net/mybooks/feed.php into the Network Library menu of FBReader and you’ll get instant access to “Alien’s Library” where you can browse Authors, Series, Categories, Recent Additions and read book descriptions, and then download the books you want to read.
If you want to always start with a list of recent library additions, you  can add ‘?page=10’ at the end of the URL so that it becomes http://foo.net/mybooks/feed.php?page=10

Hiding the cops files from the Apache DocumentRoot

Some people feel a bit anxious having these cops files accessible in their DocumentRoot – suppose someone gains access to them trough other means. You can use Apache’s “Alias” directive to install COPS in e.g. /usr/local instead. In that case, add the following lines to your /etc/httpd/httpd.conf or whatever file holds your website definition:

    <Directory "/usr/local/cops-1.1.3/">
        AllowOverride All
        Options +ExecCGI +FollowSymLinks
        Require all granted
    </Directory>
    Alias /mybooks/ /usr/local/cops-1.1.3/

You should check the validity of your httpd configuration before restarting the webserver:

# apachectl configtest
# /etc/rc.d/rc.httpd restart

And the URL to access your library will remain the same, i.e. http://foo.net/mybooks/feed.php?page=10 , only in this case your actual DocumentRoot directory will be empty.

Adding some sense of security

After following the above instructions, you now have an e-book library online whose access is not restricted in any way. If some search engine passes by it will neatly catalog all your books for other people to find. Oops!

You’ll need a basic access restriction at least. Apache offers a ‘Basic Authentication’ setup which will prompt anyone who tries to access your library for a valid account and password.
To achieve this, let’s first create a password file containing the account/password combinations for the people you want to grant access. For the sake of this example, I will grant myself and my wife access by creating account/password combinations for us in a file which we will then use in  our Apache configuration.
The first ‘htpasswd’ command below has an additional ‘-c’ parameter to create the file “/etc/httpd/passwords/htaccess.opds”:

# mkdir -p /etc/httpd/passwords/
# htpasswd -b -c /etc/httpd/passwords/htaccess.opds eric ericspassword
# htpasswd -b /etc/httpd/passwords/htaccess.opds wife wifespassword

The file will then have these contents, you see that the passwords are now MD5 encrypted:

eric:$apr1$OtKDA27W$l2ac4DAhGCG53igy6jT5A/
wife:$apr1$Zql/HEUC$wrNckoe57YPC0u2w8mL/M0

Read “man htpasswd” to find out more about this command and its parameters.

Next you need to change the Apache <Directory></Directory> block which I provided above. It will become like this:

    <Directory "/usr/local/cops-1.1.3/">
        AllowOverride All
        Options +ExecCGI +FollowSymLinks
        AuthBasicAuthoritative off
        AuthUserFile /etc/httpd/passwords/htaccess.opds
        AuthType Basic
        AuthName "OPDS Server"
        Require valid-user
        # Require all granted
    </Directory>
    Alias /mybooks/ /usr/local/cops-1.1.3/

Once you made these changes, validate the configuration and restart the webserver:

# apachectl configtest
# /etc/rc.d/rc.httpd restart

Now you will be greeted by an authentication request, next time you access your library. Only those users whose username/password combinations are stored in “/etc/httpd/passwords/htaccess.opds” will be able to get access.

Using encrypted HTTP

You should realize that these passwords will still be transmitted in cleartext if your webserver is not using HTTPS. It is possible for people to sniff the network connection and find your credentials.

I think however that instructions about enabling HTTPS for your Apache belongs in another blog post. Let me know in the comments section below if you have a need for such an article.
Also, let me know if parts of the above instructions are too cryptic and I will update the text where needed.

Good luck! Eric

« Older posts

© 2024 Alien Pastures

Theme by Anders NorenUp ↑