My thoughts on Slackware, life and everything

Tag: slackwarecloudserver (Page 2 of 2)

Slackware Cloud Server Series, Episode 2: Identity and Access Management (IAM)

Hi all!
This is the second episode in a series of articles I am writing about using Slackware as your private/personal ‘cloud server’ while we are waiting for the release of Slackware 15.0.
Below is a list of past, present and future episodes in the series. If the article has already been written you’ll be able to access it by clicking on its subject.
The first episode also contains an introduction with some more detail about what you can expect from these articles.

Identity and Access Management (IAM)

When you run a server that offers all kinds of web-based services, and you want all these services to be protected with an authentication and authorization layer (i.e. people have to login first and then you decide what kind of stuff they can access) it makes sense to let people have only one identity (one set of credentials) that can be used everywhere. Not exactly ‘Single Sign On‘ because you may have to logon to the various parts separately but you would be using that single identity everywhere.

Slackware comes with Kerberos and OpenLDAP servers which can be used  together with for instance Samba to create a Single Sign On environment on a local network. But since I am looking for something more infrastructure agnostic which works all across the Internet, I ended up with something I vaguely knew since it’s used in places in our own company to provide credential management: Keycloak.

Keycloak offers web-based Open Source Identity and Access Management (IAM). Using Keycloak, I will show you how to add authentication and authorization to the applications that I will be adding to my “Slackware Cloud Server”.
Keycloak can be used to manage your users, but if you already manage your users in an OpenLDAP server or in Active Directory, Keycloak can use these as its back-end instead of its own local SQL database.

The users of our Slackware server will authenticate with Keycloak rather than with the individual applications. Ideally, once your users have logged into Keycloak they won’t have to login again to access a different application. Likewise, Keycloak provides single-sign out, which means after a user logs out of Keycloak, that will apply to all the applications that use Keycloak.

Keycloak can also act as an Identity Broker for “social login” so that you are able to use social networks like Facebook, Google etc as Identity Providers. Our Slackware server applications that ask Keycloak to handle the user login and authorizations don’t know and don’t care about exactly how the authentication took place, as long as it’s a process the administrator trusts.


For the purposes of this article however, I will limit the use of Keycloak to  authentication through OpenID Connect (OIDC) or SAML 2.0 Identity Providers. The applications I will discuss in the upcoming articles (NextCloud, Jitsi Meet, Quay, Docker Registry) all support the OIDC protocol for off-loading authentication / authorization so we’ve got that covered.

Keycloak does not only manage the identities that allow your users to login; Keycloak can also manage authorizations.
You may want to allow specific users a different level of access to your applications, or prevent access to some of them entirely. You can assign roles to users or add them to groups and then configure the access to your applications using the available roles or group memberships.

Preamble

For the sake of this instruction and for future articles in this series, “https://sso.darkstar.lan/auth” is the base URL that eventually the Keycloak application will be available at at.

Configuring the DNS for your own domain (will probably be something else than “darkstar.lan”…) with new hostnames and then setting up web servers for the hosts in that domain is an exercise left to the reader.
Before continuing, please ensure that your real-life equivalent for the following host has a web server running:

  • sso.darkstar.lan

It doesn’t yet  have to serve any content yet but its URL needs to be accessible to all applications and users that you want to give access to your Slackware Cloud Server. We will add some blocks of configuration to the VirtualHost definition during the steps outlined in the remainder of this article.

Using a  Let’s Encrypt SSL certificate to provide encrypted connections (HTTPS) to your webserver is documented in an earlier blog article.

Setting up Keycloak

NOTE!

Starting with Keycloak 20 (released in November 2022), the WildFly based distribution is no longer supported. For the newer Quarkus distribution of Keycloak, check out the new documentation .
This article targets a pre-20 release Docker container. I will have to update the text to make it reflect the new Docker options.

Working with a Docker based server infrastructure is something I covered in the previous article of this series. I assume you have Docker up and running and are at least somewhat comfortable creating containers and managing their life-cycle.

Keycloak can be installed on a bare metal server but I have opted to go for their Docker-based install instead. The configuration will be stored in a MariaDB backend and an Apache reverse proxy will be the frontend which will handle the incoming connection requests and also enforces data encryption using a Let’s Encrypt SSL certificate.
The Keycloak github contains an example of a docker-compose solution with MariaDB and Keycloak in two separate containers (see https://github.com/keycloak/keycloak-containers/tree/master/docker-compose-examples) however the documentation states that this does not work currently because MariaDB does not start fast enough for Keycloak.
Also I think that having your database inside the container, or even mounting the host’s “/var/lib/mysql” directory into the container, will make backups difficult. So, our Keycloak application will connect to Slackware’s own included MariaDB database via the host’s IP address. There’s a paragraph further down where I share my considerations about this setup.

Here’s what we are going to do now: create the MariaDB database, create a private network for the Keycloak container so that it at least has some isolation from other containers, ensure that we have some kind of DNS service for the custom network (this example uses dnsmasq for that) and finally: start the Keycloak service as a single container, no Compose needed.

The MariaDB database

MariaDB in Slackware 15.0 is at version 10.5.13. Considering future upgrades: any security update in a stable Slackware release should not introduce breaking changes, but it’s always good to have read the MariaDB pages on database upgrades to avoid surprises. And always make backups that you have tested (are the backups useable).

Login to MariaDB as the administrator (commonly the user ‘root’):

$ mysql -uroot -p

Create the database:

> CREATE DATABASE IF NOT EXISTS keycloakdb CHARACTER SET utf8 COLLATE utf8_unicode_ci;

Create a database user for Keycloak:

> CREATE USER 'keycloakadm'@localhost IDENTIFIED BY 'your_secret_passwd';

Grant all privileges:

> GRANT ALL PRIVILEGES ON keycloakdb.* TO 'keycloakadm'@localhost;

Reload the grant tables (flush privileges) and quit the management interface:

> FLUSH PRIVILEGES;
> quit;

Network defaults in Docker

Docker claims a network range when it starts, in order to connect its containers to each other and to the real world. By default, the range is 172.17.0.0/255.255.0.0 and there is no reason to change that unless it collides with pre-existing network segments in your LAN. In that case, you would want to edit “/etc/docker/daemon.json” and add something like this to define a custom Bridge IP address and container IP range:

{
  "bip": "172.100.0.1/16",
  "fixed-cidr": "172.100.0.0/24",
}

If you would want your containers to be in yet another IP range, you could add something like:

{
  "default-address-pools": [
    {
      "base": "192.168.56.0/21",
      "size": 28
    }
  ]
}

Docker needs a restart to pick up the changes. Newly created networks will then be dynamically assigned a /28 subnet of the larger IP range “192.168.56.0/21” unless you manually specify other ranges (see below).

Private network for Keycloak

We create a Docker network “keycloak0.lan” just for Keycloak, and make it use a different IP range than Docker’s default. Otherwise it would share its network with the rest of the containers and we want to create some level of real containment, as well as the ability to assign the Keycloak container a fixed IP address.
We need a fixed IP address in order to assign a hostname on the host so that Sendmail allows emails to be sent from within the container. Otherwise we would get ‘Relaying denied: ip name lookup failed” error from Sendmail.
Summarizing: the “keycloak0.lan” network will have a range of 172.19.0.0/16; and the fixed IP address for the Keycloak service will be 172.19.0.2 (since 172.19.0.1 will be the IP address of the Docker bridge).

$ docker network create --driver=bridge --subnet=172.19.0.0/16 --ip-range=172.19.0.0/25 --gateway=172.19.0.1 keycloak0.lan

Add host and network to /etc/hosts and /etc/networks:

$ grep keycloak /etc/hosts
172.19.0.2 keycloak keycloak.keycloak0.lan
$ grep keycloak /etc/networks
keycloak0.lan 172.19

My assumption is that your host does not act as the LAN’s DNS server. We will use dnsmasq to serve our new entries from /etc/hosts and /etc/networksdnsmasq will be our local nameserver. For this we use the default, unchanged “/etc/dnsmasq.conf” configuration file.

You will also have to add this single line at the top of “/etc/resolv.conf” first, so that all DNS queries will go to our local dnsmasq:

nameserver 127.0.0.1

If you have not yet done so, (as root) make “/etc/rc.d/rc.dnsmasq” executable and start dnsmasq manually (Slackware will take care of starting it on every subsequent reboot):

# chmod +x /etc/rc.d/rc.dnsmasq
# /etc/rc.d/rc.dnsmasq start

Make Sendmail aware that the Keycloak container is a known local host by adding a line to “/etc/mail/local-host-names” and restarting the sendmail daemon:

$ grep keycloak /etc/mail/local-host-names
keycloak.keycloak0.lan

If you use Postfix instead of Sendmail, perhaps this is not even an issue, but since I do not use Postfix I cannot tell you. Leave your comments below if I should update this part of the article.

MariaDB access from the network

Since our Keycloak container will connect to the MariaDB server over the network, we need to grant the Keycloak database account access to the database when it logs in from the network instead of via the localhost.
Login to ‘mysql’ as the admin (root) user and execute these commands that come on top of the ones that we already executed earlier:

> CREATE USER 'keycloakadm'@'%' IDENTIFIED BY 'your_secret_passwd';
> GRANT ALL PRIVILEGES ON keycloakdb.* TO 'keycloakadm'@'%';
> FLUSH PRIVILEGES;
> quit;

Keycloak container

We use a Docker container to run this IAM service. Remember that containers do not persist their data. In our case it means that the complete configuration which is needed by the Keycloak application inside that container in order to start up properly, has to be passed as command-line parameters to the ‘docker run‘ command.

This is how we start the Keycloak container, connecting it to the newly created network, assigning a static IP, using the ‘keycloak.lan‘ network gateway as the IP address for the MariaDB and passing it the required database properties.
By default, Keycloak listens at port “8080” but that is a portnumber which is (ab)used by many applications including proxy servers. Instead of accepting the default “8080” value we do a port-mapping and make Keycloak available at port “8400” instead by using the “-p” argument to ‘docker run‘.
We also pass the front-end URL which is going to be used by every application that wants to interact with the service (https://sso.darkstar.lan/auth). This URL is actually served by a Apache httpd reverse-proxy which we will put between the exposed port of the Keycloak container and the rest of the world:

$ mkdir -p /usr/share/docker/data/keycloak
$ cd /usr/share/docker/data/keycloak
$ echo 'keycloakadm' > keycloak.dbuser
$ echo 'your_secret_passwd' > keycloak.dbpassword
$ docker run -d --restart always -p 8400:8080 --name keycloak \
    --net keycloak0.lan \
    --network-alias keycloak.keycloak0.lan \
    --ip 172.19.0.2 \
    -e DB_VENDOR=mariadb \
    -e DB_ADDR=172.19.0.1 \
    -e DB_DATABASE=keycloakdb \
    -e DB_USER_FILE=$(pwd)/keycloak.dbuser \
    -e DB_PASSWORD_FILE=$(pwd)/keycloak.dbpassword \
    -e KEYCLOAK_FRONTEND_URL=https://sso.darkstar.lan/auth \
    -e PROXY_ADDRESS_FORWARDING=true \
    jboss/keycloak \
    -Dkeycloak.profile.feature.docker=enabled

That last argument ‘-Dkeycloak.profile.feature.docker=enabled‘ enables the “docker-v2” protocol in Keycloak for creating a Client in the realm. A Keycloak ‘Client ID‘ is what we need to connect an application to Keycloak as the login provider. This is a topic we will visit in future Episodes.
This ‘docker-v2‘ protocol is not fully compatible with the default OIDC protocol ‘openid-connect‘ and while not enabled in Keycloak by default, it is an officially supported Client Protocol.
We need ‘docker-v2‘ when we want the Docker Registry to be able to use Keycloak for authentication / authorization (the topic of Episode 6 in our Series).

A note about the Keycloak service URL (https://sso.darkstar.lan/auth). In this example I use a hostname “sso.darkstar.lan” but you will have to pick a name which fits your own network. This hostname will be used a lot and visible everywhere, so if you are paranoid and want to avoid people realizing that your “sso.” host is the key to all user-credentials, you might want to chose a more inconspicuous name like “pasture.darkstar.lan“. YMMV.
Also, the path component “/auth” in the URL is fixed and immutable, unless you want to hurt yourself. I tried to change that path component to something that made more sense to me but that “/auth” is so ubiquitous as hard-coded strings throughout the code that I quickly gave up.

Remember that our Keycloak container listens at the non-default TCP port 8400 on the docker0 interface, which we will use for the reverse proxy setup later on.

MariaDB considerations during Keycloak setup

At first, I tried with mounting the host’s MariaDB server socket in the Keycloak container and accessing the database through it via the ‘localhost’ target like below:

-e DB_ADDR=localhost -v /var/run/mysql/mysql.sock:/var/run/mysql/mysql.sock

…but JBoss would not be able to connect for whatever reason, and the resulting error was ‘connection refused‘.
I have tried various locations for the UNIX socket inside the container:

/var/run/mysql/mysql.sock
/var/run/mysqld/mysql.sock
/var/run/mysqld/mysqld.sock
/tmp/mysql.sock

… but none of those worked either.
Eventually I had to make MariaDB listen on all interfaces and rely on my firewall to block external access, for instance using:

# iptables -A INPUT -s 172.19.0.2 -p tcp --destination-port 3306 -j ACCEPT

This is the relevant MariaDB configuration line:

$ grep bind-address /etc/my.cnf.d/server.cnf
bind-address=0.0.0.0

If you have a better solution let me know! Of course, the easy way out of this dilemma is to deploy Keycloak using Docker Compose and give it its own internal database, but we need to be 100% sure that the SQL database server is up & running before we start the Keycloak container as part of the Docker infrastructure in “/etc/rc.d/rc.local“.
Also, my goal was to have a single SQL server on my host that would be the backend database for every application that needs one. A lot easier to backup and restore.

The admin user

Create the admin user once the container is running or else you won’t be able to connect at all (use ‘docker ps’ to find the containerID), and restart the container after that, but WAIT AFTER THAT RESTART LONG ENOUGH FOR THE INITIAL CONFIGURATION TO FINISH… or you’ll end up with Keycloak trying to initialize MariaDB twice, resulting in errors and failure.

This is how you run the command to create the admin user on the host, to be executed inside the running container:

$ containerID=$(docker ps -qaf "name=^keycloak$")
$ docker exec <containerID> /opt/jboss/keycloak/bin/add-user-keycloak.sh -u admin -p your_secret_admin_pwd
$ docker restart <containerID>

Write that password down in a safe place and remove  that command-line from your Bash history! It is the key to the Realm. Literally.

Reverse proxy setup for keycloak

In your Apache configuration for the “sso.darkstar.lan” host you need to add these lines to create a reverse proxy that connects the client users of your Keycloak service to the service endpoint:

SSLProxyEngine On
SSLProxyCheckPeerCN on
SSLProxyCheckPeerExpire on
RequestHeader set X-Forwarded-Proto: "https"
RequestHeader set X-Forwarded-Port: "443"
<LocationMatch /auth>
    AllowOverride None
    Require all granted
    Order allow,deny
    Allow from all
</LocationMatch>
ProxyPreserveHost On
ProxyRequests Off
ProxyVia on
ProxyAddHeaders On
ProxyPass        /auth http://127.0.0.1:8400/auth
ProxyPassReverse /auth http://127.0.0.1:8400/auth

Restart Apache httpd after making this change.

Initial configuration of Keycloak

Now that Keycloak is up and running behind a reverse proxy and interfaces with the world via the URL https://sso.darkstar.lan/auth , and has an admin account, we are going to create a “realm“. A realm is a space where the administrator manages identifiable objects, like users, applications, roles, and groups. A user belongs to only one realm and will always login to that realm.

Our realm will be named “foundation“. Here we go:

  • Point your browser at https://sso.darkstar.lan/auth/
  • Add a new Realm:
    • Click to open Admin Console (https://sso.darkstar.lan/auth/admin).
    • Logon with above configured ‘admin’ credentials.
    • Hover the mouse over the ‘Master‘ dropdown in the top-left corner.
    • Click ‘Add realm‘, enter Name: “foundation“.
    • Click ‘Create‘, add a Display name “My Cloud Foundation” (pick any name you like), click ‘Save‘.
  • (Optionally) configure email via SMTP:
    • Click ‘Email‘ (top of the page).
    • Enter your LAN’s SMTP server, the SMTP port (587 if you enable StartTLS at the bottom), and a reasonable ‘From‘ email address.
    • (Optionally) configure other parameters specific to your setup.
    • Click ‘Test Connection‘.
    • If your Keycloak container could successfully connect to the SMTP server, click ‘Save‘ and now Keycloak will be able to send you relevant emails, provided that you configured a working email address for the admin user of course.
  • Add a first user:
    • Click ‘Users‘ (left-hand menu) > ‘Add user’ (top-right corner of table)
    • Add user “alien” with full-name “Alien BOB” (of course, use whatever makes sense for you).
    • Set initial password to be able to login:
      • Click ‘Credentials‘ (top of the page).
      • Fill in ‘Set Password‘ form with a password like “WelcomeBOB!”.
      • (Optionally) click ‘ON‘ next to ‘Temporary‘ so that it toggles to ‘OFF‘. This removes the requirement for the user to update password on first login.
      • Click ‘Set password‘.
  • Test the new user login in a different browser (or logoff the admin user first):
    • Open User Console: https://sso.darkstar.lan/auth/realms/foundation/account
    • Login as ‘alien‘ and you’ll see the message: “You need to change your password to activate your account.
    • Set a new (permanent) password and explore the configurable user profile settings, for instance enabling 2-Factor Authentication (2FA) using an app like Authy or Google Authenticator.

Considerations

Your keycloak is now running at https://sso.darkstar.lan/auth and the user login-page for the “Foundation” realm is at https://sso.darkstar.lan/auth/realms/foundation/account . That URL is nowhere to be found easily, so if you don’t know it is there (or if you don’t know about Keycloak’s URL path template) then users may complain about not finding it.
On the one hand, this adds some “security through obscurity” if you’re the paranoid type.
Plus, Keycloak will redirect users to the correct login page anyway when an application asks for a credential check.
On the other hand, you want to give your server’s users a nice experience.
As a courtesy to your users, you could consider adding a HTML redirect to the root of that server. Assuming that https://sso.darkstar.lan/ just serves an empty page or spits out a “page not found” error, you could paste the following HTML snippet into an ‘index.html‘ file in the DocumentRoot directory, which will cause an immediate redirect from the URL root to the user login page for the “foundation” realm:

 <html>
  <head>
    <title>Slackware Cloud Server Logon</title>
    <meta http-equiv="refresh" content="0; URL="https://sso.darkstar.lan/auth/realms/foundation/account">
  </head>
  <body bgcolor="#ffffff"></body>
</html>

Keycloak discovery

In the previous section I mentioned that it is not trivial for users to discover the URL of their Keycloak profile page. But how do applications do this when they want to interact with Keycloak?
OpenID Connect Discovery is a layer on top of OAuth 2.0 protocol which allows a client application not only to authenticate a user but also to obtain certain user characteristics (called “claims” in OAuth2.0) like full name or email address. To obtain these claims, the client application connects to a well-known URL where it can find out which claims are supported.
Keycloak supports this OpenID Connect Discovery and provides a Discovery URL at “/auth/realms/<realm>/.well-known/openid-configuration”. For our Keycloak server, that will be:

https://sso.darkstar.lan/auth/realms/foundation/.well-known/openid-configuration

You can query this URL for instance via the ‘curl’ program and if you also have the ‘jq‘ program installed from my repository you can generate nicely formatted human-readable colorized output:

 $ curl https://sso.darkstar.lan/auth/realms/foundation/.well-known/openid-configuration | jq

The output of that command is lengthy but it starts with:

 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current 
                                Dload  Upload   Total   Spent    Left  Speed 
100  5749  100  5749    0     0  41485      0 --:--:-- --:--:-- --:--:-- 41659 
{ 
 "issuer": "https://sso.darkstar.lan/auth/realms/foundation", 
 "authorization_endpoint": "https://sso.darkstar.lan/auth/realms/foundation/protocol/openid-connect/auth", 
 "token_endpoint": "https://sso.darkstar.lan/auth/realms/foundation/protocol/openid-connect/token", 
 "introspection_endpoint": "https://sso.darkstar.lan/auth/realms/foundation/protocol/openid-connect/token/introspect", 
 "userinfo_endpoint": "https://sso.darkstar.lan/auth/realms/foundation/protocol/openid-connect/userinfo", 
 "end_session_endpoint": "https://sso.darkstar.lan/auth/realms/foundation/protocol/openid-connect/logout", 
 "frontchannel_logout_session_supported": true, foundation
 "frontchannel_logout_supported": true, 
 "jwks_uri": "https://sso.darkstar.lan/auth/realms/foundation/protocol/openid-connect/certs", 
 "check_session_iframe": "https://sso.darkstar.lan/auth/realms/foundation/protocol/openid-connect/login-status-iframe.html"
, 
 "grant_types_supported": [
  ........

We will be able to use this Discovery protocol when setting up an Etherpad container in Episode 6.

Done

We are now ready to use Keycloak as IAM service for other applications that are waiting for us to install and configure. This setup is secure by default, but I do invite you to read more about the advanced configuration of Keycloak, they have extensive documentation at https://www.keycloak.org/documentation . For instance, do you want new users to be able to register themselves? Allow them to configure 2-Factor Authentication? Update their profile? Force password expiry? Custom password complexity configuration?

You may want to add groups to the realm and not just users, and limit the use of certain applications to specific groups (you may not want to let everyone use the Jitsi video conferencing platform for instance).

If your network already manages its users in an Identity provider like LDAP or Kerberos, Active Directory or eDirectory, or if you want to allow people to use their Google, Facebook etc identities to authenticate against your Keycloak, you should look into the functionality behind ‘Identity Providers’ in the left sidebar:

In that case, Keycloak will not store user identity information in its MariaDB database but instead use these Identity Providers as the remote backend.

Thanks

I hope you made it this far… and you liked it. Leave your comments, encouragements, fixes and general feedback in the section below.

Cheers, Eric


Attribution

Keycloak architecture images have been taken from the keycloak.org/documentation web site. The remainder are screenshots taken from my own Keycloak instance.

Slackware Cloud Server Series, Episode 1: Managing your Docker infrastructure

Hi all!
This is the first installment of a series of articles I intend to write in early 2022. With Slackware 15.0 around the corner, I think it is a good time to show people that Slackware is as strong as ever a server platform. The core of this series is not about setting up a mail, print or web server – those are pretty well-documented already. I’m going to show what is possible with Slackware as your personal cloud platform.

A lot of the work that went into developing Slackware between the 14.2 and 15.0 releases was focusing on desktop usage. The distro is equipped with the latest and greatest KDE and XFCE desktops, a low-latency preemptive kernel, Pipewire multimedia framework supporting capture and playback of audio and video with minimal latency, et cetera. I have been enjoying Slackware-current as a desktop / laptop powerhouse for many years and built a Digital Audio Workstation and a persistent encrypted Live Distro with SecureBoot support out of it.

Slackware Cloud Server Series – Summary

The imminent release of Slackware 15.0 gives me fresh energy to look into ‘uncharted’ territory. And therefore I am going to write about setting up collaborative web-based services. Whether you will do this for the members of your family, your friends, your company or any group of people that you interact with a lot, this year you will hopefully learn how to become less dependent on the big ‘cloud’ players like Google, Microsoft, Zoom, Dropbox. What I will show you, is how to setup your own collaboration platform on a server or servers that you own and control. Think of file-sharing, video-conferencing, collaborative document editing, syncing files from your desktop and your phone to your cloud server. If the series is received well, I may additionally write about expanding your cloud server to a private platform for watching movies, listening to network audio, and so on.

I have a good notion of what I will write about, but I am sure that as this series grows and I tell you stories, you will get triggered about subjects that I may not have considered yet. So here’s a challenge for you, the reader: let me know what you think would be a good addition to the series. What makes Slackware great as a “personal cloud” platform to bind people together.

Topics you can expect in this series:

  • Episode 1 (this article): Managing your Docker Infrastructure
    Here I will show you how to setup and manage Docker containers on Slackware, considering graphical management and firewalling. This is the foundation for what we will be doing in subsequent episodes.

    • Docker architecture
      • Examples
    • Docker Compose
    • Building a Docker image and running a container based on it
    • Help on commands
    • Obtain information about images and containers
    • Stop and remove a container
    • Refresh a container with the latest image from the vendor
    • Managing  containers, images, volumes, networks etc
    • Difference between load/save and import/export
    • Limiting the log size for your containers
    • Graphical management tool
    • Docker’s effect on iptables firewall
      • Generic iptables
      • UFW (Uncomplicated Firewall)
      • Firewalld
      • Alien’s Easy Firewall Generator (EFG)
    • Thanks
    • Attribution
  • Episode 2: Identity and Access management (IAM)
    Setting up Keycloak for Identity and Access Management (IAM) to provide people with a single user account for all the Slackware Server Services we will be creating.
  • Episode 3: Video Conferencing
    Setting up Jitsi Meet . This makes us independent of cloud conferencing services like MS Teams, Zoom or Google Meet. The Jitsi login is again offloaded to our Keycloak IAM provider.
  • Episode 4: Productivity Platform
    Setting up NextCloud as the productivity hub where you can store and read your documents, host your photo library, hold video meetings, manage your activities, read and send emails, keep track of where your smartphone went, chat with people and a lot more. I will also show how to integrate the Jitsi video conferencing solution which was the topic of the previous episode into our NextCloud platform.
    NextCloud will be setup on your ‘bare metal’ server. User authentication to the platform will be handled by our Keycloak IAM server.
  • Episode 5: Collaborative document editing
    Integrating Collabora Online Development Edition (CODE) with NextCloud and move from reading/displaying your documents to collaborative real-time document editing with friends, family or colleagues in your self-hosted LibreOffice Online server.
  • Episode 6: Etherpad with Whiteboard
    Setting up an improved Etherpad real-time collaborative editor with more power than the out-of-the-box version which is used with Jitsi Meet, at the same time integrating it with Keycloak IAM for authentication.
  • Episode 7: Decentralized Social Media
    Setting up Mastodon as an open source alternative to the Twitter social media platform.
  • Episode 8: Media streaming platform
    Setting up the Jellyfin media platform. If you have a decent collection of digital or digitized media (audio, video, photos) your cloudserver will become your friends’ autonomous alternative to Netflix and the likes. Single Sign On is provided to your users via Keycloak IAM.
  • Episode X: Docker Registry
    Setting up a Docker Registry. A local index and registry of Docker images allows us to become independent of Docker’s commercially driven limitations on images you create and want to share with the world or use privately. I will for instance use the public Docker Hub for all the services that are discussed in this series of articles, and I will point out the limitations of doing so.
    I am not decided on the actual implementation of the Registry. The Docker Hub is an implementation of the aforementioned Registry product and it is Open Source but it does not do authentication and it has no user interface. Docker (the company) adds a lot to this service that is not Open Source to make the Hub the popular platform that it became.
    So perhaps I will combine this Registry with Portus, which is an Open Source authorization service and user interface for the Docker Registry. It is created/maintained by the OpenSuse team. It can use Keycloak as the OIDC (OpenID Connect) provider, and so our Keycloak IAM server will do the Single Sign On.
    Or I will use Quay, which is Red Hat’s solution for self-hosted Docker Registry including a web-based user interface that looks a lot like the Docker Hub. Quay can also offload the authentication/authorization to an OIDC provider like Keycloak. In addition, it allows scanning your hosted images for known security vulnerabilities using Clair, another Red Hat product and part of Project Quay.

Managing your Docker Infrastructure

In a previous article, I have shared my Docker related packages with you for Slackware 14.2 and 15.0 (well, -current formally since we are still waiting for 15.0). I assume you are running the newest Slackware almost-at-version 15.0 and have already installed those four Docker packages (containerd, runc, docker and docker-compose), you have added your user account to the ‘docker‘ group and have at least once rebooted your computer or manually ran “/etc/rc.d/rc.docker start“.
I also assume that you have a basic understanding of containers and how they differ from Virtual Machines. A brief summary:

Containers run directly on the host kernel and the ‘bare metal’. The Docker engine hides the host operating system particulars from the applications that run in the containerized environment. Docker uses cgroups and Linux kernel capabilities to shield container processes from other containers and the host, but using a “ps” command on the host you can easily see all the container processes.
A Virtual Machine on the other hand, is a virtualized machine hardware environment created by a hypervisor program like QEMU, VMWare, VirtualBox etc. Inside that virtual machine you can run a full Operating System which thinks it is running on real hardware. It will be unaware that it is in fact running inside a VM on a host computer which  probably runs a completely different OS.
From the host user perspective, nothing that goes on in the VM is actually visible; the user merely sees the hypervisor running.
And from a Docker container as well as VM user perspective, anything that happens outside that environment is invisible. The host OS can not be reached directly from within the guest.

What I am not going to discuss in the scope of this article is orchestration tools like Kubernetes (K8s) or Docker Swarm, that make it possible or at least a lot more convenient to manage a cluster of servers running Docker containers with multiple micro-services that are able to scale on-demand.

Docker architecture

You create and run a Docker container from (layers of) pre-existing images that inherit from each other, using the ‘docker run‘ command. You can save a container that you created, ‘flattening’ the layers the container consists of into one new image using ‘docker commit‘.

You start and run a container by referring the Docker image it should be based on, and if that image is not available locally then Docker will download it from a Docker Registry (by default hub.docker.com but it can also be a private Registry you run and control yourself). You can also manually download an image using the ‘docker pull‘ command in advance of the ‘docker run‘ command which is going to use that image so that you don’t have to wait for an image download at the moment suprême of starting your container.

Building an image yourself using the ‘docker build‘ command is essentially ruled by a Dockerfile and a context (files on the local filesystem which are used in the creation of the image). The container which is based on that image typically runs an application or performs a task.

Examples

An example: running an application which is not provided by your Slackware OS. A container can run a Postgres database server which you then use on your host computer or in other containers. This Postgres container will typically be started on boot and keeps running until shutdown of the host computer. It’s as trivial as running:

$ docker run -d --restart unless-stopped --name slack-postgres -e POSTGRES_PASSWORD=**** postgres

Another example: building Slackware packages reliably and without introducing unwanted dependencies. In this case, you use a container based from a Slackware Docker image, to download and compile a Slackware package, and then export the resulting package from within the container to the host computer’s local filesystem. Such a Docker container based on a Slackware image will be able to offer the same OS with the same configuration, every time it starts and functions independently of the host. You could be compiling Slackware packages on a Debian host for instance.
Exercise left for the reader: you will have to create that Slackware Docker image yourself… it is not available online. 

Docker Compose

Also created by the Docker company, docker-compose is a separate application which uses the capabilities of Docker. The 2.x release of Docker Compose which I offer in my repository is written in the Go language which makes it fully self-contained. The previous 1.x releases that you can obtain from SlackBuilds.org were heavily dependent on Python and for that version of docker-compose you had to install 13 dependent packages.

Docker Compose expands on the Docker concept of a single container with a single purpose. Compose is a tool for defining and running multi-container Docker applications. The individual Docker applications themselves are defined by a Dockerfile, which makes their deployment reproducible in any Docker environment.
You write a Compose file (typically called docker-compose.yml) which defines the container services that you want to run. This Compose file is written in YAML and according to the Compose file specification . The file describes an application service which is built out of multiple Docker containers  as well as their shared resources and private communication channels. Docker-compose reads the yaml file and creates data volumes and internal networks, starts the defined containers and connects them to the just-created volumes and networks. There’s a lot more you can do in a Compose file, check out the reference.

The result of the ‘docker-compose up -d‘ command is the start-up of a complex service running multiple inter-dependent applications (you’ll see those referred to as micro-services) that usually have a single point of interaction with its users (an IP address/port or a https URL). Essentially a black-box where all the internal workings are hidden inside that collection of containers. As an administrator there’s little to nothing to configure and as an user you don’t have to know anything about that complexity.
And again you could be trivially running applications in Docker containers on your Slackware host that you would not be able to run on the bare host OS without a lot of weeping and gnashing of teeth .

Docker Compose offers a lot of versatility that regular Docker does not. Whether you use one or the other depends on the complexity of that what you want to achieve. In future episodes of this article series, I will give examples of both.

Building a Docker image and running a container based on it

The ‘docker build’  command has a lot of parameters but it accepts a single argument which is either a directory containing a Dockerfile and the required context (local files to be used in the image creation process), or a URL the content of which it will download and interpret as a Dockerfile. You can also use the “-f” switch to specify a filename if it’s not called “Dockerfile” literally, or pipe a Dockerfile content into the ‘docker build‘ command via STDIN. It’s good practice to ‘tag’ the resulting image so that you’ll understand its purpose later on. You can then upload or ‘push’ the image to a Docker Registry, either the public one or a private registry that you or your company control.
Let me give a comprehensive example. We will build a new Slackware image which is based on my Slackware ‘base‘ image on the public Docker Hub (docker pull liveslak/slackware:latest). The Dockerfile will add a couple of packages, create a user account called “alien” and start a login shell if needed.
Note that in this example I will write the Dockerfile on the fly using ‘cat‘ and a ‘here-document‘ to pipe it into the ‘docker build‘ command. I will tag the resulting image as ‘slacktest‘ using the “-t” switch of the ‘docker build‘ command:

$ cat <<'EOT' | docker build -t slacktest -
FROM liveslak/slackware:latest 
MAINTAINER Eric Hameleers <alien@slackware.com> 
ARG SL_UID="1000" 
ARG SL_GID="users" 
ARG SL_USER="alien" 
# Install compiler toolchain and supporting tools. 
RUN rm -f /etc/slackpkg/templates/compilertoolchain.template
RUN for PKG in \ 
ca-certificates \ 
curl \ 
cyrus-sasl \ 
gc \ 
gcc \ 
git \ 
glibc \ 
glibc-profile \ 
glibc-zoneinfo \ 
guile \ 
intltool \ 
kernel-headers \ 
libmpc \ 
libffi \ 
libtasn1 \ 
make \ 
mpfr \ 
nettle \ 
p11-kit \ 
perl \ 
; do echo $PKG >> /etc/slackpkg/templates/compilertoolchain.template ; done 
RUN slackpkg -batch=on -default_answer=y update gpg 
RUN slackpkg -batch=on -default_answer=y update 
RUN slackpkg -batch=on -default_answer=y install-template compilertoolchain 
# Refresh SSL certificates: 
RUN /usr/sbin/update-ca-certificates -f 
# Create the user to switch to: 
RUN useradd -m -u "${SL_UID}" -g "${SL_GID}" -G wheel "${SL_USER}" && \ 
sed -ri 's/^# (%wheel.*NOPASSWD.*)$/\1/' /etc/sudoers 
USER "${SL_USER}" 
ENV HOME /home/"${SL_USER}" 
WORKDIR /home/"${SL_USER}" 
# Start a bash shell if the container user does not provide a command: 
CMD bash -l
EOT

The output of this ‘docker build’ command shows that the process starts with downloading (pulling) the slackware base image:

Sending build context to Docker daemon  3.072kB 
Step 1/16 : FROM liveslak/slackware:base_x64_14.2 
base_x64_14.2: Pulling from liveslak/slackware 
Digest: sha256:352219d8d91416519e2425a13938f94600b50cc9334fc45d56caa62f7a193748 
Status: Downloaded newer image for liveslak/slackware:base_x64_14.2 
---> 3a9e2b677e58 
Step 2/16 : MAINTAINER Eric Hameleers <alien@slackware.com> 
---> Running in d7e0e17f68e6 
Removing intermediate container d7e0e17f68e6 
---> 75b0ae363daf
...(lot of output snipped) ...
Successfully built e296afb023af
Successfully tagged slacktest:latest
$ docker images |grep slacktest
slacktest                latest          e296afb023af   2 minutes ago   387MB

Here you see how a Dockerfile uses ‘FROM’ to download a pre-existing image from the Hub and then RUNs several commands which it applies to the base image which was downloaded from Docker Hub, creating a new image and then naming it “slacktest” with a tag “latest” and an ID of “e296afb023af“. In that process, several intermediate layers are created, basically every command in the Dockerfile which you would normally execute on the command-line will create one. The end result is an image which consists of multiple layers. You can use the ‘history’ command so see how these were built:

$ docker history e296afb023af 
IMAGE          CREATED          CREATED BY                                      SIZE      COMMENT 
e296afb023af   49 minutes ago   /bin/sh -c #(nop)  CMD ["/bin/sh" "-c" "bash…   0B 
e1224d15abac   49 minutes ago   /bin/sh -c #(nop) WORKDIR /home/alien           0B 
4ce31fa86cca   49 minutes ago   /bin/sh -c #(nop)  ENV HOME=/home/alien         0B 
365bffd67f07   49 minutes ago   /bin/sh -c #(nop)  USER alien                   0B 
05d08fe94b47   49 minutes ago   |3 SL_GID=users SL_UID=1000 SL_USER=alien /bi…  336kB 
3a300b93311f   49 minutes ago   |3 SL_GID=users SL_UID=1000 SL_USER=alien /bi…  218kB 
7a14f55146dd   50 minutes ago   |3 SL_GID=users SL_UID=1000 SL_USER=alien /bi…  233MB 
a556359ec6d3   51 minutes ago   |3 SL_GID=users SL_UID=1000 SL_USER=alien /bi…  6.21MB 
da19c35bed72   52 minutes ago   |3 SL_GID=users SL_UID=1000 SL_USER=alien /bi…  2.13kB 
604fece72366   52 minutes ago   |3 SL_GID=users SL_UID=1000 SL_USER=alien /bi…  161B 
1c72e034ef06   52 minutes ago   |3 SL_GID=users SL_UID=1000 SL_USER=alien /bi…  0B 
3f020d0c2024   52 minutes ago   /bin/sh -c #(nop)  ARG SL_USER=alien            0B 
91a09ad8693a   52 minutes ago   /bin/sh -c #(nop)  ARG SL_GID=users             0B 
96d433e0162d   52 minutes ago   /bin/sh -c #(nop)  ARG SL_UID=1000              0B 
75b0ae363daf   52 minutes ago   /bin/sh -c #(nop)  MAINTAINER Eric Hameleers…   0B 
3a9e2b677e58   11 days ago                                                      148MB     Imported from -

That last line shows the image “3a9e2b677e58” which is the slackware ‘base‘ image which I downloaded from Docker Hub already 11 days ago.

If we now use ‘docker run’  to create a container based on this image we would expect it to give us a bash prompt as user ‘alien’. Let’s try it and run an interactive container connecting it to a pseudo terminal using the “-ti” switch:

$ docker run -ti slacktest 
alien@88081b465a98:~$ id 
uid=1000(alien) gid=100(users) groups=100(users),10(wheel) 
alien@88081b465a98:~$ exit
logout
$ docker ps -a | grep slacktest
88081b465a98   slacktest                   "/bin/sh -c 'bash -l'"   2 minutes ago   Exited (0) 16 seconds ago
$ docker rm 88081b465a98 
88081b465a98
$

Indeed we ended up as user “alien” in a machine whose hostname is the ID of the container (88081b465a98). And with the final command, we could remove the container without first stopping it, because it was already stopped when we issued the “exit” command.

Help on commands

Working with Docker containers, it is good practice that you get familiar with the command-line client (aptly name ‘docker’). You’ll find a lot of command-line examples in this article which should really become part of your hands-on toolkit.
The ‘docker’ tool has some 30 sub-commands and all of them have built-in help. To get the full list of available  docker commands run:

$ docker --help

The help per command can be found by suffixing it with “–help”. Many commands have sub-commands and these will also show help:

$ docker system --help
$ docker system prune --help

Obtain information about images and containers

The ‘docker’ command-line utility can give you a lot of information about your docker infrastructure. I will discuss a graphical management tool a bit further down, but here are some useful commands:

Get a listing of all images on your Docker host:

$ docker images

Get a listing of all running containers:

$ docker ps

Also list those containers that are in a stopped state:

$ docker ps -a

Show diskspace used by all your images, containers, data volumes and caches:

$ docker system df

Show the logs of a container:

$ docker logs CONTAINER_ID

Get the ID of a container or an image:

$ docker ps -aqf "name=containername"

The “containername” is interpreted as a regular expression. If you omit the “q” switch you will see the full info about the container, instead of only the container ID. Find an image ID using the following command (if you use a containername instead of an imagename you have a second way to get a container ID):

$ docker inspect --format="{{.Id}}" imagename

Note that in this case, “imagename” needs to be the exact string of the image (or container) as shown in the ‘docker ps‘ output, not a regular expression or part of the name.

Get the filesystem location of a data volume:

Data volumes are managed by Docker, on a Linux host the actual physical location of a Docker volume is somewhere below “/var/lib/docker/volumes/“. but on a Windows Docker host that will be something entirely different.
Using the ‘docker inspect‘ command you can find the actual filesystem location of a data volume by evaluating the “Mountpoint” value in the output. Example from my local server:

$ docker inspect volume portainer_data
[
  {
    "CreatedAt": "2021-08-01T12:44:24+02:00",
    "Driver": "local",
    "Labels": {},
    "Mountpoint": "/var/lib/docker/volumes/portainer_data/_data",
    "Name": "portainer_data",
    "Options": {},
    "Scope": "local"
  }
]

You could use this information to access data inside that volume directly, without using Docker tools.

More:

There’s a lot more but I leave it up to you to find other commands that are informational for you.

Stop and remove a container

Note that when you stop and remove a container, you do not delete the image(s) on which the container is based! So what is the usefulness of removing containers? Simple: sometimes a container crashes or gets corrupted. In such a case you simply stop the container, remove it and start it again using the exact same command which was used earlier to start it. The container will be rebuilt from scratch from the still existing images on your local file-system and the service it provides will start normally again.
As an example, create and run the ‘hello-world‘ container, then stop and remove it again:

$ docker run hello-world
Hello from Docker! 
This message shows that your installation appears to be working correctly.
...
$ docker ps -a |grep hello-world
0369206ddf2d   hello-world                 "/hello"  ...
$ CONTAINER_ID=$(docker ps -a |grep hello-world | awk '{print $1}')
$ docker stop $CONTAINER_ID
$ docker rm $CONTAINER_ID

Refresh a container with the latest image from the vendor

Suppose you are running a Docker container for some complex piece of software. You can’t simply run “slackpkg upgrade-all” to upgrade that software to its latest version or to fix some security vulnerability. Instead, what you do with a container is to refresh the Docker image which underpins that container, then stop the container, remove the container layer, remove the old vendor image, and then start the container again. On startup  of the container, it will now be based on the latest vendor image.
Configuration of a container happens either inside of data volumes that you maintain outside of the container and/or on the startup command-line of that container. Deleting the container or the image layers is not touching any of your configuration or your data.
This is the strong point of container life cycle management – it’s so easy to automate and scale up.

As an example, let’s refresh a container that I have running here at home (the Portainer container that’s discussed further down).

Download the latest Portainer Community Edition image from Docker Hub:

$ docker pull portainer/portainer-ce

Find the ID (first comumn) of the portainer container that we currently have running:

$ docker ps -af "name=portainer"

Stop and remove the container:

$ docker stop CONTAINER_ID
$ docker rm CONTAINER_ID

Find the ID of the portainer image which is oldest (look in the CREATED column) and remove it:

$ docker images | grep portainer
$ docker image rm OLDIMAGE_ID

Check the ID of the remaining image:

$ docker images | grep portainer

Start the container using the same command you would have used before to start it:

$ docker run -d --restart unless-stopped -p 127.0.0.1:9000:9000 --name portainer -v /var/run/docker.sock:/var/run/docker.sock -v /usr/share/docker/data/portainer:/data portainer/portainer-ce

And now we are running the same Portainer configuration, but with the latest available software version. A matter of seconds if you would script this.

Managing  containers, images, volumes, networks etc

As time passes and you experiment with containers, you will inevitably end up with stuff on your host computer that is no longer being used. Think of images which you wanted to try out with ‘docker run‘ or downloaded using ‘docker pull‘. Or cached data resulting from image building. The ‘docker system df‘ command I showed you earlier will give you a scare perhaps if you run it… see here the output when I run it on my server:

TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 11 10 3.424GB 390.3MB (11%)
Containers 10 3 531.8MB 32.01MB (6%)
Local Volumes 3 1 0B 0B
Build Cache 0 0 0B 0B

The “reclaimable” percentage is data which is no longer being used by any container that Docker knows about. You can safely delete that data. The following command will do that for you:

$ docker system prune

If you want to make sure that every un-used scrap of image data is deleted, not just the dangling symlinked images, run:

$ docker system prune -a

And if you want to remove un-used data volumes, you would run:

$ docker system prune --volumes

. . . of course you execute this command only after you have verified that the affected volumes do not contain data you still need.

Difference between load/save and import/export

Docker manages images, containers, data volumes etc in an OS independent way. Meaning you are able to transport data volumes, images and containers from one host to another, or even from one type of OS (Linux) to another (MS Windows) and preserve compatibility.

Import and export commands – used for migration of container filesystems

Docker command ‘docker export‘ allows you to create a (compressed) tar archive containing the bare file system of a container:

$ docker export my_container | xz > mycontainer.tar.xz

This filesystem tarball can again be imported into a new ‘filesystem image‘. Also look at my example of creating a Slackware base image using ‘docker import‘.

$ xz -cd mycontainer.tar.xz | docker import - my_newcontainer

The export and import commands don’t preserve the container’s configuration and underlying image information, you are dealing with just the filesystem. This makes these commands quite suited for creating Docker base images which you can then base your future containers and images on via the “FROM” keyword in a Dockerfile.
For actual Docker migrations to another host, the load and save commands are better suited. Read on:

Load and save commands – used for host migration of images

Docker command ‘docker save‘  allows you to create a (compressed) tar archive of a Docker image, containing all its layers, tags, history and configuration. The only thing that is not added to this tarball are the data volumes.

If you want to migrate a container, you first save (commit) its changes as a new Docker image using the ‘docker commit’ command.

$ docker commit CONTAINER_ID my_important_image

… and then save that image in a OS-agnostic way into a tarball. The ‘save’ command does not compress the data so we use ‘xz’ for that:

$ docker save my_important_image | xz > my_important_image.tar.xz

This compressed tarball can be copied to another host (potentially with a different OS running on the host) and ‘docker load‘ allows you to load this image (even when compressed) with all its tag information and history into the new host’s Docker environment.

$ cat my_important_image.tar.xz | docker load

You can then use the ‘docker run‘ command as usual to start a new container based off that image.

And what to do with the data volumes? Docker manages its volumes in an OS agnostic way which means we can probably not just create a tarball from the volume directory on the host, since that may not be compatible with how a data volume is implemented on another host. If we use Docker commands to save a data volume and load it on another host, we should be covered. The Docker documentation on volumes has an example on how to migrate data volumes.

Saving a data volume to a tarball:

Suppose your ‘portainer‘ container uses a data volume called ‘portainer_data‘ which is mounted on the container’s “/data” directory, containing important data which you want to migrate to another host.
To generate a tarball of the data in this volume, you create a one-shot container based on some base image (I’ll use ‘alpine’) that does not do anything in itself, and give that container the command to create the backup of your ‘portainer_data‘ volume. The steps are:

  • Launch a new container and mount the volume from the ‘portainer‘ container
  • Mount a local host directory as /backup
  • Pass a command that tars the contents of the ‘portainer_data‘ volume to a backup.tar file inside our /backup directory.
  • When the command completes and the container stops, we are left with a backup of our ‘portainer_data‘ volume; the file “backup.tar” in our current directory.
$ docker run --rm --volumes-from portainer -v $(pwd):/backup alpine tar cvf /backup/backup.tar /data

… and on a new host, load this tarball into the data volume of the ‘portainer’ container which you have already setup there but it’s still missing its data:

$ docker run --rm --volumes-from portainer -v $(pwd):/backup alpine bash -c "cd /data && tar xvf /backup/backup.tar --strip 1"

Voilà!

Limiting the log size for your containers

Docker captures the standard & error output (stdout/stderr) for all your containers and writes these to logfiles in JSON format. By default, Docker does not impose size restrictions on its log files.
These logfiles are shown when you run a ‘docker logs CONTAINER_ID‘ command.
This behavior could potentially fill up your hard drive if you run a container which generates an extensive amount of logging data. By modifying Docker’s “/etc/docker/daemon.json” configuration file we can limit the maximum size of the container log files. This is what you need to add to the Docker daemon configuration to limit the size of any log file to 50 MB and keep at most two backups after log rotation:

{
  "log-driver": "json-file",
    "log-opts": {
    "max-size": "50m",
    "max-file": "3"
    }
}

More details to be found at https://docs.docker.com/config/containers/logging/json-file/.

Graphical management tool

There’s a nice graphical web-based administration tool for your Docker infrastructure which itself runs in a container (you can also install it on bare metal as a self-contained executable), called portainer. Portainer will listen at a TCP port of your host and you can point your web browser at http://localhost:9000/ to access the GUI. The Community Edition (portainer-ce) is open source and free and this is what we’ll install and run as a Docker container.
The Portainer developers posted a tutorial and feature walk-through of their product on Youtube:

Some considerations before we start implementing this.

Data persistence:
Portainer wants to keep some data persistent across restarts. When it is run from within a container, you want to have that persistent data outside of your container, somewhere on the local filesystem. The Docker concept for persistent data management is called “volumes”. Docker volumes are created as directories within “/var/lib/docker/volumes/” on Linux. A volume is made available in the container using the “-v” switch to the ‘docker run‘ command, which maps a host volume to an internal container directory. Docker manages its volumes in an OS agnostic way, making it possible to transparently move your containers from Linux to Windows for instance.
We will not use a Docker volume in this example but instead create the “/usr/share/docker/data/portainer” directory and pass that as the external location for the container’s internal “/data” directory.

Network security:
We will expose the Portainer network listen port to only localhost and then use an Apache reverse proxy to let clients connect. Note that port 9000 exposes the Portainer UI without SSL encryption; we will use Apache reverse proxy to handle the encryption.

Application security:
Below you will see that I am mapping the Docker management socket “/var/run/docker.sock” into the container. Normally that would be very bad practice, but Portainer is going to be our Docker management tool and therefore it needs the access to this socket.

This batch of commands will get your Portainer-CE instance up and running and it will be restarted when Docker starts up (eg. after rebooting).

# mkdir -p /usr/share/docker/data/portainer
# docker run -d --restart unless-stopped -p 127.0.0.1:9000:9000 --name portainer -v /var/run/docker.sock:/var/run/docker.sock -v /usr/share/docker/data/portainer:/data portainer/portainer-ce

Two of the three network ports that Portainer opens up are not being exposed (mapped) to the outside world:
Port 8000 exposes a SSH tunnel server for Portainer Agents, but we will not use this. Portainer Agents are installed on Docker Swarm Clusters to allow central management of your Swarm from a single Portainer Server.
Also, the Portainer container generates a self-signed SSL certificate on first startup and uses it to offer https encrypted connections on port 9443 but our reverse proxy solution allows the use of a proper Let’s Encrypt certificate.

Now that the Portainer container is running, connect a browser on your host computer to http://localhost:9000/ to perform initial setup; basically defining the password for the Portainer admin user account.

The reverse proxy configuration including adding a Let’s Encrypt certificate is a bit beyond the scope of this article, but you can read this earlier blog post to learn about securing your webserver with SSL certificates.

I will share the block that you would have to add to your Apache configuration somehow. With this configuration and proper SSL setup your Portainer would be accessible via https://your_host_name/portainer/ instead of only at http://localhost:9000/ :

# Create a reverse proxy:
<Location /portainer>
    ProxyPass http://127.0.0.1:9000
    ProxyPassReverse http://127.0.0.1:9000
</Location>

Here is a screenshot of my Docker infrastructure at home as interpreted by Portainer:

Have a look at Portainer to see if it adds something useful to your Docker management toolkit. For me the most useful feature is the real-time container logging display, which is a lot more convenient than trying to scroll back in a Linux screen session.

Docker’s effect on iptables firewall

Docker dynamically modifies your host’s iptables ruleset on Linux when it starts, in order to provide container isolation. See https://docs.docker.com/network/iptables/ for the rationale. This can sometimes interfere with pre-existing services, most prominently the Virtual Machines that you may also be running and whose virtual network has been bridged to your host’s network interface.
I use vde (virtual distributed ethernet) for instance to provide transparent bridged network connectivity for the QEMU VM’s that I use when building packages for Slackware.
With this bridged network setup, your host computer essentially acts as a router, forwarding network packets across the bridge.
When Docker starts, it will add two custom iptables chains named ‘DOCKER’ and ‘DOCKER-USER’, adds its own custom rules to the ‘DOCKER’ chain and then sets the default FORWARD policy to DROP. Thereby killing the network connectivity for the already bridged networks… Bad. But even though Docker has a configurable parameter to disable iptables, this only prevents the creation of these custom chains and rules when the Docker daemon starts. Docker confesses that there is no guarantee that later on when you start containers, your iptables rulesets will stay un-touched. So, disabling iptables management in Docker is not an option. I’ll tell you a bit on how to deal with this (all information gleaned from the Internet).

Generic iptables

Since Docker sets the policy for the FORWARD chain to DROP, for your bridged networks this will result loss of connectivity (packets are no longer forwarded across the bridge).
You can use the custom DOCKER-USER chain to ensure that this bridging is not disrupted. Docker inserts these custom chains in such a way that rules defined in the DOCKER-USER chain have precedence over rules in the DOCKER chain. You have to add an explicit ACCEPT rule to that DOCKER-USER chain:

# iptables -I DOCKER-USER -i br0 -j ACCEPT

Where ‘br0‘ is the name of your network bridge (adapt if needed).

UFW (Uncomplicated Firewall)

UFW is a tool created for Ubuntu but popular on all distros, which aims at simplifying iptables ruleset management by providing a higher-level commandline interface. There’s a package build script for it on SlackBuilds.org.

UFW allows to insert rules that are evaluated  after all other chains have been traversed. This allows the user to nullify the unwanted behavior of Docker which kills bridged networks. See this github discussion for the full details.

Append the following at the end of /etc/ufw/after.rules (replace eth0 with your external facing interface):

# Put Docker behind UFW
*filter
:DOCKER-USER - [0:0]
:ufw-user-input - [0:0]

-A DOCKER-USER -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-USER -m conntrack --ctstate INVALID -j DROP
-A DOCKER-USER -i eth0 -j ufw-user-input
-A DOCKER-USER -i eth0 -j DROP
COMMIT

And undo any and all of:

  • Remove “iptables”: “false” from /etc/docker/daemon.json
  • Revert to DEFAULT_FORWARD_POLICY=”DROP” in /etc/default/ufw
  • Remove any docker related changes to /etc/ufw/before.rules

Firewalld

Firewalld is a dynamically managed firewall which works with a ‘zones‘ concept and gets its triggers from a D-Bus interface. There’s a package build script available on SlackBuilds.org.

If you are running Docker version 20.10.0 or higher with firewalld on your system with and Docker has iptables support enabled, Docker automatically creates a firewalld zone called docker and inserts all the network interfaces it creates (for example, docker0) into the docker zone to allow seamless networking.

Consider running the following firewalld command to remove the docker interface from the ‘trusted‘ zone.

# Please substitute the appropriate zone and docker interface
$ firewall-cmd --zone=trusted --remove-interface=docker0 --permanent
$ firewall-cmd --reload

Restarting the docker daemon (/etc/rc.d/rc.docker restart) will then insert the interface into the docker zone.

Alien’s Easy Firewall Generator (EFG)

If you used my EFG to create a firewall configuration, you will be bitten by the fact that Docker and Docker Compose create new network devices . Permissive rules need to be be added to the INPUT and OUTPUT chains of your “rc.firewall” or else ‘dmesg’ will be full of ‘OUTPUT: packet died’.

The Docker network interface is called “docker0” and for each container, Docker creates an additional bridge interface, the name of which starts with “br-” and is followed by a hash value. Therefore, these lines need to be applied at the appropriate locations in the script:

DOCK_IFACE="docker0"
BR_IFACE="br-+"
$IPT -A INPUT -p ALL -i $DOCK_IFACE -j ACCEPT
$IPT -A INPUT -p ALL -i $BR_IFACE -j ACCEPT
$IPT -A OUTPUT -p ALL -o $DOCK_IFACE -j ACCEPT
$IPT -A OUTPUT -p ALL -o $BR_IFACE -j ACCEPT

Thanks

… for reading until the end! I hope you gained some knowledge. Feel free to leave constructive feedback in the comments section below.

Cheers, Eric


Attribution

Most of the firewall-related information comes from docker.com and github.com. Please inform me about errors.
All images in this article were taken from docs.docker.com website.

Newer posts »

© 2024 Alien Pastures

Theme by Anders NorenUp ↑