Cluster Hat setup - Part 4 - Docker Registry

Submitted by code_admin on Wed, 07/25/2018 - 15:23

Cluster Hat setup - Part 1
Cluster Hat setup - Part 2
Cluster Hat setup - Part 3 - Ansible
Cluster Hat setup - Part 4 - Docker Registry
Cluster Hat setup - Part 5 - Access Point Setup
Cluster Hat - Other Processes
Set up a new Raspberry Pi 3 to join the cluster

Step 20 - Setup the controller as a docker regisrty

(I referenced https://docs.docker.com/registry/deploying/ and http://matthewkwilliams.com/index.php/2015/03/29/swarming-raspberry-pi-… for this step)

We will need a directory for our registry data

  1. mkdir ~/dockerregistry
  2. mkdir ~/dockerregistry/data
  3. mkdir ~/dockerregistry/certs

Next we need to create a certificate for the registry.

  1. openssl req -newkey rsa:4096 -nodes -sha256 -keyout ~/dockerregistry/certs/controller.metcarob-local.com.key -x509 -days 365 -out ~/dockerregistry/certs/controller.metcarob-local.com.crt

I made the Common Name (CN)=controller.metcarob-local.com

**TODO CACERTS NOT IN USE - change this config to use secure connection to docker repo

Now run the docker registery. This is a docker image I can run on the controller.

  1. docker run \
  2.     --name registry \
  3.     -p 5000:5000 \
  4.     -v ~/dockerregistry/data:/tmp/registry \
  5.     -d \
  6.     -e MIRROR_SOURCE=https://registry-1.docker.io  \
  7.     -e MIRROR_SOURCE_INDEX=https://index.docker.io \
  8.     -e GUNICORN_OPTS=["--preload"] \
  9.     nimblestratus/rpi-docker-registry

You can test the controller is working by running:

If something goes wrong and I need to stop and delete the image and try again I use:

  1. docker stop registry && docker rm -v registry

Next set up docker so it will use the registry by changing it's startup options.

  1. sudo mkdir /etc/systemd/system/docker.service.d
  2. sudo nano /etc/systemd/system/docker.service.d/overlay.conf

Enter the following text into the file then save:

  1. [Service]
  2. ExecStart=
  3. ExecStart=/usr/bin/dockerd --storage-driver overlay -H fd:// --insecure-registry controller.metcarob-local.com:5000

I then had to stop and delete the instance and restart docker.

  1. docker stop registry && docker rm -v registry
  2. sudo systemctl daemon-reload
  3. sudo systemctl restart docker
  4. ps -aux | grep docker

The final command will output the command line used to start the deamon and confirm if this was correct.

I then re-started the registry image using the same command as before before pushing the apachepi image to the registry:

  1. docker push controller.metcarob-local.com:5000/apachepi:v1

Once I am satisified that I have a working image for the docker registry I need to make it a service so it starts and stops automatically with the controller.
To do this I create a file: /etc/systemd/system/docker_registry.service

  1. [Unit]
  2. Description=Docker Registry
  3. Requires=docker.service
  4. After=docker.service
  5.  
  6. [Service]
  7. Restart=always
  8. ExecStart=/usr/bin/docker start -a registry
  9. ExecStop=/usr/bin/docker stop -t 2 registry
  10.  
  11. [Install]
  12. WantedBy=multi-user.target
  1. sudo chmod "u=rwx,g=rwx,o=rwx" /etc/systemd/system/docker_registry.service
  2. sudo systemctl enable docker_registry

Now after a restart the docker registry will start on port 5000.

Step 21 - Deploy docker containers to nodes in the cluster

Note: I had lots of problems with python versions. Ansible modules use both python versions 2 and 3 and i got many compatibility errors. The first time I did this process it was working but when I came back to it dependencies had changed. I am leaving these steps in the tutorial so you will know how I approached creating playbooks but if you are trying to follow these instructions you may get errors. Comparing files here with the versions I have in the repo (https://github.com/rmetcalf9/metcarob-local_cluster) might help.

The following yml file will install the docker image onto the nodes in the cluster:
(~/ansible/install_apahcepi.yml)

  1. ---
  2. - hosts: webservers
  3.   become: true
  4.   vars:
  5.     service:
  6.       longname: Apache Webserver
  7.       shortname: apachepi_container
  8.  
  9.   tasks:
  10.   - name: install pip
  11.     apt:
  12.       name: python-pip
  13.       state: present
  14.  
  15.   - name: pull apachepi image
  16.     docker_image:
  17.       name: controller.metcarob-local.com:5000/apachepi:v1
  18.  
  19.   - name: Create apache docker container
  20.     docker_container:
  21.       name: "{{ service.shortname }}"
  22.       image: controller.metcarob-local.com:5000/apachepi:v1
  23.       ports:
  24.       - "80:80"
  25.  
  26.   - name: Install apachepi.service file
  27.     template:
  28.       src: docker_any.service
  29.       dest: /etc/systemd/system/docker_{{ service.shortname }}.service
  30.       owner: root
  31.       group: root
  32.       mode: "u=rwx,g=rwx,o=rwx"
  33.  
  34.   - name: Enable apache service
  35.     systemd:
  36.       name: docker_{{ service.shortname }}
  37.       enabled: true
  38.       state: started
  39.       masked: no

Create files required for the playbook
~/ansible/docker_any.service

  1. [Unit]
  2. Description={{ service.longname }}
  3. Requires=docker.service
  4. After=docker.service
  5.  
  6. [Service]
  7. Restart=always
  8. ExecStart=/usr/bin/docker start -a {{ service.shortname }}
  9. ExecStop=/usr/bin/docker stop -t 2 {{ service.shortname }}
  10.  
  11. [Install]
  12. WantedBy=multi-user.target

~/ansible/install_docker.sh

  1. #!/bin/bash
  2.  
  3. curl -sSL https://get.docker.com | sh
  4.  
  5. if [[ $? -ne 0 ]]
  6. then
  7.     exit 1
  8. fi
  9.  
  10. exit 0

Run the playbook:

  1. cd ~/ansible
  2. ansible-playbook -i ~/ansible/hosts install_apahcepi.yml

We now have the webservers running on p1 and p2.

Step 22 - Checkpoint

Finally lets check it is all working.
First we should check we can access the apache web servers running on the Pi Zero's from the controller

SSH to the controller and goto the home directory.

  1. cd ~
  2. wget p1
  3. wget p2

If all is working you should end up with two index.html files in the home directory.

For completeness I restarted the cluster completely and retried this test. This showed up a problem for me but I have gone back through the instructions and fixed the image so you should not see this problem.

Next Part

Of course webservers on un-reacable nodes in our cluster are useless so I will address this in the next part:
Cluster Hat setup - Part 5 - Access Point Setup

RJM Article Type
Public Article