Cluster Hat setup - Part 1
Cluster Hat setup - Part 2
Cluster Hat setup - Part 3 - Ansible
Cluster Hat setup - Part 4 - Docker Registry
Cluster Hat setup - Part 5 - Access Point Setup
Cluster Hat - Other Processes
Set up a new Raspberry Pi 3 to join the cluster
Step 20 - Setup the controller as a docker regisrty
(I referenced https://docs.docker.com/registry/deploying/ and http://matthewkwilliams.com/index.php/2015/03/29/swarming-raspberry-pi-… for this step)
We will need a directory for our registry data
-
mkdir ~/dockerregistry
-
mkdir ~/dockerregistry/data
-
mkdir ~/dockerregistry/certs
Next we need to create a certificate for the registry.
-
openssl req -newkey rsa:4096 -nodes -sha256 -keyout ~/dockerregistry/certs/controller.metcarob-local.com.key -x509 -days 365 -out ~/dockerregistry/certs/controller.metcarob-local.com.crt
I made the Common Name (CN)=controller.metcarob-local.com
**TODO CACERTS NOT IN USE - change this config to use secure connection to docker repo
Now run the docker registery. This is a docker image I can run on the controller.
-
docker run \
-
--name registry \
-
-p 5000:5000 \
-
-v ~/dockerregistry/data:/tmp/registry \
-
-d \
-
-e MIRROR_SOURCE=https://registry-1.docker.io \
-
-e MIRROR_SOURCE_INDEX=https://index.docker.io \
-
-e GUNICORN_OPTS=["--preload"] \
-
nimblestratus/rpi-docker-registry
You can test the controller is working by running:
If something goes wrong and I need to stop and delete the image and try again I use:
-
docker stop registry && docker rm -v registry
Next set up docker so it will use the registry by changing it's startup options.
-
sudo mkdir /etc/systemd/system/docker.service.d
-
sudo nano /etc/systemd/system/docker.service.d/overlay.conf
Enter the following text into the file then save:
-
[Service]
-
ExecStart=
-
ExecStart=/usr/bin/dockerd --storage-driver overlay -H fd:// --insecure-registry controller.metcarob-local.com:5000
I then had to stop and delete the instance and restart docker.
-
docker stop registry && docker rm -v registry
-
sudo systemctl daemon-reload
-
sudo systemctl restart docker
-
ps -aux | grep docker
The final command will output the command line used to start the deamon and confirm if this was correct.
I then re-started the registry image using the same command as before before pushing the apachepi image to the registry:
-
docker push controller.metcarob-local.com:5000/apachepi:v1
Once I am satisified that I have a working image for the docker registry I need to make it a service so it starts and stops automatically with the controller.
To do this I create a file: /etc/systemd/system/docker_registry.service
-
[Unit]
-
Description=Docker Registry
-
Requires=docker.service
-
After=docker.service
-
-
[Service]
-
Restart=always
-
ExecStart=/usr/bin/docker start -a registry
-
ExecStop=/usr/bin/docker stop -t 2 registry
-
-
[Install]
-
WantedBy=multi-user.target
-
sudo chmod "u=rwx,g=rwx,o=rwx" /etc/systemd/system/docker_registry.service
-
sudo systemctl enable docker_registry
Now after a restart the docker registry will start on port 5000.
Step 21 - Deploy docker containers to nodes in the cluster
Note: I had lots of problems with python versions. Ansible modules use both python versions 2 and 3 and i got many compatibility errors. The first time I did this process it was working but when I came back to it dependencies had changed. I am leaving these steps in the tutorial so you will know how I approached creating playbooks but if you are trying to follow these instructions you may get errors. Comparing files here with the versions I have in the repo (https://github.com/rmetcalf9/metcarob-local_cluster) might help.
The following yml file will install the docker image onto the nodes in the cluster:
(~/ansible/install_apahcepi.yml)
-
---
-
- hosts: webservers
-
become: true
-
vars:
-
service:
-
longname: Apache Webserver
-
shortname: apachepi_container
-
-
tasks:
-
- name: install pip
-
apt:
-
name: python-pip
-
state: present
-
-
- name: pull apachepi image
-
docker_image:
-
name: controller.metcarob-local.com:5000/apachepi:v1
-
-
- name: Create apache docker container
-
docker_container:
-
name: "{{ service.shortname }}"
-
image: controller.metcarob-local.com:5000/apachepi:v1
-
ports:
-
- "80:80"
-
-
- name: Install apachepi.service file
-
template:
-
src: docker_any.service
-
dest: /etc/systemd/system/docker_{{ service.shortname }}.service
-
owner: root
-
group: root
-
mode: "u=rwx,g=rwx,o=rwx"
-
-
- name: Enable apache service
-
systemd:
-
name: docker_{{ service.shortname }}
-
enabled: true
-
state: started
-
masked: no
Create files required for the playbook
~/ansible/docker_any.service
-
[Unit]
-
Description={{ service.longname }}
-
Requires=docker.service
-
After=docker.service
-
-
[Service]
-
Restart=always
-
ExecStart=/usr/bin/docker start -a {{ service.shortname }}
-
ExecStop=/usr/bin/docker stop -t 2 {{ service.shortname }}
-
-
[Install]
-
WantedBy=multi-user.target
~/ansible/install_docker.sh
-
#!/bin/bash
-
-
curl -sSL https://get.docker.com | sh
-
-
if [[ $? -ne 0 ]]
-
then
-
exit 1
-
fi
-
-
exit 0
Run the playbook:
-
cd ~/ansible
-
ansible-playbook -i ~/ansible/hosts install_apahcepi.yml
We now have the webservers running on p1 and p2.
Step 22 - Checkpoint
Finally lets check it is all working.
First we should check we can access the apache web servers running on the Pi Zero's from the controller
SSH to the controller and goto the home directory.
-
cd ~
-
wget p1
-
wget p2
If all is working you should end up with two index.html files in the home directory.
For completeness I restarted the cluster completely and retried this test. This showed up a problem for me but I have gone back through the instructions and fixed the image so you should not see this problem.
Next Part
Of course webservers on un-reacable nodes in our cluster are useless so I will address this in the next part:
Cluster Hat setup - Part 5 - Access Point Setup