To install dcos with virtual box is easy. Follow the instructions on https://github.com/dcos/dcos-vagrant/tree/v0.6.0 and you are finished in short time.
In this guide I want to show to install DCOS with you own virtual machines like an on promise install.
Parts to install
In DCOS we have several parts to install. This picture out of DCOS security guide present the layers.
In this guide we install only the admin/master part and private agents. For installation we also need a bootstrap node.
Preparation of VM host
Install DNS to resolve the hostname of all VMs. I installed bind on the vm host and added a forward and reverse zone equal to the VMs.
- yum install bind
zone 33.168.192.in-addr.arpa IN { type master; file /etc/bind/zones/master/33.168.192.in-addr.arpa allow-update { none; }; }; zone minions.local { type master; file /etc/bind/zones/master/db.minions.local };
Generate ssh keys
I generated a key pair with ssh-keygen and copied the key into folder .ssh at the base sync folder for the vagrant VMs. The bootstrap.sh script copies the keys inside the VMs to the root user. I added an authorized_keys in this folder, too. See appendix for the script.
Installing VMs with Vagrant
I use vagrant to install the VMs with a bootstrap.sh script. These are the detailed steps:
- download a VM image
- download all software needed in the VMs
- install ssh keys on all VM nodes
I installed the following numbers of VMs:
- 1 bootstrap node
- 3 master nodes
- 4 agent nodes
Here is an example to modify a standard vagrant file. The bootstrap.sh, mentioned above, you can find at the end of the blog. The file installs missing packages and docker in the nodes.
Vagrant.configure(2) do |config| config.vm.box = centos/7 config.vm.provision :shell, path: bootstrap.sh config.vm.define min1 do |min1| min1.vm.network private_network, ip: "192.168.33.11 min1.vm.hostname = minions-1.minions.local end config.vm.define min2 do |min2| min2.vm.network private_network, ip: 192.168.33.12 min2.vm.hostname = minions-2.minions.local end ... config.vm.define min8 do |min3| min3.vm.network private_network, ip: 192.168.33.18 min3.vm.hostname = minions-8.minions.local end # config.vm.provider virtualbox do |vb| vb.name = minion vb.memory = 3072 end end
Start install DCOS
All commands are executed as root!
After all VMs are up and runnig we prepare the machines. We log into the first vm (vagant ssh min1). We try to connect to all other VMs. This should run without passords. The first time you must accept the key.
[root@minions-1 ~]# ssh minions-2 The authenticity of host 'minions-2 (192.168.33.12)' can't be established. ECDSA key fingerprint is db:8b:09:62:3e:43:37:14:f2:36:4c:fd:27:c1:54:60. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'minions-2,192.168.33.12' (ECDSA) to the list of known hosts. Last login: Mon May 16 14:34:48 2016 from 192.168.33.11
If you want to use a shared external storage for Exhibitor you must add a zookeeper instance at your bootstrap node.
sudo docker run -d -p 2181:2181 -p 2888:2888 -p 3888:3888 -v /var/zookeeper/dcos:/tmp/zookeeper --name=dcos_int_zk jplock/zookeeper
See https://docs.mesosphere.com/administration/installing/custom/gui/
Download the installation files – you find all versions here: https://dcos.io/releases. I took the newest one.
Create some configuration files to have a easy install. All files are placed under the directory genconf in home of root user.
- config.yaml
--- agent_list: - 192.168.33.15 - 192.168.33.16 - 192.168.33.17 - 192.168.33.18 bootstrap_url: file:///opt/dcos_install_tmp cluster_name: DCOS exhibitor_storage_backend: static ip_detect_filename: /genconf/ip-detect master_discovebitor_storage_backend: zookeeper exhibitor_zk_hosts: 192.168.33.11:2181 exhibitor_zk_path: /dcos master_discovery: static master_list: - 192.168.33.12 - 192.168.33.13 - 192.168.33.14 process_timeout: 10000 resolvers: - 8.8.8.8 - 8.8.4.4 ssh_port: 22 ssh_user: root
- ip-detect
This script detects your IP address of the node. The output of the script is the IP address for communication between the nodes.
#!/usr/bin/env bash set -o nounset -o errexit export PATH=/usr/sbin:/usr/bin:$PATH echo $(ip addr show eth0 | grep -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | head -1)
- ssh_key
Copy .ssh/id_rsa to getconf/ssh_key for passwordless authentication.
Start the installer, if you want to check your input in graphical mode:
- bash dcos_generate_config.sh –web
Connect to the console on port 9000 – check the input parameters and you pre flight should be successful.
Go further like documented in installation guide att dcos.io. You find the admin page on port 80 of your master nodes.
Appendix bootstrap.sh
#!/usr/bin/env bash #update it # use 7.2 sudo yum upgrade --assumeyes --tolerant sudo yum update --assumeyes sudo tee /etc/modules-load.d/overlay.conf << 'EOF' overlay EOF sudo systemctl stop firewalld sudo systemctl disable firewalld sudo tee /etc/yum.repos.d/docker.repo << 'EOF' [dockerrepo] name=Docker Repository baseurl=https://yum.dockerproject.org/repo/main/centos/$releasever/ enabled=1 gpgcheck=1 gpgkey=https://yum.dockerproject.org/gpg EOF sudo yum install -y docker-engine sudo systemctl start docker sudo systemctl enable docker sudo mkdir -p /etc/systemd/system/docker.service.d sudo tee /etc/systemd/system/docker.service.d/override.conf << EOF [Service] ExecStart= ExecStart=/usr/bin/docker daemon --storage-driver=overlay -H fd:// EOF sudo sed -i s/SELINUX=enforcing/SELINUX=permissive/g /etc/selinux/config sudo groupadd nogroup mkdir /root/.ssh cp sync/.ssh/id_rsa /root/.ssh/id_rsa cp sync/.ssh/id_rsa.pub /root/.ssh/id_rsa.pub cp sync/.ssh/authorized_keys /root/.ssh/authorized_keys sudo reboot
seems to bee issues with overlay filesystem and Centos/Redhat see
https://github.com/docker/docker/issues/10294 ext4 doesn’t work – logs like
I0517 17:37:04.696264 7625 exec.cpp:143] Version: 0.28.1
I0517 17:37:04.703060 7626 exec.cpp:217] Executor registered on slave 6068d81f-fe07-4edd-812a-993100102ee4-S4
docker: Error response from daemon: mkdir /var/lib/docker/overlay/7f871e91a401bc4bcd0c1dbf56616531e202acc0aa8a14b40d49f8e952bf91b9-init/merged/dev/shm: invalid argument.
See ‘docker run –help’.
W0517 17:37:04.703060 7631 logging.cpp:88] RAW: Received signal SIGTERM from process 1429 of user 0; exiting
in bootstrap.sh please convert &, <, etc. back to ASCII.
I have edited it – but please be aware of docker version!