Installing DCOS with your own Virtualbox VMs

To install dcos with virtual box is easy. Follow the instructions on https://github.com/dcos/dcos-vagrant/tree/v0.6.0 and you are finished in short time.

In this guide I want to show to install DCOS with you own virtual machines like an on promise install.

Parts to install

In DCOS we have several parts to install. This picture out of DCOS security guide present the layers.

In this guide we install only the admin/master part and private agents. For installation we also need a bootstrap node.

Preparation of VM host

Install DNS to resolve the hostname of all VMs. I installed bind on the vm host and added a forward and reverse zone equal to the VMs.

• yum install bind

zone 33.168.192.in-addr.arpa IN {
type master;
file /etc/bind/zones/master/33.168.192.in-addr.arpa
allow-update { none; };
};

zone minions.local {
type master;
file /etc/bind/zones/master/db.minions.local
};

Generate ssh keys

I generated a key pair with ssh-keygen and copied the key into folder .ssh at the base sync folder for the vagrant VMs. The bootstrap.sh script copies the keys inside the VMs to the root user. I added an authorized_keys in this folder, too. See appendix for the script.

Installing VMs with Vagrant

I use vagrant to install the VMs with a bootstrap.sh script. These are the detailed steps:

• download a VM image
• download all software needed in the VMs
• install ssh keys on all VM nodes

I installed the following numbers of VMs:

• 1 bootstrap node
• 3 master nodes
• 4 agent nodes

Here is an example to modify a standard vagrant file. The bootstrap.sh, mentioned above, you can find at the end of the blog. The file installs missing packages and docker in the nodes.

Vagrant.configure(2) do |config|
config.vm.box = centos/7

config.vm.provision :shell, path: bootstrap.sh

config.vm.define min1 do |min1|
min1.vm.network private_network, ip: "192.168.33.11
min1.vm.hostname = minions-1.minions.local
end

config.vm.define min2 do |min2|
min2.vm.network private_network, ip: 192.168.33.12
min2.vm.hostname = minions-2.minions.local
end

...

config.vm.define min8 do |min3|
min3.vm.network private_network, ip: 192.168.33.18
min3.vm.hostname = minions-8.minions.local
end

#
config.vm.provider virtualbox do |vb|
vb.name = minion
vb.memory = 3072
end
end

Start install DCOS

All commands are executed as root!

After all VMs are up and runnig we prepare the machines. We log into the first vm (vagant ssh min1). We try to connect to all other VMs. This should run without passords. The first time you must accept the key.

[root@minions-1 ~]# ssh minions-2
The authenticity of host 'minions-2 (192.168.33.12)' can't be established.
ECDSA key fingerprint is db:8b:09:62:3e:43:37:14:f2:36:4c:fd:27:c1:54:60.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'minions-2,192.168.33.12' (ECDSA) to the list of known hosts.
Last login: Mon May 16 14:34:48 2016 from 192.168.33.11

If you want to use a shared external storage for Exhibitor you must add a zookeeper instance at your bootstrap node.

sudo docker run -d -p 2181:2181 -p 2888:2888 -p 3888:3888 -v /var/zookeeper/dcos:/tmp/zookeeper --name=dcos_int_zk jplock/zookeeper

See https://docs.mesosphere.com/administration/installing/custom/gui/

Download the installation files – you find all versions here: https://dcos.io/releases. I took the newest one.

• curl -O https://downloads.dcos.io/dcos/EarlyAccess/dcos_generate_config.sh

Create some configuration files to have a easy install. All files are placed under the directory genconf in home of root user.

• config.yaml

---
agent_list:
- 192.168.33.15
- 192.168.33.16
- 192.168.33.17
- 192.168.33.18
bootstrap_url: file:///opt/dcos_install_tmp
cluster_name: DCOS
exhibitor_storage_backend: static
ip_detect_filename: /genconf/ip-detect
master_discovebitor_storage_backend: zookeeper
exhibitor_zk_hosts: 192.168.33.11:2181
exhibitor_zk_path: /dcos
master_discovery: static
master_list:
- 192.168.33.12
- 192.168.33.13
- 192.168.33.14
process_timeout: 10000
resolvers:
- 8.8.8.8
- 8.8.4.4
ssh_port: 22
ssh_user: root

• ip-detect

This script detects your IP address of the node. The output of the script is the IP address for communication between the nodes.

#!/usr/bin/env bash
set -o nounset -o errexit
export PATH=/usr/sbin:/usr/bin:$PATH
echo $(ip addr show eth0 | grep -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | head -1)

• ssh_key

Copy .ssh/id_rsa to getconf/ssh_key for passwordless authentication.

Start the installer, if you want to check your input in graphical mode:

• bash dcos_generate_config.sh –web

Connect to the console on port 9000 – check the input parameters and you pre flight should be successful.

Go further like documented in installation guide att dcos.io. You find the admin page on port 80 of your master nodes.

Appendix bootstrap.sh

#!/usr/bin/env bash

#update it
# use 7.2
sudo yum upgrade --assumeyes --tolerant
sudo yum update --assumeyes

sudo tee /etc/modules-load.d/overlay.conf << 'EOF'
overlay
EOF

sudo systemctl stop firewalld
sudo systemctl disable firewalld

sudo tee /etc/yum.repos.d/docker.repo << 'EOF'
[dockerrepo]
name=Docker Repository
baseurl=https://yum.dockerproject.org/repo/main/centos/$releasever/
enabled=1
gpgcheck=1
gpgkey=https://yum.dockerproject.org/gpg
EOF

sudo yum install -y docker-engine
sudo systemctl start docker
sudo systemctl enable docker

sudo mkdir -p /etc/systemd/system/docker.service.d 

sudo tee /etc/systemd/system/docker.service.d/override.conf  << EOF

[Service]
ExecStart=
ExecStart=/usr/bin/docker daemon --storage-driver=overlay -H fd://
EOF

sudo sed -i s/SELINUX=enforcing/SELINUX=permissive/g /etc/selinux/config
sudo groupadd nogroup 
mkdir /root/.ssh
cp sync/.ssh/id_rsa /root/.ssh/id_rsa
cp sync/.ssh/id_rsa.pub /root/.ssh/id_rsa.pub
cp sync/.ssh/authorized_keys /root/.ssh/authorized_keys

sudo reboot