Docker and VirtualBox(1)Set up Shared Disk for Virtual Box

Docker and VirtualBox(1)Set up Shared Disk for Virtual Box

I have a lot of things on my VirtualBox. But I am think to change the way I deploy, I should use docker more that I thought.

I plan to share the disk across some virtual machines and deploy only docker images.

Here is a list of what I have on these machines.
ubuntu-master
1. Spark
cd /opt/spark
sbin/start-all.sh

visit the page
http://ubuntu-master:8080/

2. opscenter
cd /opt/opscenter
bin/opscenter -f  #start in the front ground
bin/opscenter     #start in the back ground

visit the page
http://ubuntu-master:8888/opscenter/

3. tomcat
cd /opt/tomcat
bin/startup.sh

visit the page
http://ubuntu-master:8081/bldinfo.txt
http://ubuntu-master:8081/LMMAdmin

4. kafka Monitor
cd /opt/kafka-monitor
nohup bin/startup.sh &

visit the page
http://ubuntu-master:8082/

5. memcached Monitor
memcache-top --sleep 3 --instance ubuntu-dev1,ubuntu-dev2

6. Switch Java Version
sudo update-alternatives --config java
sudo update-alternatives --config javac

7. Hadoop hdfs and yarn
cd /opt/hadoop
sbin/start-dfs.sh
sbin/start-yarn.sh

visit the page
http://ubuntu-master:50070/dfshealth.html#tab-overview
http://ubuntu-master:8088/cluster

8. Haproxy
cd /opt/haproxy
sudo sbin/haproxy -f conf/haproxy.conf

Visit the page
http://ubuntu-master/haproxy-status

9. Mysql cluster database
mysql -u cluster -pcluster

10. hbase
cd /opt/hbase
bin/start-hbase.sh

visit the page
http://ubuntu-master:16030/master-status

11. solr
cd /opt/solr/example
java -jar start.jar

Visit Page
http://ubuntu-master:8983/solr/#/jobs

ubuntu-dev1
1. cassandra
cd /opt/cassandra
nohup cassandra -f conf/cassandra.yaml &

2. zookeeper
cd /opt/zookeeper
zkServer.sh start zoo-dev1.cfg

3. kafka
cd /opt/kafka
nohup bin/kafka-server-start.sh config/server-dev1.properties &

4. rabbitmq
cd /opt/rabbitmq
sudo RABBITMQ_NODE_PORT=5672 RABBITMQ_SERVER_START_ARGS="-rabbitmq_management listener [{port,15672}]" RABBITMQ_NODENAME=rabbit1 sbin/rabbitmq-server -detached

visit page
http://ubuntu-dev1:15672   sillycat/kaishi

5. memcached
cd /opt/memcached
memcached -d -m 50 -p 11211 -u carl -l 0.0.0.0

6. Switch Java Version
sudo update-alternatives --config java
sudo update-alternatives --config javac

ubuntu-dev2
1. cassandra
cd /opt/cassandra
nohup cassandra -f conf/cassandra.yaml &

2. zookeeper
cd /opt/zookeeper
zkServer.sh start zoo-dev2.cfg

3. kafka
cd /opt/kafka
nohup bin/kafka-server-start.sh config/server-dev2.properties &

4. rabbitmq
cd /opt/rabbitmq
sudo RABBITMQ_NODE_PORT=5672 RABBITMQ_SERVER_START_ARGS="-rabbitmq_management listener [{port,15672}]" RABBITMQ_NODENAME=rabbit2 sbin/rabbitmq-server -detached

enable cluster

sudo sbin/rabbitmqctl -n rabbit2 stop_app
sudo sbin/rabbitmqctl -n rabbit2 join_cluster rabbit1@ubuntu-dev1
sudo sbin/rabbitmqctl -n rabbit2 start_app

visit page
http://ubuntu-dev2:15672     sillycat/kaishi

5. memcached
cd /opt/memcached
memcached -d -m 50 -p 11211 -u carl -l 0.0.0.0

6. Switch Java Version
sudo update-alternatives --config java
sudo update-alternatives --config javac

I will export all of them and backup them into my home NAS. I will delete all of them on my local and set up fresh ubuntu LTS for that.

1 Set up Share Driver
I create virtual machine ubuntu-master, ubuntu-dev1, ubuntu-dev2 and all of them have their own driver, and a shared driver with name shared-nas.
> sudo fdisk -l

Disk /dev/sda: 85.9 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders, total 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000645fe

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048   159582207    79790080   83  Linux
/dev/sda2       159584254   167770111     4092929    5  Extended
/dev/sda5       159584256   167770111     4092928   82  Linux swap / Solaris

Disk /dev/sdb: 85.9 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders, total 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdb doesn't contain a valid partition table

Solution:
> sudo mkfs -t ext3 /dev/sdb

It does not work.

Initiate the disk with following
> sudo fdisk /dev/sdb
>m  >n >p  >1
default for other information, then >w

It works.
> sudo fdisk -l

Disk /dev/sda: 85.9 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders, total 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000645fe

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048   159582207    79790080   83  Linux
/dev/sda2       159584254   167770111     4092929    5  Extended
/dev/sda5       159584256   167770111     4092928   82  Linux swap / Solaris

Disk /dev/sdb: 85.9 GB, 85899345920 bytes
86 heads, 10 sectors/track, 195083 cylinders, total 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x070ebcd0

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048   167772159    83885056   83  Linux

Format to ext3
> sudo mkfs -t ext3 /dev/sdb1

It works. Then I will configure the docker to use this share driver.

2 Auto Mount the Driver to System
Create the folder
>sudo mkdir /mnt/nas

List the UUID of my disk
> ls -l /dev/disk/by-uuid
total 0
lrwxrwxrwx 1 root root 10 Dec 11 17:45 6624adb3-86bc-4801-8275-063cdfae706b -> ../../sda5
lrwxrwxrwx 1 root root  9 Dec 11 17:45 75595e30-261b-4ee1-a855-4b4f5428d583 -> ../../sdb
lrwxrwxrwx 1 root root 10 Dec 11 17:45 8e864fd6-914f-489f-abf7-85744496c664 -> ../../sda1
lrwxrwxrwx 1 root root 10 Dec 11 17:45 adcfd350-219a-414f-92a6-e175b334d970 -> ../../sdb1

Copy the one of sdb1, edit the file fstab
> sudo vi /etc/fstab
UUID=adcfd350-219a-414f-92a6-e175b334d970 /mnt/nas        auto  defaults 0 0

Amount the disk
> sudo mount -a

Check the amount
> df -hl /mnt/nas/
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb1        79G   56M   75G   1% /mnt/nas

3 Install and Set up Docker
> sudo mkdir /mnt/nas/docker

Installation
> sudo curl -sSL https://get.docker.com/ | sh
> sudo usermod -aG docker carl
or
> sudo gpasswd -a carl docker

Change the configuration
> sudo vi /etc/default/docker
DOCKER_OPTS="-g /mnt/nas/docker"

> docker info
Containers: 0
Images: 0
Server Version: 1.9.1
Storage Driver: aufs
Root Dir: /mnt/nas/docker/aufs

Done and try
> docker pull centos:7
> df -hl
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        75G  1.3G   70G   2% /
none            4.0K     0  4.0K   0% /sys/fs/cgroup
udev            1.9G  4.0K  1.9G   1% /dev
tmpfs           386M  484K  386M   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            1.9G     0  1.9G   0% /run/shm
none            100M     0  100M   0% /run/user
/dev/sdb1        79G  242M   75G   1% /mnt/nas

File are stored in the shared NAS disk.

The problem is that I try to create a shared vdi and ask all the clients ubuntu system to load the same NAS disk. It does not work. I will try shared folder.
> sudo apt-get install virtualbox-guest-additions-iso
> sudo apt-get install virtualbox-guest-utils
> sudo mount -t vboxsf nas /mnt/nas

It works. But when I start the docker, docker throw exception in logging
> sudo less /var/log/upstart/docker.log
ESC[31mERROESC[0m[0000] [graphdriver] prior storage driver "aufs" failed: invalid argument
ESC[31mFATAESC[0m[0000] Error starting daemon: error initializing graphdriver: invalid argument

> VBoxManage modifymedium --type shareable nas.vdi

It works.

References:
https://help.ubuntu.com/community/InstallingANewHardDrive
http://tutorialforlinux.com/2014/08/01/ubuntu-how-to-automatically-mount-a-ext2-ext3-ext4-linux-partition-on-startup-easy-guide/
https://forums.docker.com/t/how-do-i-change-the-docker-image-installation-directory/1169
http://www.oracledistilled.com/virtualbox/creating-shared-drives-in-oracle-vm-virtualbox/
http://www.giannistsakiris.com/2008/04/09/virtualbox-access-windows-host-shared-folders-from-ubuntu-guest/
http://superuser.com/questions/745868/mount-failed-unknown-file-system-in-virtualbox-in-ubuntu-14-04-and-windows-7-a
http://stackoverflow.com/questions/30969435/where-is-the-docker-daemon-log

评论

1 楼 sillycat 2016-02-03  

sudo usermod -aG docker ec2-user