Docker + RexRay + Virtualbox

*EDIT*

A better tutorial is here: https://github.com/cloud-coder/docker-lab-2016/tree/master/part-3

This post is about getting RexRay to start successfully inside a Docker container in order to create a Docker volume service for a Swarm.

The setup:

• I'm very new to Docker
• RexRay with libstorage using Virtualbox as service
• Have all VMs, and Swarm running on my local machine (desktop)
• Have been following guides/tutorials from the following resources:
• Use-docker-swarm-with-a-data-persistence-layer
• Docker 1.12.1 Swarm Mode & Persistent Storage with DellEMC RexRay on AWS Platform
• RexRay configuration doc
• RexRay User Guide
• Libstorage doc

The Objective:

• To test a setup for shared volume for a Docker Swarm

The problem:

• Everytime I try to run RexRay, I'm getting "default module(s) failed to initialize" error message. e.g.

Solution:

• Must have VirtualBox web service running beforehand.
• For some reasons, this crucial step is missing from most of the tutorials I saw ...

Details 

System/Parts/Components:

• Host OS: Ubuntu 16.04 64-bit
• Docker: 1.12
• VirtualBox: 5.1.10
• With Extension Pack: VirtualBox_Extension_Pack-5.1.10-112026
• RexRay: semVer: 0.6.0, Branch: v0.6.0
• libStorage: semVer: 0.3.1, Branch: v0.6.0 (automatically installed with RexRay installation, I believe)

Setting Up Docker Swarm

In order to create a Swarm, Docker needs at least 3 hosts (virtual machines). In case you're a complete beginner like me, here's how to create, use, and manage a Docker host inside a VM using docker-machine.

To create the Docker Swarm, I followed the Docker Swarm getting-started guide.

• Note: the Docker Swarm guide uses three VMs: manager1, worker1, and worker2. Use the docker-machine guide to make them.

Installing RexRay

To install RexRay, execute the following script in Terminal/Shell:

$ docker-machine ssh $machine "curl -sSL https://dl.bintray.com/emccode/rexray/install | sh -"

where $machine is the name of the machine. For example, for manager1:

$ docker-machine ssh manager1 "curl -sSL https://dl.bintray.com/emccode/rexray/install | sh -"

RexRay Configuration

Some of the tutorials I followed skipped through the RexRay configuration file, which results in this error when attempting to run RexRay:

To configure RexRay, you must create a config.yml file and put it in the /etc/rexray directory inside the VM. This must be done on all VMs since they must have RexRay running. For more info on RexRay configuration, see RexRay configuration doc.

1. Ssh to the VM (e.g. manager1):
   
   $ docker-machine ssh manager1
2. Edit the config file (must be root):
   
docker@manager1:~$ sudo vi /etc/rexray/config.yml

OR
docker@manager1:~$ sudo su

root@manager1:/home/docker# vi /etc/rexray/config.yml
3. Enter your configuration:
     libstorage:
       service: virtualbox
     virtualbox:
       endpoint: http://:18083
       volumePath: /home/VirtualBox/Volumes
       controllerName: SATA
4. Note:
1. is the IP address of the host OS (not the VM's). To check, open a new Terminal and enter:
   
   $ ifconfig
   
   If your machine is connected to the internet, the IP address will be under 'eth0' interface (if wired/ethernet) or 'wlan0' (if through wifi). Otherwise, the IP address is under 'lo' interface, which typically is: 127.0.0.1.
2. The port 18083 is the port number of the VirtualBox Web Service, which we will set up in the next section.
3. The volumePath should be a directory on your host OS. In my case, I manually created the /VirtualBox/Volumes directory. e.g.:
   
   $ cd ~
   $ mkdir -p VirtualBox/Volumes
5. Save the file.

At this point, if you try to run RexRay (from inside the VM) using this command:

docker@manager1:~$ sudo rexray start -f -l debug
 
It will fail to run because the VirtualBox Web Service is not yet running. Let's do that next.

Running VirtualBox Web Service

First, make sure the VirtualBox Extenstions package has been installed. A simple way to check is to run this on the Terminal:

$ which vboxwebsrv

If nothing comes up, then the Extensions package hasn't been installed. If so, follow this link to install it.
Edit: sounds like vboxwebsrv executable should come with the VirtualBox installation by default.

First disable authentication for the web service for testing purposes:

$ VBoxManage setproperty websrvauthlibrary null

To run the VirtualBox Web Service, open a new Terminal and run the following script on the Terminal:

$ vboxwebsrv -H 0.0.0.0 -v

It's important to run it NOT as root - otherwise, RexRay wouldn't work (or it'll need some additional authorization in the configuration, which I currently don't know).

If successful, you should see something like below:

Now we're ready to run RexRay.

Side note: I could not find a basic/simple tutorials on using VirtualBox Web Service. But here's the link to the .pdf document from virtualbox.org, specifically for the Programming Guide and Reference where the discussion on disabling the web service authentication is around page 26 (p.s. let me know if Oracle doesn't want people direct-linking their docs. But honestly, their website is not making it easy to find the docs, at least not by today's standards).

Running RexRay

To run RexRay:

1. Ssh to the VM (if not already):
   
   $ docker-machine ssh manager1
2. Run the following command:
   
   docker@manager1:~$ sudo rexray start

If successful, you should get the following output:

But Not Quite ...

I have to end this post without getting into the actual setup/demonstration of a working shared volume for a Docker Swarm using RexRay. In this post I just want to highlight the steps in getting RexRay up-and-running that were missing from most of the tutorials I found, and I had to spend quite some time to figure it out. In particular, the missing steps are: 1) configuring RexRay for local deployment, and 2) running the VirtualBox Web Service.

My real breakthrough was finding this Github RexRay thread. Thanks to those guys for making things a bit clearer.

Although RexRay seem to start successfully, when I run with the debug option, there were some errors in the log message which you can see below. I'm still looking into it.

I will try to post a Part 2 in the near future that covers eliminating the log errors and a working Docker Swarm shared volume.

p.p.s. I might move this blog to another platform that's more Markdown-friendly...