Running MongoDB on docker with compose and data volumes

Dockerizing MongoDB

As best practices it is not good to run databases in a single container since it limits the flexibility due to the container not being ephemeral anymore. Usually mounting a directory from the host to the container and persisting the data there solves that issue. However, with Boot2Docker on the mac there is an existent bug which makes MongoDB crash when mounting a volume from the host, specifically because MongoDB uses memory mapped files and it is not possible to use it through vboxsf to your host vbox bug.

The solution was to create a data volume and persist the data there instead. This way we still gain the flexibility of making the mongo container ephemeral since even if the container is destroyed the data wouldn’t be lost since is saved into a data volume.

An example where that would be handy follows below:

  • Run a mongo container
  • Configure users and roles
  • Need to modify some run time settings

The only way to accomplish that would be to destroy the container which would make the users and roles be lost as well as any other saved data. If instead the data is persisted to a data volume, the container can be destroyed as many times as wanted since is doesn’t hold any data.

Steps of running mongo with a data volume

  • Create data volume: docker create --name mongo-data-volume -v /data/db mongo:3.0.4 /bin/true
    • by using the same image for the data volume as for the mongo container it saves disk space since the file system layers are shared across them
  • Run mongo container: docker run -d --name mongo -p 27017:27017 --volumes-from mongo-data-volume mongo:3.0.4 --smallfiles --rest --auth
    • when the smallfiles flag is not set the exception “Insufficient free space for journal files” is raised

In the end the docker-compose file looked something like this

  image: mysql:5.6.24
    - "3306:3306"
    - MYSQL_ROOT_PASSWORD=password
    - MYSQL_USER=user
    - MYSQL_PASSWORD=password
    - MYSQL_DATABASE=sphereds
  image: memcached:1.4.24
    - "11211:11211"
  image: mongo:3.0.4
    - /data/db
  command: --break-mongo
  image: mongo:3.0.4
    - mongodata
    - "27017:27017"
  command: --smallfiles --rest --auth
  • Note that for the mongodata container we had to set a –break-mongo flag since docker-compose tries to run all the containers. However, with a data-volume we only want to create the container, not run it. This seems to be an existing issue with docker-compose. The workaround is to start the mongo deamon with a non existent flag which it breaks its execution and leaves the container in a stop state.

Configuring users

  • Two users will be created, system and cdynamics. Exosphere will use the cdyanmics user to establish a connection to mongo. System user has both root and restore role on the admin database. Cdynamics user has root role on the admin database and readWrite on the catalog database
  • mongo –host < ~/cdos/build-scripts/app/create-mongo-users.js
    • it is important to switch to the database that the users will be authenticating with since by default if no database is selected the commands run under the test database scope ``` use admin; db.createUser({ user: “system”, pwd: “password”, roles: [{ role: “root”, db: “admin” }, { role: “restore”, db: “admin” } ] });

use voyager; db.createUser({ user: “cdynamics”, pwd: “password”, “roles”: [ { “role”: “readWrite”, “db”: “voyager” } ] });

**Authentication workaround**

The official mongo documentation states that ```"The localhost exception allows you to enable authorization before creating the first user in the system. When active, the localhost exception allows connections from the localhost interface to create the first user on the admin database. The exception applies only when there are no users created in the MongoDB instance."```
However, I kept getting this exception when trying to create the first user on mongo: ```2015-07-08T18:43:57.320-0400 E QUERY    Error: couldn't add user: not authorized on admin to execute command { createUser: "system", pwd: "xxx", roles: [ { role: "root", db: "admin" }, { role: "restore", db: "admin" } ], digestPassword: false, writeConcern: { w: "majority", wtimeout: 30000.0 } } at Error (<anonymous>)    at DB.createUser (src/mongo/shell/db.js:1096:11)    at (shell):1:4 at src/mongo/shell/db.js:1096```

The work around was to first start the mongo container without authentication enabled and after creating the users restart the container with auth enabled

**Authentication mechanisms**

Initially when using [Jongo]( an authentication mechanism of MONGODB-CR type was being used. However, after enabling authentication on the mongo database we started to get an exception of ```"conn43] Failed to authenticate [email protected] with mechanism MONGODB-CR: AuthenticationFailed key mismatch"``` After checking which authentication mechanism the mongo CLI was using we discovered that instead of MONGODB-CR the default mechanism was of SCRAM-SHA-1. To solve the issue we had to update the authentication mechanism used by Jongo to SCRAM-SHA-1

MongoCredential credential = MongoCredential.createScramSha1Credential(“admin”, “database”, “password”); ```

Some extra notes

  • Using official docker image: version 2.8.0
  • Make sure there is a hosts entry for the domain and the boot2docker IP:
  • Running the container without compose: docker run -d --name mongo -p 27017:27017 --volumes-from mongo-data-volume mongo:3.0.4 --smallfiles --rest --auth
  • Starting compose: docker-compose up -d
  • Login to mongo through the CLI: mongo voyager --username cdynamics --password password --host

MongoDB GUI for mac

  • Installing with brew
    • brew update
    • brew upgrade mongo
    • installed version: MongoDB shell version: 3.0.4
  • Github repo, there are a few other repos hosting the code for MongoHub-Mac, make sure you get it from jeromelebel.
  • Created 3 connections
    • no user to admin database
    • system user with no database
    • cdynamics user with no database