Docker Workshop - Docker Compose

 

Docker Compose is a command line tool to create and manage multi-container applications in the Docker ecosystem with a single command.

Imagine you’re developing a web application that consists of a frontend, backend, and several microservices like payment and ordering. Each service will most likely be written in different programming languages. Thus, each service will require a Dockerfile and specific configuration to be able to create a container, and Docker Compose makes it possible to manage all of that without any complexity.

Install Docker-Compose

Get the latest release:

$ wget https://github.com/docker/compose/releases/download/1.29.2/docker-compose-Linux-x86_64 -O /usr/local/bin/docker-compose

$ chmod +x /usr/local/bin/docker-compose

Test it:

root@ubuntu-local:~# docker-compose version
docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.7.10
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019

Available Commands

The list of available commands:

Commands:
  build              Build or rebuild services
  config             Validate and view the Compose file
  create             Create services
  down               Stop and remove resources
  events             Receive real time events from containers
  exec               Execute a command in a running container
  help               Get help on a command
  images             List images
  kill               Kill containers
  logs               View output from containers
  pause              Pause services
  port               Print the public port for a port binding
  ps                 List containers
  pull               Pull service images
  push               Push service images
  restart            Restart services
  rm                 Remove stopped containers
  run                Run a one-off command
  scale              Set number of containers for a service
  start              Start services
  stop               Stop services
  top                Display the running processes
  unpause            Unpause services
  up                 Create and start containers
  version            Show version information and quit

There are 3 essentials commands to manage the life cycle of containers:

  • up - To create and start containers. Most commonly used with -d, to run containers in the background.
  • down - To stop and remove containers. This includes all resources: containers, networksm images, and volumes.
  • ps - To display the status of running containers. Helpful for troubleshooting and healthchecks.

Docker-Compose file

Like Docker requires a Dockerfile, docker-compose(by default) requires a docker-compose.yaml or .yml file.

In this file, you need to specify the version, at least one service, and optionally volumes, and networks.

  • version - Defines the version. This is mandatory.
  • services - Define containers that will be built and started.
  • volumes - Define volumes that will be mounted inside containers(in the services section)
  • networks - Define networks

Services

The services section will need two essential options when creating containers. You can build a container by specifying a Dockerfile or specify an Image.

Building an Image:

version: "3"
services:
  myserver:
    build:
      context: ./myserver
      dockerfile: Dockerfile-myserver

This tells Docker to:

  • Create a service or container named myserver
  • The context or the directory to locate the Dockerfile. In this case ./myserver
  • The Dockerfile is Dockerfile-myserver

You only need to specify the dockerfile if it’s named something different instead of just Dockerfile.

Using an Image

To use an image, you just define a service with the image option:

version: "3"
services:
  myserver:
    build: 
    context: ./myserver
    dockerfile: Dockerfile-myserver
    ports:
     - "88:80"
  database:
    image: mariadb
    ports:
      - "3306:3306"

This is the directory structure of the above configuration:

MacBook-Pro:dockerserver kavish$ pwd
/tmp/dockerserver
MacBook-Pro:dockerserver kavish$
MacBook-Pro:dockerserver kavish$ tree
.
└── myserver
    └── Dockerfile-myserver

1 directory, 1 file
MacBook-Pro:dockerserver kavish$

If you start docker-compose with the above config, it will create a network called dockerserver_default. It uses the current directory as the name of the network. Then both containers, myserver and database will join that network.

Exercise 5.01 - Getting started with Docker Compose

Web servers in containers require operational tasks before starting, such as configuration, file downloads, or dependency installations. With docker-compose, it is possible to define those operations as multi-container applications and run them with a single command. In this exercise, you will create a preparation container to generate static files, such as index.html files. Then, the server container will serve the static files, and it will be reachable from the host machine by the network configuration. You will also manage the life cycle of the application using various docker-compose commands.

To complete the exercise, execute the following steps:

  1. Create a folder with the name server-with-compose, and navigate into it:
root@ubuntu-local:~/Docker_Workshop# mkdir server-with-compose
root@ubuntu-local:~/Docker_Workshop# cd server-with-compose/
  1. Create a folder with the name init and navigate into it:
root@ubuntu-local:~/Docker_Workshop/server-with-compose# mkdir init
root@ubuntu-local:~/Docker_Workshop/server-with-compose# cd init
  1. Create a bash script with the following content and save it as prepare.sh
#!/usr/bin/env sh
rm /data/index.html
echo "<h1>Welcome from Docker Compose!</h1>" >> /data/index.html
  1. Create a Dockerfile with the following content:
FROM busybox

ADD prepare.sh /usr/bin/prepare.sh

RUN chmod +x /usr/bin/prepare.sh

ENTRYPOINT ["sh", "/usr/bin/prepare.sh"]
  1. Change to the parent folder with cd ../, and create a docker-compose.yaml file with the following content:
version: "3"
services:
  init:
    build:
      context: ./init
    volumes:
      - static:/data

  server:
    image: nginx
    volumes:
      - static:/usr/share/nginx/html
    ports:
      - "88:80"

volumes:
  static:

docker-compose will create a volume named static, and two services: init and server. The volume will be mounted on both containers, and server will expose port 80 via port 88 on your host.

  1. Start the application in detach mode
root@ubuntu-local:~/Docker_Workshop/server-with-compose# docker-compose up --detach                     h
Creating network "server-with-compose_default" with the default driver
Creating volume "server-with-compose_static" with default driver
Building init
Sending build context to Docker daemon  3.072kB
Step 1/4 : FROM busybox
latest: Pulling from library/busybox
92f8b3f0730f: Pull complete
Digest: sha256:b5fc1d7b2e4ea86a06b0cf88de915a2c43a99a00b6b3c0af731e5f4c07ae8eff
Status: Downloaded newer image for busybox:latest
 ---> d3cd072556c2
Step 2/4 : ADD prepare.sh /usr/bin/prepare.sh
 ---> 893147a1a516
Step 3/4 : RUN chmod +x /usr/bin/prepare.sh
 ---> Running in c634316f581c
Removing intermediate container c634316f581c
 ---> b5bb8e83cf56
Step 4/4 : ENTRYPOINT ["sh", "/usr/bin/prepare.sh"]
 ---> Running in 2d2830e6710c
Removing intermediate container 2d2830e6710c
 ---> 5b7ed0e9b6b0
Successfully built 5b7ed0e9b6b0
Successfully tagged server-with-compose_init:latest
WARNING: Image for service init was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating server-with-compose_server_1 ... done
Creating server-with-compose_init_1   ... done

The containers are running in the background. The server-with-compose_default network, and the server-with-compose_static volume were created. The init container will run prepare.sh and exits. The server container will still be running.

  1. Check the status with docker-compose ps.
root@ubuntu-local:~/Docker_Workshop/server-with-compose# docker-compose ps
            Name                          Command               State                 Ports
---------------------------------------------------------------------------------------------------------
server-with-compose_init_1     sh /usr/bin/prepare.sh           Exit 0
server-with-compose_server_1   /docker-entrypoint.sh ngin ...   Up       0.0.0.0:88->80/tcp,:::88->80/tcp

The init container exited successfully, and the server is up and running.

  1. View the web page:
root@ubuntu-local:~/Docker_Workshop/server-with-compose# curl -s http://localhost:88
<h1>Welcome from Docker Compose!</h1>

The volume data is located in:

/var/lib/docker/volumes/server-with-compose_static/_data/
  1. Stop and remove all resources with docker-compose down.
root@ubuntu-local:~/Docker_Workshop/server-with-compose# docker-compose down
Stopping server-with-compose_server_1 ... done
Removing server-with-compose_init_1   ... done
Removing server-with-compose_server_1 ... done
Removing network server-with-compose_default

The volume would still be available. To remove it in one go, you would run docker-compose down -v. To remove a volume manually, run docker volume ls to list all the available volumes, and then remove it with docker volume rm [volume_name]:

root@ubuntu-local:~# docker volume ls
DRIVER    VOLUME NAME
local     0eaf25e85b3dde5da507a5bdf9b077c58d39ee04b554a30b50e4978044b15aa7
local     3d6151babad8bce9176eebc711b82bc96e82eedc36942cff7f5e791729add1f5
local     server-with-compose_static
root@ubuntu-local:~#
root@ubuntu-local:~#
root@ubuntu-local:~# docker volume rm server-with-compose_static
server-with-compose_static

Environment Variables

Containers relies heavily on environment variables. The main advantage is that the ENVs can be changed without changing the source code of the application.

ENVs are set under the environment section inside a service. There are 3 ways to set them:

  • Inside the Compose file

  • By passing ENVs on the command line(to modify the default value that was set in the compose file or to pass a value for empty variables that was set in the compose file)

  • Using an environment or .env file(if the number of variables is high)

Here’s an example for the myserver service:

myserver:
  environment:
    - LOG_LEVEL=DEBUG
    - PORT=8484

Here’s another example for values that are not set:

myserver:
  environment:
    - HOSTNAME

For unset vars, you can leave it empty(it won’t produce an errors), or pass the value on the command line.

Stored variables in .env file, looks like this:

PORT=8484
GITHUB_TOKEN=001001001001

An env is set under the env_file section inside a service:

myserver:
  env_file:
   - myenvfile.env

All ENVs inside the file will be set inside the container.

Exercise 5.02 - Configuring Services with Docker Compose

Services in Docker Compose are configured by environment variables. In this exercise, you will create a Docker Compose application that is configured by different methods of setting variables. In a file called print.env, you will define two environment variables. In addition, you will create and configure one environment variable in the docker-compose.yaml file and pass one environment variable from the Terminal on the fly. You will see how four environment variables from different sources come together in your container.

  1. Start by create a folder named server-with-configuration and navigate into it:
mkdir server-with-configuration; cd !$
  1. Create a .env file named print.env with the following content:
ENV_FROM_ENV_FILE_1=HELLO
ENV_FROM_ENV_FILE_2=WORLD
  1. Create a docker-compose.yaml with the following content:
version: "3"
services:
  print:
    image: busybox
    command: sh -c "sleep 5 && env"
    env_file:
    - print.env
    environment:
    - ENV_FROM_COMPOSE_FILE=HELLO
    - ENV_FROM_SHELL
  1. Export ENV_FROM_SHELL on your docker host:
export ENV_FROM_SHELL=WORLD
  1. Start the application with docker-compose up
root@ubuntu-local:~/Docker_Workshop/5.02-server-with-configuration# docker-compose up
Creating network "502-server-with-configuration_default" with the default driver
Creating 502-server-with-configuration_print_1 ... done
Attaching to 502-server-with-configuration_print_1
print_1  | HOSTNAME=f2aca738bde6
print_1  | SHLVL=1
print_1  | HOME=/root
print_1  | PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
print_1  | ENV_FROM_ENV_FILE_1=HELLO
print_1  | ENV_FROM_ENV_FILE_2=WORLD
print_1  | ENV_FROM_COMPOSE_FILE=HELLO
print_1  | ENV_FROM_SHELL=WORLD
print_1  | PWD=/
502-server-with-configuration_print_1 exited with code 0

The ENV_FROM_SHELL declared on the host, is available inside the container.

Service Dependency

All containers by default are independent microservices. But you can make them depend on each other. For example, a Wordpress service depends on a MySQL database.

The depends_on option is used to achieve this. Here’s an example:

version: "3"
services:
  mysql:
    image: busybox
  test:
    image: busybox
    depends_on:
     - "first"
  wordpress:
   image: busybox
   depends_on:
    - "second"

Now the containers will start in the order of mysql, test, and wordpress. If the depends_on option was not used, the Wordpress container could have started first, and the whole stack will break.

Exercise 5.03 - Service Dependency with Docker Compose

Services in Docker Compose can be configured to depend on other services. In this exercise, you will create an application with four containers. The first three containers will run consecutively to create a static file that will be served by the fourth container.

  1. Create a folder named server-with-dependency and navigate into it :
mkdir server-with-dependency; cd !$
  1. Create a file with the name docker-compose.yaml and the following content:
version: "3"
services:
  clean:
    image: busybox
    command: "rm -rf /static/index.html"
    volumes:
     - ./static:/static
  init:
    image: busybox
    command: "sh -c 'echo This is from init container >> /static/index.html'"
    volumes:
     - ./static:/static
    depends_on:
     - "clean"
  pre:
    image: busybox
    command: "sh -c 'echo This is from pre container >> /static/index.html'"
    volumes:
     - ./static:/static
    depends_on:
     - "init"
  server:
    image: nginx
    volumes:
     - ./static:/usr/share/nginx/html
    ports:
     - "88:80"
    depends_on:
     - "pre"

I’m using ./static here. I’m keeping everything in the current directory.

  1. Move one directory up, and create a folder named static:
cd ../ ; mkdir static
  1. Start the app:
root@ubuntu-local:~/server-with-dependency# docker-compose up
Starting server-with-dependency_clean_1 ... done
Starting server-with-dependency_init_1  ... done
Starting server-with-dependency_pre_1   ... done
Starting server-with-dependency_server_1 ... done
Attaching to server-with-dependency_clean_1, server-with-dependency_init_1, server-with-dependency_pre_1, server-with-dependency_server_1
server_1  | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
server_1  | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
server-with-dependency_init_1 exited with code 0
server_1  | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
server_1  | 10-listen-on-ipv6-by-default.sh: info: IPv6 listen already enabled
server-with-dependency_clean_1 exited with code 0
server_1  | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
server_1  | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
server_1  | /docker-entrypoint.sh: Configuration complete; ready for start up
server-with-dependency_pre_1 exited with code 0

Now open a new terminal, and browse to http://localhost:88:

root@ubuntu-local:~# curl -s http://localhost:88
This is from init container
This is from pre container


root@ubuntu-local:~# curl -s http://localhost:88/index.php
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.19.8</center>
</body>
</html>
root@ubuntu-local:~#

If you look at the docker compose terminal, you’ll see the requests logs:

server_1  | 172.19.0.1 - - [14/Jun/2021:09:25:39 +0000] "GET / HTTP/1.1" 200 55 "-" "curl/7.68.0" "-"
server_1  | 172.19.0.1 - - [14/Jun/2021:09:25:42 +0000] "GET / HTTP/1.1" 200 55 "-" "curl/7.68.0" "-"
server_1  | 2021/06/14 09:25:47 [error] 24#24: *3 open() "/usr/share/nginx/html/index.php" failed (2: No such file or directory), client: 172.19.0.1, server: localhost, request: "GET /index.php HTTP/1.1", host: "localhost:88"
server_1  | 172.19.0.1 - - [14/Jun/2021:09:25:47 +0000] "GET /index.php HTTP/1.1" 404 153 "-" "curl/7.68.0" "-"

Press CTRL+C to exit:

^CGracefully stopping... (press Ctrl+C again to force)
Stopping server-with-dependency_server_1 ... done

Activity 5.01 - Installing WordPress Using Docker Compose

You are assigned to design and deploy a blog with its database as microservices in Docker. You will be using WordPress since it is the most popular Content Management System (CMS), used by more than one-third of all the websites on the internet.

Also, the development and testing teams require the installation of both WordPress and the database multiple times on different platforms with isolation. Therefore, you are required to design it as a Docker Compose application and manage it with the docker-compose CLI.

Perform the following steps to complete this activity:

  1. Start by creating a directory for your docker-compose.yaml file.

  2. Create a service for the database using MySQL and a volume defined in the docker-compose.yaml file. Ensure that the MYSQL_ROOT_PASSWORD, MYSQL_DATABASE, MYSQL_USER, and MYSQL_PASSWORD environment variables are set.

  3. Create a service for WordPress defined in the docker-compose.yaml file. Ensure that the WordPress containers start after the database. For the configuration of WordPress, do not forget to set the WORDPRESS_DB_HOST, WORDPRESS_DB_USER, WORDPRESS_DB_PASSWORD, and WORDPRESS_DB_NAME environment variables in accordance with step 2. In addition, you need to publish its port to be able to reach it from the browser.

  4. Start the Docker Compose application in detached mode. Upon successful deployment, you will have two containers running. Verify it with docker-compose ps.

The content of docker-compose.yaml:

version: "3"
services:
  database:
    image: mysql
    volumes:
    - ./data:/var/lib/mysql
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: root
      MYSQL_DATABASE: wpdb
      MYSQL_USER: wpuser
      MYSQL_PASSWORD: wppassword
  wordpress:
   depends_on:
    - database
   image: wordpress
   ports:
    - "88:80"
   restart: always
   environment:
     WORDPRESS_DB_HOST: database:3306
     WORDPRESS_DB_USER: wpuser
     WORDPRESS_DB_PASSWORD: wppassword
     WORDPRESS_DB_NAME: wpdb

Run docker-compose up, open a new terminal, browse to the directory of the configuration file, and run docker-compose ps:

root@ubuntu-local:~/wordpress-compose# docker-compose ps
            Name                           Command              State              Ports
----------------------------------------------------------------------------------------------------
wordpress-compose_database_1    docker-entrypoint.sh mysqld     Up      3306/tcp, 33060/tcp
wordpress-compose_wordpress_1   docker-entrypoint.sh apach      Up      0.0.0.0:88->80/tcp,:::88->80
                                ...                                     /tcp

Verify it with curl:

root@ubuntu-local:~/wordpress-compose# curl http://localhost:88/wp-admin/install.php -s | grep -i wordpress
  <title>WordPress &rsaquo; Installation</title>
<p id="logo">WordPress</p>

Restart Policy

The restart policy was set in the above docker-compose.yaml file. After a reboot, the Docker daemon will restart the containers even after you delete the compose file. To get around this, you can simply remove the containers, or modify the restart policy with the following command:

docker container update --restart=no [container_name_or_id]

If the compose file was deleted, docker-compose won’t work, because you have to be in the proper directory. docker-compose is responsible to create the containers, but not for restartting them. So, keep your compose file safe, or else you’ll have to remove everything manually.

The restart policy can also be set with docker run.

Activity 5.02 - Installing the Panoramic Trekking App Using Docker Compose

You are tasked with creating a deployment of the Panoramic Trekking App using Docker Compose. You will take advantage of the three-tier architecture of the Panoramic Trekking App and create a three-container Docker application, with containers for the database, the web backend, and nginx. Therefore, you will design it as a Docker Compose application and manage it with the docker-compose CLI.

Perform the following steps to complete this activity:

  1. Create a directory for your docker-compose.yaml file.

  2. Create a service for the database using PostgreSQL and a volume defined in the docker-compose.yaml file. Ensure that the POSTGRES_PASSWORD environment variable is set to docker. In addition, you need to create a db_data volume in docker- compose.yaml and mount it to the /var/lib/postgresql/data/ to store the database files.

  3. Create a service for the Panoramic Trekking App defined in the docker-compose.yaml file. Ensure that you are using the packtworkshops/the-docker-workshop:chapter5-pta-web Docker image, which is prebuilt and ready to use from the registry. In addition, since the application is dependent on the database, you should configure the container to start after the database. To store the static files, create a static_data volume in docker-compose.yaml and mount it to /service/static/.

Finally, create a service for nginx and ensure that you are using the packtworkshops/the-docker- workshop:chapter5-pta-nginx Docker image from the registry. Ensure that the nginx container starts after the Panoramic Trekking App container. You also need to mount the same static_data volume to the /service/static/ location. Do not forget to publish nginx port 80 to 8000 to reach from the browser.

  1. Start the Docker Compose application in detached mode. Upon successful deployment, you will have three containers running.

The content of docker-compose.yaml:

version: "3"
services:
  database:
    image: postgres
    environment:
      POSTGRES_PASSWORD: docker
    volumes:
     - ./db_data:/var/lib/postgresql/data
  panoramic:
    image: packtworkshops/the-docker-workshop:chapter5-pta-web
    volumes:
     - ./static_data:/service/static
    depends_on:
     - database
  web:
    image: packtworkshops/the-docker-workshop:chapter5-pta-nginx
    volumes:
     - ./static_data:/service/static
    depends_on:
     - panoramic
    ports:
     - 8000:80

Start the app in the background:

root@ubuntu-local:~/panoramic# docker-compose up -d
Starting panoramic_database_1 ... done
Starting panoramic_panoramic_1 ... done
Starting panoramic_web_1       ... done

Verify it with docker-compose ps:

root@ubuntu-local:~/panoramic# docker-compose ps
        Name                       Command               State                  Ports
----------------------------------------------------------------------------------------------------
panoramic_database_1    docker-entrypoint.sh postgres    Up      5432/tcp
panoramic_panoramic_1   ./entrypoint.sh gunicorn p ...   Up
panoramic_web_1         nginx -g daemon off;             Up      0.0.0.0:8000->80/tcp,:::8000->80/tcp

Browse to the app on(use IP instead of localhost):

root@ubuntu-local:~/panoramic# curl -s http://192.168.100.104:8000/admin | grep -i django > /dev/null; echo $?
0

The credentials are admin:changeme.