Using Makefiles to build and publish docker containers

»

For years I have run most of the services I deploy on servers in Docker containers. The benefits are plethora, from spinning up new instances in a few minutes, to migrating quickly to new hardware without a hitch. But the shear number of containers can still make management a bit of a headache. My home server hosts over 50 containers alone!

When I got started with containers I did what lots of beginners do, and used Portainer to help manage my growing collection of Docker stacks. Portainer is a great platform. It provides an easy-to-use web interface, and I would recommend it to beginners. But using Portainer can be quite limiting and adds a dependency layer between you and Docker. It was not long before I wanted to have greater control over my docker infrastructure and leverage custom scripts and command line tools to stream line administration and deployment of stacks.

Enter make files

If you want to run or update a task, the make utility can be incredibly useful. Many programmers are familiar with make, and use it when compiling applications. Most open source projects use make to compile a final executable binary, which can then be installed using make install. Make is also commonly used to help manage, build, and deploy docker containers. Some docker images also use make to provide functionality inside running containers. The make utility requires a file, Makefile (or makefile), which defines set of tasks to be executed.

In this article I will introduce you to the Makefile I use to manage my docker stacks, and provide a few tips on how it can be expanded in useful ways.

What the Makefile does

The makefile below provides several features:

  • Simplifies and shortens commands for common Docker tasks
  • Reduces errors and typos when executing complex commands
  • Makes targeting containers in a stack for operations like exec or shell access easier
  • Gives my stacks access to a set of ‘global’ environment variables I use in multiple projects
  • Allows for easier implementation of new functionality with additional sub commands

Without further ado here it is:

#!/usr/bin/make
# Makefile readme (en): <https://www.gnu.org/software/make/manual/html_node/index.html#SEC_Contents>
include ../global.env
include .env
export

SHELL = /bin/bash
RUN_ARGS = -f docker-compose.yml
INTERACTIVE := $(shell [ -t 0 ] && echo 1)
ERROR_ONLY_FOR_HOST = @printf "\033[33mThis command is only for the host machine\033[39m\n"
ERROR_PROJECT_DOWN = @printf "\033[33m%s\033[0m\n" 'The ${PROJECT_NAME} project has not been started.'
ERROR_NO_SERVICE = @printf "\033[33m%s\033[0m\n" 'No service specified.'
ERROR_NOT_SERVICE = @printf "\033[33m%s\033[0m\n" '$(word 1, $(TARGET_ARGS)) is not a service.'

# combine environment variables and pass them to docker compose
GLOBAL_ENV_LIST != cat ../global.env | tr '\n' ' ' 
LOCAL_ENV_LIST != cat .env | tr '\n' ' ' 
ENV_LIST := $(GLOBAL_ENV_LIST)$(LOCAL_ENV_LIST)
SERVICES_LIST != $(GLOBAL_ENV_LIST) sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) config --services | tr '\n' ' '

# targets that support multiple arguments
SUPPORTED_TARGETS := up down restart down-up du pull logs exec shell console build update
# is first argument a supported target? 
SUPPORTS_ARGS := $(findstring $(firstword $(MAKECMDGOALS)), $(SUPPORTED_TARGETS))
# if it is ...
ifneq "$(SUPPORTS_ARGS)" ""
  TARGET_ARGS := $(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS))
  # make arguments do-nothing targets
  $(eval $(TARGET_ARGS):;@:)
endif

# check if make is running in a container
ifndef INSIDE_DOCKER_CONTAINER
  INSIDE_DOCKER_CONTAINER = 0
endif

# check if any any services in the project are started
ifeq ($(shell sudo docker compose ls | grep $(PROJECT_NAME)),)
  PROJECT_STATUS := down
else
  PROJECT_STATUS := up
endif

# check if the first argument is a valid service
ifeq ($(findstring $(word 1, $(TARGET_ARGS)), $(SERVICES_LIST)),)
  SERVICE_EXISTS := false
else
  SERVICE_EXISTS := true
endif

# check if the first argument is a started service
ifneq ($(TARGET_ARGS),)
SERVICE_NAME := $(word 1, $(TARGET_ARGS))
ifeq ($(shell sudo docker ps --format '{{.Names}}' | grep $(SERVICE_NAME)),)
  SERVICE_STATUS := down
else
  SERVICE_STATUS := up
endif
endif

.DEFAULT_GOAL := help
.PHONY: help

# output the help for each task. source: https://marmelab.com/blog/2016/02/29/auto-documented-makefile.html
help: ## show this help
  @printf "\033[33m%s:\033[0m\n" 'available commands'
  @awk 'BEGIN {FS = ":.*?## "} /^[A-Za-z0-9_-]+:.*?## / {printf "  \033[32m%-18s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST)

up: ## start all services or specify one
ifeq ($(INSIDE_DOCKER_CONTAINER), 0)
ifneq ($(TARGET_ARGS), )
ifeq ($(SERVICE_EXISTS), true)
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) up -d $(word 1, $(TARGET_ARGS))
else
  $(ERROR_NOT_SERVICE)
endif
else
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) up -d --remove-orphans
endif
else
  $(ERROR_ONLY_FOR_HOST)
endif

down: ## stop all services or specify one
ifeq ($(INSIDE_DOCKER_CONTAINER), 0)
ifeq ($(PROJECT_STATUS), up)
ifneq ($(TARGET_ARGS), )
ifeq ($(SERVICE_EXISTS), true)
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) down $(word 1, $(TARGET_ARGS))
else
  $(ERROR_NOT_SERVICE)
endif
else
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) down
endif
else
  $(ERROR_PROJECT_DOWN)
endif
else
  $(ERROR_ONLY_FOR_HOST)
endif

restart: ## restart all services or specify one
ifeq ($(INSIDE_DOCKER_CONTAINER), 0)
ifeq ($(PROJECT_STATUS), up)
ifneq ($(TARGET_ARGS), )
ifeq ($(SERVICE_EXISTS), true)
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) restart $(word 1, $(TARGET_ARGS))
else
  $(ERROR_NOT_SERVICE)
endif
else
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) restart
endif
else
  $(ERROR_PROJECT_DOWN)
endif
else
  $(ERROR_ONLY_FOR_HOST)
endif

down-up du: ## stop & start all services or specify one
ifeq ($(INSIDE_DOCKER_CONTAINER), 0)
ifeq ($(PROJECT_STATUS), up)
ifneq ($(TARGET_ARGS), )
ifeq ($(SERVICE_EXISTS), true)
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) down $(word 1, $(TARGET_ARGS))
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) up -d $(word 1, $(TARGET_ARGS))
else
  $(ERROR_NOT_SERVICE)
endif
else
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) down
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) up -d --remove-orphans
endif
else
  $(ERROR_PROJECT_DOWN)
endif
else
  $(ERROR_ONLY_FOR_HOST)
endif

pull: ## pull all services' images or specify one
ifeq ($(INSIDE_DOCKER_CONTAINER), 0)
ifneq ($(TARGET_ARGS), )
ifeq ($(SERVICE_EXISTS), true)
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) pull $(word 1, $(TARGET_ARGS))
else
  $(ERROR_NOT_SERVICE)
endif
else
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) pull
endif
else
  $(ERROR_ONLY_FOR_HOST)
endif

logs: ## tail project logs - list service names for only those logs 
ifeq ($(INSIDE_DOCKER_CONTAINER), 0)
ifeq ($(PROJECT_STATUS), up)
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) logs -f ${TARGET_ARGS}
else
  $(ERROR_PROJECT_DOWN)
endif
else
  $(ERROR_ONLY_FOR_HOST)
endif

shell console: ## start shell into container
ifeq ($(INSIDE_DOCKER_CONTAINER), 0)
ifeq ($(PROJECT_STATUS), up)
ifneq ($(TARGET_ARGS), )
ifeq ($(SERVICE_EXISTS), true)
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) exec $(OPTION_T) $(word 1, $(TARGET_ARGS)) sh
else
  $(ERROR_NOT_SERVICE)
endif
else
  $(ERROR_NO_SERVICE)
endif
else
  $(ERROR_PROJECT_DOWN)
endif
else
  $(ERROR_ONLY_FOR_HOST)
endif

exec: ## run a command in the app container
ifeq ($(INSIDE_DOCKER_CONTAINER), 0)
ifeq ($(PROJECT_STATUS), up)
ifneq ($(TARGET_ARGS), )
ifeq ($(SERVICE_EXISTS), true)
  @sudo -E docker compose -p $(PROJECT_NAME) exec $(OPTION_T) $(word 1, $(TARGET_ARGS)) sh -c "$(wordlist 2,$(words $(TARGET_ARGS)),$(TARGET_ARGS))"
else
  $(ERROR_NOT_SERVICE)
endif
else
  $(ERROR_NO_SERVICE)
endif
else
  $(ERROR_PROJECT_DOWN)
endif
else
  $(ERROR_ONLY_FOR_HOST)
endif

ps: ## containers status
ifeq ($(INSIDE_DOCKER_CONTAINER), 0)
ifeq ($(PROJECT_STATUS), up)
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) ps
else
  $(ERROR_PROJECT_DOWN)
endif
else
  $(ERROR_ONLY_FOR_HOST)
endif

build: ## build all images or specify a service
ifeq ($(INSIDE_DOCKER_CONTAINER), 0)
ifneq ($(TARGET_ARGS), )
ifeq ($(SERVICE_EXISTS), true)
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) build --pull $(word 1, $(TARGET_ARGS))
else
  $(ERROR_NOT_SERVICE)
endif
else
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) build --pull
endif
else
  $(ERROR_ONLY_FOR_HOST)
endif

update: ## update all containers or specify a service
ifeq ($(INSIDE_DOCKER_CONTAINER), 0)
ifeq ($(PROJECT_STATUS), up)
ifneq ($(TARGET_ARGS), )
ifeq ($(SERVICE_EXISTS), true)
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) pull $(word 1, $(TARGET_ARGS))
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) down $(word 1, $(TARGET_ARGS))
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) up -d --remove-orphans $(word 1, $(TARGET_ARGS))
else
  $(ERROR_NOT_SERVICE)
endif
else
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) pull
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) down
  @sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) up -d --remove-orphans
endif
else
  $(ERROR_PROJECT_DOWN)
endif
else
  $(ERROR_ONLY_FOR_HOST)
endif

The comments in code should help to explain what’s happening, but there are a few requirements to use the Makefile. Put two variables in .env that the script will use to name a reference the stack, and optionally identify the primary image in the stack.

PROJECT_NAME=<stack/project name>
PRIMARY_SERVICE_NAME=<optional name of image to target by default>

Then put any environment variables in ../global.env that you want to share with multiple projects. I typically share things like timezone settings, uid/gid, and health check preferences. Here are a few examples:

G_PUID=1000
G_PGID=1000
G_UMASK=0002
G_TIMEZONE=America/Los_Angeles

G_HOST_IP=<ip>
G_EMAIL=<email>

G_HEALTHCHECK_INTERVAL=60s
G_HEALTHCHECK_TIMEOUT=3s
G_HEALTHCHECK_START_PERIOD=5s
G_HEALTHCHECK_RETRIES=3

G_CLOUDFLARE_API_ACCOUNT=<account>
G_CLOUDFLARE_API_TOKEN=<token>
G_CLOUDFLARE_TLS_PROTOCOLS=tls1.3
G_CLOUDFLARE_ADDR=173.245.48.0/20\ 103.21.244.0/22\ 103.22.200.0/22\ 103.31.4.0/22\ 141.101.64.0/18\ 108.162.192.0/18\ 190.93.240.0/20\ 188.114.96.0/20\ 197.234.240.0/22\ 198.41.128.0/17\ 162.158.0.0/15\ 104.16.0.0/13\ 104.24.0.0/14\ 172.64.0.0/13\ 131.0.72.0/22

G_BASIC_AUTH_USER=<user>
G_BASIC_AUTH_PASS=<pass>

Custom build variables

In cases where I am building an image for a project it can be useful to pass custom variables to the Dockerfile. Doing that is easy with few modifications. As an example suppose your Makefile has a version variable for Alpine:

ARG ALP_VER=3.12
FROM alpine:$ALP_VER

To pass that using the make build command just requires a simple modification to the script.

@sudo -E docker compose ${RUN_ARGS} -p $(PROJECT_NAME) build --pull --build-arg ALP_VER=${alpver}

Now optionally including the variable passes it to the Makefile like this: make build alpver=3.11

Running the Makefile in docker container

You may have noticed lots of if statements with the variable INSIDE_DOCKER_CONTAINER throughout the script. They check to make sure the command is being executed on the host machine, but a command can be modified to only be accessible in a container as well. For images where I want to be able to frequently execute commands I will copy the Makefile at build time, and add sub commands for in-container execution.

Leave a Reply