Blackbeard is a namespace manager for Kubernetes. 👍 It helps you to develop and test with Kubernetes using namespaces.

Blackbeard helps you to deploy your Kubernetes manifests on multiple namespaces, making each of them running a different version of your microservices. You may use it to manage development environment (one namespace per developer) or for testing purpose (one namespace for each feature to test before deploying in production).

When working in a quite large team or, in a quite large project, have you ever experienced difficulties to test multiple features at the same time?

Usually, teams have 2 alternatives :

• Stack “features to test” in a queue and wait for the staging environment to be available;
• Try more or less successfully to create and maintain an “on demand” staging environment system, where each environment is dedicated to test a specified feature.

Blackbeard helps you to create ephemeral environments where you can test a set of features before pushing in production.

It also provide a very simple workflow, making things easier if you plan to run automated end-to-end testing.

Playbooks

Blackbeard use playbooks to manage namespaces. A playbook is a collection of kubernetes manifest describing your stack, written as templates. A playbook also require a default.json, providing the default values to apply to the templates.

Playbooks are created as files laid out in a particular directory tree.

The following section describe how to quickly start to work with Blackbeard.

A full example, using 2 applications (an API and a front web app), is available on github. Have a look at it and try it on a local Kubernetes, for example (using docker-or-desktop or mini-kube).

Recommanded

The simplest way of installing Blackbeard is to use the installation script :

curl -sf https://raw.githubusercontent.com/Meetic/blackbeard/master/install.sh | sh

Manually

Download your preferred flavor from the releases page and install manually.

Using Go Get

Note : this method requires Go 1.9+ and dep.

go get github.com/Meetic/blackbeard
cd $GOPATH/src/github.com/Meetic/blackbeard
make build

Blackbeard provide a CLI interface in addition to a REST API. This way, you can use it either to run automated tests in a CI pipeline or plug your own UI for manuel deployment purpose.

Creating a new isolated env

blackbeard create -n my-feature

This command actually create a namespace and generate a JSON configuration file containing default values. This file, called an inventory, is where you may update the values to apply specifically to your new namespace (such as microservice version)

Applying changes

blackbeard apply -n my-feature

This command apply your kubernetes manifest (modified by the values you have put in the generated inventory previously) to your newly created namespace

Getting services endpoint / ports

blackbeard get services -n my-feature

This step will prompt a list of exposed services in the namespace. If you need to connect to a database in order to test data insertion, it is where you will find useful info.

Getting back to previous state

blackbeard delete -n my-feature

Delete all generated files and delete the namespace.

Working with Blackbeard actually means working with a playbook. Blackbeard manage Kubernetes namespaces based on a playbook.

First thing, is to create your own playbook. Once done, you have 2 ways to manage your namespaces using blackbeard :

• Using CLI
• Using the REST API

Blackbeard provide a set of CLI command.

CLI usage may be use for different purpose.

Locally, for developers, it is useful for managing multiple namespace either to develop a microservice calling other applications running a specified version, or to test the entire stack using specified version of multiple services in the stack.

In a CI/CD pipeline, for automated end-to-end testing.

Create a new env

cd {your playbook}
blackbeard create -n {namespace name}

• create a Kubernetes namespace;
• generate a inventory file for the newly created namespace;
• generate a set of yml manifest based on the playbook templates.

Update values & apply changes

cd {your playbook}/inventories
## edit the inventory file you want to update
cd ..
blackbeard apply -n {namespace name}

• apply values defined in the inventory file to the playbook templates;
• update the yml manifest using the newly updated values from the inventory file;
• run a kubectl apply command and apply changes in the manifest to the namespace.

List namespaces

blackbeard get namespaces

• prompt a list of available Kubernetes namespace;

for each namespace :

• indicate if the namespace is managed by a local inventory or not.
• indicate the status of the namespace (aka : percentage of pods in a “running” state)

Exemple :

Namespace	Phase	Status	Managed
backend		Active	100%	false
john    	Active	73% 	true
default		Active	0%    false
kevin   	Active	73%   false
team1	   	Active	73%   true

Get useful informations about services

blackbeard get services -n my-feature

• prompt a list of exposed services

Get Help

Usage:
  blackbeard [command]

Available Commands:
  apply       Apply a given inventory to the associated namespace
  create      Create a namespace and generated a dedicated inventory.
  delete      Delete a namespace
  get         Show informations about a given namespace.
  help        Help about any command
  reset       Reset a namespace based on the template files and the default inventory.
  serve       Launch the blackbeard server

Flags:
      --config string   config file (default is $HOME/.blackbeard.yaml)
      --dir string      Use the specified dir as root path to execute commands. Default is the current dir.
  -h, --help            help for blackbeard

Use "blackbeard [command] --help" for more information about a command.

Blackbeard also provide a web server and a websocket server exposing a REST api.

You can launch the Blackbeard server using the command :

blackbeard serve --help

Usage:
  blackbeard serve [flags]

Flags:
      --cors   Enable cors
  -h, --help   help for serve

Global Flags:
      --config string   config file (default is $HOME/.blackbeard.yaml)
      --dir string      Use the specified dir as root path to execute commands. Default is the current dir.

The REST api documentation is written following the OpenAPI specifications.

This documentation is available in an HTML format, using Swagger UI.

Blackbeard uses Playbooks to manage technical stack deployments across multiple namespaces.

A Playbook is organized as a collection of files inside of a directory. Inside of this directory, Blackbeard will expect a structure that matches this:

• A templates directory, containing Kubernetes manifests, written as templates. Those are typical configuration files for K8s (yaml).
• A defaults.json file, defining the default values to apply (to the manifest templates)
• An inventories directory that will contains the future inventories (One per namespace). The content of this directory should not be versioned. Inventories are variance of the defaults.json file.
• A configs directory that will contains the future manifests files (one sub-dir per namespace). The content of this directory should not be versioned as well. Manifests are generated by applying the inventory values to the template

By default, Blackbeard will try to use the current directory as a Playbook. You can also specify a default playbook using a configuration file.

Thus, a Playbook made for an application called “MyApp” will look like :

MyApp
├── README.md
├── configs                                     #Directory containing generated manifest for each inventory
│   ├── john                                    #john is a namespace
│   │   ├── api.yml                             #Kubernetes manifest containing deployments and services for the api app.
│   │   └── front.yml                           #Kubernetes manifest containing deployments and services for the front-end app.
│   └── awesome-feature-to-test                 #awesome-feature-to-test is a namespace
│       ├── api.yml
│       └── front.yml
├── defaults.json                               #Default inventory file
├── inventories                                 #Directory containing inventories for each namespace
│   ├── john_inventory.json                     #inventory for john namespace
│   └── awesome-feature-to-test_inventory.json  #inventory for awesome-feature-to-test namespace
└── templates                                   #Directory containing the Kubernetes manifest templates
    ├── api.yml.tpl
    └── front.yml.tpl

Playbooks expect templates and a default inventory.

Templates are very simple. The only constraints are :

• Templates must contains a valid Kubernetes manifest (yaml)
• Template files must have a .tpl extension

Example : api.yml.tpl

---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: api
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
        - name: api
          image:  myCompany/MyApp:{{.Values.api.version}}
          args: ["-Xms{{.Values.api.memoryLimit}}", "-Xmx{{.Values.api.memoryLimit}}", "-Dconfig.resource=config.conf"]
          imagePullPolicy: Always
---
kind: Service
apiVersion: v1
metadata:
  name: api
spec:
  selector:
    app: api
  ports:
    - protocol: TCP
      port: 8080

An inventory is a file containing a json object. This object is made available inside the templates by Blackbeard.

The only constraints on an inventory json object are :

• An inventory must contains a namespace key, containing a string. The string value must be the name of the namespace the inventory is associated to;
• An inventory must contains a values key. This key may contains whatever you want.
• An inventory file must be located in the inventories directory
• An inventory file must be named after the template : {{namespace}}_inventory.json

Example : john_inventory.json

{
    "namespace": "john",
    "values": {
        "api": {
            "version": "1.2.0",
            "memoryLimit": "128m"
        },
        "front": {
            "version": "1.2.1"
        }
    }
}

Inventories are generated from the defaults.json file. Blackeard copy the defaults.json file content, create a inventory for the given namespace (located in the inventories directory), past the content default values and change the namespace key value with the corresponding namespace

The defaults.json file is the default inventory. It contains defaults values to apply on the templates. Blackbeard uses it to generated per namespace inventory.

The only constraint on the default.json file is :

• Must contains a namespace key, containing the value “default”.

Example : defaults.json

{
    "namespace": "default",
    "values": {
        "api": {
            "version": "1.0.0",
            "memoryLimit": "128m"
        },
        "front": {
            "version": "1.0.0"
        }
    }
}