Published: January 14, 2023
Updated: January 15, 2023
Tags: empusa, etcd
5 min read

Empusa, A Lightweight Service Registry On Top Of etcd

Empusa has several limitations regarding accessing etcd clusters. Please refer to the project’s repository.

The main branch of empusa only supports etcd v2.
To use empusa on top of etcd v3 with /v3alpha endpoint, refer to https://gitlab.com/testing-farm/empusa/-/merge_requests/8.

Introduction To etcd

etcd [1] is a distributed key-value data store.
It is easily accessible using HTTP, and data is stored hierarchically, similar to a file system:

├── key1
│   ├── value1
│   └── value2
└── key2
    ├── value1
    └── value2

etcd is a robust and stable tool used in many projects, including Kubernetes [2].

What Is A Service Registry?

A service registry is a data store containing data structures for applications to consume.

This is useful in cloud-native applications where various microservices [3] need to access information.
A separate service registry allows each microservice to be designed to fill its role in the application while having a single source of truth that all of them use.

┌────────────────────────┐                                  ┌────────────────────────┐
│                        │                                  │                        │
│     Microservice 1     ◄─────────┐              ┌─────────►     Microservice 1     │
│                        │         │              │         │                        │
└────────────────────────┘    ┌────▼──────────────▼────┐    └────────────────────────┘
                              │                        │
                              │    Service Registry    │
                              │                        │
┌────────────────────────┐    └────▲──────────────▲────┘    ┌────────────────────────┐
│                        │         │              │         │                        │
│     Microservice 2     ◄─────────┘              └─────────►     Microservice 2     │
│                        │                                  │                        │
└────────────────────────┘                                  └────────────────────────┘

Empusa

empusa [4] is a tool built to enable a service registry-like approach on top of etcd. It is maintained by engineers working on Fedora and RHEL CI [5].
It leverages the key-value data structure to build a trivial service registry.

empusa only supports etcd v2 natively (refer to merge request [6]).

Data Structure

In etcd v3, data structure is different compared to v2, refer to https://etcd.io/docs/v3.3/rfc/.

All empusa data is hosted under a root key/directory (depending on the API version of etcd).
Depending on the entity type, empusa creates a key under the root key.
For example, if we create a service, the service key will be populated under the /example key. In the etcd v2 API, this will look like this:

curl -s http://0.0.0.0:2379/v2/keys/example/service?recursive=true | jq .
{
  "action": "get",
  "node": {
    "key": "/example/service",
    "dir": true,
    "nodes": [
      {
        "key": "/example/service/TEST",
        "dir": true,
        "nodes": [
          {
            "key": "/example/service/TEST/MY",
            "value": "foo:123",
            "modifiedIndex": 4,
            "createdIndex": 4
          }
        ],
        "modifiedIndex": 4,
        "createdIndex": 4
      }
    ],
    "modifiedIndex": 4,
    "createdIndex": 4
  }
}

empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/example' service list --format='json' | jq .
[
  {
    "service": "TEST/MY",
    "location": "foo:123",
    "ttl": null
  }
]

Available Entity Types

empusa has two entity types.

Services

An entity storing a mapping of the service name and its location:

+----------------+------------------+-------+
| Service        | Location         | TTL   |
|----------------+------------------+-------|
| datastore/etcd | 10.0.77.101:2379 |       |
+----------------+------------------+-------+

Services describe the available service in the directory. They have a TTL (time to live) that can be set to refresh or expire using empusa or external automation.

Switches

An entity storing a mapping of switch name and boolean value:

+----------+---------+-------+
| Switch   | Value   | TTL   |
|----------+---------+-------|
| feat_1   | yes     |       |
| feat_2   | yes     |       |
+----------+---------+-------+

Switches describe an entity (like features) that can be toggled based on availability. They have a TTL (time to live) that can be set to refresh or expire using empusa or external automation.

Installing Empusa

empusa is hosted on PyPI [7] and can be downloaded using pip:

pip install empusa

Installing etcd

empusa supports etcd >= 3.3.27.

etcd can be installed as a binary [8] or using a container.
In this blog post, we will be using an etcd container:

# Create a temporary directory
mkdir /tmp/etcd-data
# Set directory permissions
chmod 0700 /tmp/etcd-data/
# Launch container in the foreground
docker run --rm \
           -p 2379:2379 \
           -p 2380:2380 \
           --volume /tmp/etcd-data:/etcd-data:Z \
           --name etcd-gcr-v3.3.27 \
           gcr.io/etcd-development/etcd:v3.3.27 \
           /usr/local/bin/etcd \
           --name etcd-node-01 \
           --data-dir /etcd-data \
           --listen-client-urls http://0.0.0.0:2379 \
           --advertise-client-urls http://0.0.0.0:2379 \
           --listen-peer-urls http://0.0.0.0:2380 \
           --initial-advertise-peer-urls http://0.0.0.0:2380 \
           --initial-cluster etcd-node-01=http://0.0.0.0:2380 \
           --initial-cluster-token tkn

Usage

After configuring etcd and installing empusa, a command line tool empusa will be installed.
empusa can parse specific arguments from the environment variables.
The following table describes which environment variables are parsed to which arguments in empusa:

Environment Variable empusa argument Example
EMPUSA_LOGLEVEL INFO
EMPUSA_ETCD_API_VERSION --etcd-api-version 2
EMPUSA_ETCD_ENDPOINT --etcd-endpoint 0.0.0.0:2379
EMPUSA_ETCD_PROTOCOL --etcd-protocol http
EMPUSA_TREE_ROOT --tree-root /example
EMPUSA_SERVICE_TYPE --service-type datastore
EMPUSA_SERVICE_NAME --service-name etcd
EMPUSA_SERVICE_LOCATION --service-location foo:12345
EMPUSA_SERVICE_TTL --ttl 300
EMPUSA_SERVICE_REFRESH_EVERY --refresh-every 180
EMPUSA_SWITCH_TTL --ttl 300

For detailed help, use the --help argument.

Examples

Registering A Service

There are two ways to register a service:

  1. Using environment variables:
    EMPUSA_ETCD_ENDPOINT='0.0.0.0:2379' EMPUSA_TREE_ROOT='/available' EMPUSA_SERVICE_TYPE='datastore' EMPUSA_SERVICE_NAME='etcd' EMPUSA_SERVICE_LOCATION='0.0.0.0:2379' empusa service register
    INFO:root:service etcd (datastore) registered
    
  2. Using full command line arguments:
    empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' service register --service-type="datastore" --service-name="etcd" --service-location="0.0.0.0:2379"
    INFO:root:service etcd (datastore) registered
    

Unregestring A Service

There are two ways to unregister a service:

  1. Using environment variables:
    EMPUSA_ETCD_ENDPOINT='0.0.0.0:2379' EMPUSA_TREE_ROOT='/available' EMPUSA_SERVICE_TYPE='datastore' EMPUSA_SERVICE_NAME='etcd' EMPUSA_SERVICE_LOCATION='0.0.0.0:2379' empusa service unregister
    INFO:root:service etcd (datastore) unregistered
    
  2. Using full command line arguments:
    empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' service unregister --service-type="datastore" --service-name="etcd"
    INFO:root:service etcd (datastore) unregistered
    

Listing Services

There are two available formats to list services:

  1. Table (default):
    empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' service list --format='table'
    +----------------+--------------+-------+
    | Service        | Location     | TTL   |
    |----------------+--------------+-------|
    | datastore/etcd | 0.0.0.0:2379 |       |
    +----------------+--------------+-------+
    
  2. JSON:
    empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' service list --format='json'
    [{"service": "datastore/etcd", "location": "0.0.0.0:2379", "ttl": null}]
    

Setting A Switch

(refer to previous examples to re-use environment variables if preferred)

Setting a switch (will create if it doesn't exist or update if exists):

empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' switch set test true
INFO:root:switch test set to true

Removing A Switch

(refer to previous examples to re-use environment variables if preferred)

Removing a switch:

empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' switch remove test
INFO:root:switch test removed

Listing Switches

(refer to previous examples to re-use environment variables if preferred)

Listing switches:

empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' switch list
+----------+---------+-------+
| Switch   | Value   | TTL   |
|----------+---------+-------|
| test     | true    |       |
+----------+---------+-------+

Toggling A Switch

(refer to previous examples to re-use environment variables if preferred)

Toggle a switch, and flip the boolean value:

empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' switch toggle test
INFO:root:switch test set to no
empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' switch toggle test
INFO:root:switch test set to yes

Get The Value Of A Switch

(refer to previous examples to re-use environment variables if preferred)

Get the value of a switch:

empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' switch get test
yes

Final Notes

In this blog post, we have discussed how empusa can enable you to build a lightweight service registry on top of a robust tool like etcd.

How well it integrates into your architecture will depend on your requirements from a service registry.


  1. etcd homepage ↩︎

  2. Kubernetes homepage ↩︎

  3. microservices architecture blog ↩︎

  4. empusa Gitlab repository page ↩︎

  5. Testing Farm documentation ↩︎

  6. Gitlab - empusa: Add etcd API v3 Support Code ↩︎

  7. PyPI - empusa ↩︎

  8. etcd documentation v3.3 - Install ↩︎