Empusa, A Lightweight Service Registry On Top Of etcd
Tags:
#etcd
Warning
Empusa has several limitations regarding accessing etcd clusters. Please refer to the project’s repository.
Warning
The main branch of empusa only supports etcd v2. To use empusa on top of etcd v3 with /v3alpha endpoint, refer to https://gitlab.com/testing-farm/empusa/-/merge_requests/8.
Introduction To etcd
etcd is a distributed key-value data store.
It is easily accessible using HTTP, and data is stored hierarchically, similar to a file system:
├── key1
│ ├── value1
│ └── value2
└── key2
├── value1
└── value2
etcd is a robust and stable tool used in many projects, including Kubernetes.
What Is A Service Registry?
A service registry is a data store containing data structures for applications to consume.
This is useful in cloud-native applications where various microservices need to access information.
A separate service registry allows each microservice to be designed to fill its role in the application while having a single source of truth that all of them use.
┌───────────────────┐
│ │
│ Microservice 1 ◄──────────┐
│ │ │
└───────────────────┘ ┌────▼───────────────┐
│ │
│ Service Registry │
│ │
┌───────────────────┐ └────▲───────────────┘
│ │ │
│ Microservice 2 ◄──────────┘
│ │
└───────────────────┘
Empusa
empusa is a tool built to enable a service registry-like approach on top of etcd. It is maintained by engineers working on Fedora and Red Hat Enterprise Linux CI.
It leverages the key-value data structure to build a trivial service registry.
empusa only supports etcd v2 natively (refer to merge request for etc v3).
Data Structure
Note
In etcd v3, data structure is different compared to v2, refer to https://etcd.io/docs/v3.3/rfc/.
All empusa data is hosted under a root key/directory (depending on the API version of etcd).
Depending on the entity type, empusa creates a key under the root key.
For example, if we create a service, the service
key will be populated under the /example
key. In the etcd v2 API, this will look like this:
# Query empusa service example directly from etcd
curl -s http://0.0.0.0:2379/v2/keys/example/service?recursive=true | jq .
# Output
{
"action": "get",
"node": {
"key": "/example/service",
"dir": true,
"nodes": [
{
"key": "/example/service/TEST",
"dir": true,
"nodes": [
{
"key": "/example/service/TEST/MY",
"value": "foo:123",
"modifiedIndex": 4,
"createdIndex": 4
}
],
"modifiedIndex": 4,
"createdIndex": 4
}
],
"modifiedIndex": 4,
"createdIndex": 4
}
}
# Query empusa service example using empusa
empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/example' service list --format='json' | jq .
# Output
[
{
"service": "TEST/MY",
"location": "foo:123",
"ttl": null
}
]
Available Entity Types
empusa has two entity types.
Services
An entity storing a mapping of the service name and its location:
+----------------+------------------+-------+
| Service | Location | TTL |
|----------------+------------------+-------|
| datastore/etcd | 10.0.77.101:2379 | |
+----------------+------------------+-------+
Services describe the available service in the directory. They have a TTL (time to live) that can be set to refresh or expire using empusa or external automation.
Switches
An entity storing a mapping of switch name and boolean value:
+----------+---------+-------+
| Switch | Value | TTL |
|----------+---------+-------|
| feat_1 | yes | |
| feat_2 | yes | |
+----------+---------+-------+
Switches describe an entity (like features) that can be toggled based on availability. They have a TTL (time to live) that can be set to refresh or expire using empusa or external automation.
Installing Empusa
empusa is hosted on PyPI and can be downloaded using pip
:
pip install empusa
Installing etcd
empusa supports etcd <= 3.3.27
.
etcd can be installed as a binary or using a container.
In this blog post, we will be using an etcd container:
# Create a temporary directory
mkdir /tmp/etcd-data
# Set directory permissions
chmod 0700 /tmp/etcd-data/
# Launch container in the foreground
docker run --rm \
-p 2379:2379 \
-p 2380:2380 \
--volume /tmp/etcd-data:/etcd-data:Z \
--name etcd-gcr-v3.3.27 \
gcr.io/etcd-development/etcd:v3.3.27 \
/usr/local/bin/etcd \
--name etcd-node-01 \
--data-dir /etcd-data \
--listen-client-urls http://0.0.0.0:2379 \
--advertise-client-urls http://0.0.0.0:2379 \
--listen-peer-urls http://0.0.0.0:2380 \
--initial-advertise-peer-urls http://0.0.0.0:2380 \
--initial-cluster etcd-node-01=http://0.0.0.0:2380 \
--initial-cluster-token tkn
Usage
After configuring etcd and installing empusa, a command line tool empusa
will be installed.
empusa
can parse specific arguments from the environment variables.
The following table describes which environment variables are parsed to which arguments in empusa
:
Environment Variable | empusa argument |
Example |
---|---|---|
EMPUSA_LOGLEVEL | INFO |
|
EMPUSA_ETCD_API_VERSION | –etcd-api-version | 2 |
EMPUSA_ETCD_ENDPOINT | –etcd-endpoint | 0.0.0.0:2379 |
EMPUSA_ETCD_PROTOCOL | –etcd-protocol | http |
EMPUSA_TREE_ROOT | –tree-root | /example |
EMPUSA_SERVICE_TYPE | –service-type | datastore |
EMPUSA_SERVICE_NAME | –service-name | etcd |
EMPUSA_SERVICE_LOCATION | –service-location | foo:12345 |
EMPUSA_SERVICE_TTL | –ttl | 300 |
EMPUSA_SERVICE_REFRESH_EVERY | –refresh-every | 180 |
EMPUSA_SWITCH_TTL | –ttl | 300 |
For detailed help, use the --help
argument.
Examples
Registering A Service
There are two ways to register a service:
-
Using environment variables:
EMPUSA_ETCD_ENDPOINT='0.0.0.0:2379' EMPUSA_TREE_ROOT='/available' EMPUSA_SERVICE_TYPE='datastore' EMPUSA_SERVICE_NAME='etcd' EMPUSA_SERVICE_LOCATION='0.0.0.0:2379' empusa service register # Output INFO:root:service etcd (datastore) registered
-
Using full command line arguments:
empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' service register --service-type="datastore" --service-name="etcd" --service-location="0.0.0.0:2379" # Output INFO:root:service etcd (datastore) registered
Unregestring A Service
There are two ways to unregister a service:
-
Using environment variables:
EMPUSA_ETCD_ENDPOINT='0.0.0.0:2379' EMPUSA_TREE_ROOT='/available' EMPUSA_SERVICE_TYPE='datastore' EMPUSA_SERVICE_NAME='etcd' EMPUSA_SERVICE_LOCATION='0.0.0.0:2379' empusa service unregister # Output INFO:root:service etcd (datastore) unregistered
-
Using full command line arguments:
empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' service unregister --service-type="datastore" --service-name="etcd" # Output INFO:root:service etcd (datastore) unregistered
Listing Services
There are two available formats to list services:
-
Table (default):
empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' service list --format='table' # Output +----------------+--------------+-------+ | Service | Location | TTL | |----------------+--------------+-------| | datastore/etcd | 0.0.0.0:2379 | | +----------------+--------------+-------+
-
JSON:
empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' service list --format='json' # Output [{"service": "datastore/etcd", "location": "0.0.0.0:2379", "ttl": null}]
Setting A Switch
(refer to previous examples to re-use environment variables if preferred)
Setting a switch (will create if it doesn’t exist or update if exists):
empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' switch set test true
# Output
INFO:root:switch test set to true
Removing A Switch
(refer to previous examples to re-use environment variables if preferred)
Removing a switch:
empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' switch remove test
# Output
INFO:root:switch test removed
Listing Switches
(refer to previous examples to re-use environment variables if preferred)
Listing switches:
empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' switch list
# Output
+----------+---------+-------+
| Switch | Value | TTL |
|----------+---------+-------|
| test | true | |
+----------+---------+-------+
Toggling A Switch
(refer to previous examples to re-use environment variables if preferred)
Toggle a switch, and flip the boolean value:
# Toggle switch
empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' switch toggle test
# Output
INFO:root:switch test set to no
# Toggle switch again
empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' switch toggle test
# Output
INFO:root:switch test set to yes
Get The Value Of A Switch
(refer to previous examples to re-use environment variables if preferred)
Get the value of a switch:
empusa --etcd-endpoint='0.0.0.0:2379' --tree-root='/available' switch get test
# Output
yes
Final Notes
In this blog post, we have discussed how empusa can enable you to build a lightweight service registry on top of a robust tool like etcd
.
How well it integrates into your architecture will depend on your requirements from a service registry.