Redis vs etcd with Golang

Photo by Clovis Wood Photography on Unsplash

Today we will compare Redis and etcd.
Let’s compare them in terms of performance. Consider their areas of application - cases.

Redis — in-memory data structure store, used as a database, cache, and message broker. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets, and sorted sets.

Etcd — is a strongly consistent, distributed key-value store that provides a reliable way to store data that needs to be accessed by a distributed system or cluster of machines. etcd gracefully handles master elections during network partitions and will tolerate machine failure, including the master.

Pros of Redis
- Performance
- In-memory cache
- Advanced key-value cache
- Open source
- Simplicity and ease-of-use

Pros of etcd
- Service discovery
- Fault tolerant key value store
- Bundled with coreos
- Open Source
- Privilege Access Management
- Secure

Redis has exelent documentaion https://redis.io/.
For quite a few years, Redis has been the go-to tool for developers all over the world.
Unlike other key-value, Redis data types include Strings, List, Sets, Stored Sets, Hashes, Bitmaps, HyperLogLogs, Streams, Geospatial.

Redis use cases are very variety:
- cache
- chat, messaging, and queues
- session store
- media streaming
- geospatial
- machine learning

etcd has modern documentation https://etcd.io/.
Out of the box etcd has HTTP API (tools). And your application can “watch” over specific keys or directories for changes and react to changes in values.

A lot of organizations use etcd to implement production systems
- all Kubernetes systems.
- OpenTable — internal service discovery and cluster configuration management.
- cycoresys.com — provides architecture and engineering for computing systems.
- Radius Intelligence — multiple internal tools, Kubernetes clusters, bootstrappable system configs.
- Vonage — kubernetes, vault backend, system configuration for microservices, scheduling, locks (future — service discovery).
- PD — PD(Placement Driver) is the central controller in the TiDB cluster.
- Huawei — System configuration for overlay network (Canal).
- Qiniu Cloud — system configuration for microservices, distributed locks.
- QingCloud — appcenter cluster for service discovery as metad backend.
- Yandex — system configuration for services, service discovery.
- Tencent Games — Meta data and configuration data for service discovery, Kubernetes, etc.
- Hyper.sh — Kubernetes, distributed locks, etc.
- Meitu — system configuration for services, service discovery, Kubernetes in a test environment.
- Grab — system configuration for services, service discovery.
- DaoCloud.io — container management.
- Branch.io — in Kubernetes systems.
- Baidu Waimai — SkyDNS, Kubernetes, UDC, CMDB, and other distributed systems.
- Salesforce.com — in Kubernetes systems.
- Hosted Graphite — Service discovery, locking, ephemeral application data.
- Transwarp — Transwarp Data Cloud, Transwarp Operating System, Transwarp Data Hub, Sophon.

Also, etcd is a CNCF project. This is really cool.

Etcd use cases:
- container Linux by CoreOS.
- Kubernetes systems.
- fast, distributed, fault tolerant and simple configuration system for all kinds of applications.

More about use cases and areas to use see the page https://etcd.io/docs/v3.3/learning/why/.

Let’s start with a basic Redis + Golang application.

Of course, you can easily start to use it locally or on your host machine.
For example, run Redis with docker locally in a couple of commands:

run Redis on your machine

Now connect to Redis from the Golang app. Create a really basic and simple golang app and connect to Redis host.

init go app

And the “hello-world” basic application itself.

cat ~/my-golang-redis-app/main.go

Check the operation of this application.

That’s all to get you started with Redis in Golang. Then you apply code organization practices and general architectural patterns adopted in your team.

And now about the basic etcd + Golang application.

Let’s talk about how to start etcd and golang application.

If you don’t already have etcd on your machine, let’s install it.
And check its work with a couple of commands. At the same time, let’s get acquainted directly with its cli.

Go to the official installation documentation page https://etcd.io/docs/v3.5/install/.

Make the contents of main.go match

These were concept-like applications.

Now checkout out how many times it will take on operations — write/read in the loop. See this loop on code below:

for i := 0; i < cnt; i++ {
 // write/read
}

Write and read in a loop

In the end, we see how long it is taken to execute 10 thousand operations write, and reads.
Redis shows the same time for both writing and reading — 7+ sec.
But etcd shows that it “writes” slowly, but reads much faster.
And if we compare only read operations, then etcd wins 5 times.

And of course, we are “obliged” to check how much time and memory is spent on the internal operation of the golang application — to conduct built-in benchmarks.

main_test.go

Benchmarks results

Here, as expected, we see a strong advantage in terms of resource consumption and time when writing to etcd.
At the same time, etcd became the leader in terms of recording.

What about limits?

What you should pay attention to first of all is, of course, the limits of Redis and etcd in terms of performance.
And they include:
- resources for bringing these technologies into the team;
- resources needed to maintain and restore (in case of failure) Redis and etcd for various environments (dev, stage, and production);
- also an integral part of the operation of any resource is their technical limitations
— throughput (performance) taking into account the accumulation of data
— and final limits on memory and data storage (per node/host and cluster as a whole).

Limits can be allocated for Redis:
- 1 Million small Keys -> String Value pairs use ~ 85MB of memory.
- 1 Million Keys -> Hash value, representing an object with 5 fields, use ~ 160 MB of memory.
- maximum number of keys a single Redis instance — 2³² keys.
In practice, you limit yourself to RAM.

Etcd limits:
- the maximum size (by default) of any request is 1.5 MiB (This limit is configurable through — max-request-bytes flag for etcd server).
- storage size — 8GB. But in practice, you can use, etc even with 16GB. And over 20GB you will catch timeouts.

Redis and etcd are of course key-value storage of a different class and differ in purpose, but they also have significant intersections.
First of all, it is key-value storage. And if you need to read much more from storage, then you should take a closer look at etcd.
And if you need to write/read often and plus you want more features, then you should take a closer look at Redis.

As for the current situation, most developers and architectures choose Redis. Firstly, this is a familiar tool. And it allows you to expand your functionality.
But at the same time, I met a couple of times proposals to use “non-traditional” Redis, namely etcd as key-value storage for configuration storage.

By the way, configuring the application is a big topic and it is worth highlighting a separate article on this. What do you think about it?

All source code available in this repo — https://github.com/romanitalian/redis_vs_etc_with_go.