Introducing Amazon MemoryDB for Redis, which adds durability to Redis. If the server-side framework is fast, it can process 100,000 transactions per second.

https://dev.classmethod.jp/articles/aws-release-durable-redis-amazon-memorydb-for-redis/

 

must-see articles

Amazon MemoryDB for Redis is here, adding durability to Redis

• #Amazon MemoryDB for Redis
• #Redis
• #Amazon ElastiCache
• #AWS

irbbb

2021.08.20

541

132

81

With the popular in-memory database Redis, reads and writes can be processed in sub-milliseconds, making it a great cache layer for performance-critical systems. On the other hand, although persistence mechanisms such as AOF files exist, they have the problem of data loss if the primary node fails, making them unsuitable for permanent data stores.

The same is true for Amazon ElastiCache for Redis provided by AWS.

The solution to this problem is Amazon MemoryDB for Redis announced this time   .

Introducing Amazon MemoryDB for Redis – A Redis-Compatible, Durable, In-Memory Database Service | AWS News Blog

Transaction logs are stored in multiple AZs and support consistent reads as only successfully written data can be read. No data loss in the event of node failure or failover.

*The  figure is from the official page

Features of Amazon MemoryDB for Redis

• Redis compatible API
• The transaction log at the time of writing is distributed to multiple AZs. No data loss
• Sub-millisecond read latency and single-digit millisecond write latency (regular Redis writes are also sub-millisecond)
• 470K reads/sec, 100K writes/sec
• Throughput of 1GB/s read and 100 MB/s write

I tried

Try to start Amazon MemoryDB For Redis.

Very importantly, it is currently not available in the Tokyo region and is only available in the following regions:

• northern virginia
• Dublin
• Mumbai
• sao paulo

From the AWS console, navigate to "Amazon MemoryDB For Redis".

Create a cluster from "Create cluster".

Building a cluster is very similar to ElastiCache.

Since MemoryDB only has what ElastiCache calls "cluster mode", specify the number of shards and replicas.

Once the cluster status is "Available", connect to the "Configuration endpoint" endpoint from your Redis client.

This time I installed Redis CLI on EC2 and connected.

$ sudo amazon-linux-extras install redis6
$ HOST=foo.xxx.clustercfg.memorydb.eu-west-1.amazonaws.com
$ redis-cli -h $HOST -c
172.31.10.170:6379> ping
pong
172.31.10.170:6379> set foo bar
172.31.10.170:6379> get foo
bar

Benchmarking ElastiCache and MemoryDB

We built a cluster (r6g.large) with 2 shards x 3 nodes/shards = a total of 6 nodes (r6g.large) with ElastiCache/MemoryDB,  redis-benchmark and performed a simple benchmark from within EC2 (t3.large) ( $ redis-benchmark -q -n 100000 -h $HOST --cluster).

As per the docs , ElastiCache is fast with less than 1msec read/write, while MemoryDB is very slow with writes.

Not all write-type commands are slow, some, such as set-type ( SADD/ ZADDetc.) commands, work fast. It's interesting that it's not good at  list operations ( LPUSH/ etc.),  and even simple commands like incrementing are slow.LPOPINCR

node type

Currently, only the r6g variant of memory-optimized Graviton2 is provided.

node type	vCPUs	Memory (GiB)	Network Performance (Gbps)
db.r6g.large	2	13.07	~Ten
db.r6g.xlarge	Four	26.32	~Ten
db.r6g.2xlarge	8	52.82	~Ten
db.r6g.4xlarge	16	105.81	~Ten
db.r6g.8xlarge	32	209.55	12
db.r6g.12xlarge	48	317.77	20
db.r6g.16xlarge	64	419.09	twenty five

The node type prefix is ​​"db", same as RDS/Aurora. It's not ElastiCache's "cache".

usage fee

The usage fee for MemoryDB is

• instance
• Write amount ($0.20/GB) *Fixed for all regions

This is a double feature.

Compare on-demand usage costs for N. Virginia between MemoryDB/ElastiCache/Aurora.

instance type	service	$/Hour
cache.r6g.large	Elasticache	0.206
db.r6g.large	Aurora	0.260
db.r6g.large	MemoryDB	0.309

MemoryDB is about 1.5 times more expensive than ElastiCache and about 1.2 times more expensive than Aurora.

Note that MemoryDB, which emphasizes durability, only has a "cluster mode" in ElastiCache, so you will be charged {number of shards} x {number of nodes/shards} x {instance usage fee}. .

At the end

Amazon MemoryDB for Redis, which can use Redis as a persistent data store, was born.

Reads are just as fast as Redis, with slower writes as a trade-off for data persistence.

The client just throws a Redis command and MemoryDB does something good, so it seems to be easy to use. In the future, there may be more cases such as considering it as an alternative to DynamodB + DAX, or consolidating RDB + cache Redis systems into MemoryDB.

From the naming of Amazon MemoryDB for Redis, the MemoryDB series lineup is expected to increase in the future. I'm curious as to what the next database engine will be.

Well then.

reference

• Introducing Amazon MemoryDB for Redis – A Redis-Compatible, Durable, In-Memory Database Service | AWS News Blog
• Amazon MemoryDB for Redis – Redis-Compatible In-Memory Database – Amazon Web Services

appendix

redis-benchmark benchmark result.

ElastiCache

$ redis-benchmark -q -n 100000 -h $EC --cluster
Cluster has 2 master nodes:

Master 0: 1d65e0ae87105d66acb8e619eb3f736a842a89f4 172.31.18.117:6379
ERROR: ERR unknown command `CONFIG`, with args beginning with: `GET`, `save`,
ERROR: failed to fetch CONFIG from 172.31.18.117:6379
WARN: could not fetch node CONFIG 172.31.18.117:6379
Master 1: de0e23d31a3a380ce0ae462c8ee315360dedea1a 172.31.8.77:6379
ERROR: ERR unknown command `CONFIG`, with args beginning with: `GET`, `save`,
ERROR: failed to fetch CONFIG from 172.31.8.77:6379
WARN: could not fetch node CONFIG 172.31.8.77:6379

PING_INLINE: 79681.27 requests per second, p50=0.271 msec
PING_MBULK: 79428.12 requests per second, p50=0.455 msec
SET: 79617.83 requests per second, p50=0.471 msec
GET: 79681.27 requests per second, p50=0.255 msec
INCR: 79808.46 requests per second, p50=0.247 msec
LPUSH: 99601.60 requests per second, p50=0.255 msec
RPUSH: 99700.90 requests per second, p50=0.271 msec
LPOP: 99601.60 requests per second, p50=0.271 msec
RPOP: 99601.60 requests per second, p50=0.255 msec
SADD: 66269.05 requests per second, p50=0.455 msec
HSET: 79744.82 requests per second, p50=0.255 msec
SPOP: 79617.83 requests per second, p50=0.327 msec
ZADD: 79681.27 requests per second, p50=0.447 msec
ZPOPMIN: 79491.26 requests per second, p50=0.319 msec
LPUSH (needed to benchmark LRANGE): 79681.27 requests per second, p50=0.407 msec
LRANGE_100 (first 100 elements): 44247.79 requests per second, p50=0.519 msec
LRANGE_300 (first 300 elements): 18921.48 requests per second, p50=1.359 msec
LRANGE_500 (first 500 elements): 12383.90 requests per second, p50=1.919 msec
LRANGE_600 (first 600 elements): 10441.68 requests per second, p50=2.247 msec
MSET (10 keys): 79681.27 requests per second, p50=0.463 msec

MemoryDB

$ redis-benchmark -q -n 100000 -h $MDB --cluster
Cluster has 2 master nodes:

Master 0: ae6fc4a323b891776433048ee44fcf728a097097 172.31.10.170:6379
ERROR: ERR unknown command `CONFIG`, with args beginning with: `GET`, `save`,
ERROR: failed to fetch CONFIG from 172.31.10.170:6379
WARN: could not fetch node CONFIG 172.31.10.170:6379
Master 1: c80783804e67afc9a25039fa26f77bb5773a5947 172.31.4.127:6379
ERROR: ERR unknown command `CONFIG`, with args beginning with: `GET`, `save`,
ERROR: failed to fetch CONFIG from 172.31.4.127:6379
WARN: could not fetch node CONFIG 172.31.4.127:6379

PING_INLINE: 66357.00 requests per second, p50=0.559 msec
PING_MBULK: 66401.06 requests per second, p50=0.543 msec
SET: 15923.57 requests per second, p50=2.999 msec
GET: 66445.18 requests per second, p50=0.551 msec
INCR: 15931.18 requests per second, p50=2.991 msec
LPUSH: 15951.51 requests per second, p50=2.999 msec
RPUSH: 15943.88 requests per second, p50=2.983 msec
LPOP: 15918.50 requests per second, p50=3.015 msec
RPOP: 15941.34 requests per second, p50=2.991 msec
SADD: 66313.00 requests per second, p50=0.551 msec
HSET: 15938.79 requests per second, p50=3.007 msec
SPOP: 66006.60 requests per second, p50=0.551 msec
ZADD: 66401.06 requests per second, p50=0.551 msec
ZPOPMIN: 66357.00 requests per second, p50=0.551 msec
LPUSH (needed to benchmark LRANGE): 15948.96 requests per second, p50=2.991 msec
LRANGE_100 (first 100 elements): 44228.22 requests per second, p50=0.719 msec
LRANGE_300 (first 300 elements): 18018.02 requests per second, p50=1.431 msec
LRANGE_500 (first 500 elements): 12402.33 requests per second, p50=2.063 msec
LRANGE_600 (first 600 elements): 10734.22 requests per second, p50=2.375 msec
MSET (10 keys): 14762.33 requests per second, p50=3.103 msec

Introducing AWS related articles on DevelopersIO

This site is written mainly by engineers of the operating company, Classmethod . We will deliver a wide range of technical information, from practical tips obtained from the AWS comprehensive support service " Class Method Members " provided by our company to applications of the latest AWS services.

AWS articles on DevelopersIO

share this article

event

[10/5 (Wednesday)] I want to know now when the yen is weak! AWS cost optimization points and case studies

[9/28 (Wednesday) Remote] We will hold a company briefing session for Class Method-Freelance Engineer Edition-

[9/27 (Tue)] "TTP-based Hunting" realized by Sumo Logic

[September 13 (Tuesday)] nOps, which optimizes AWS costs and security collectively

[9/16 (Friday)] Easy! certainty! Modern and secure code development ~Snyk for the first time~

[9/21 (Wednesday)] Let's touch Snowflake! Hands-on seminar for beginners

see more