Benchmarking RedisGreen: A Redis Hosting Service

written in other

We recently found ourselves needing a redis instance for an app to be deployed on Heroku. It would serve as a shared cache between Node.js instances across multiple dynos. While interfacing with an EC2 instance could offer a static IP address, and the Proximo addon’s Elite or Enterprise plans could give us a way around this limitation, we decided to look at hosted services.

Services for what? Redis. A major benefit to starting off with redis, rather than say memcached, is that we can easily make use of its data structures for job queues and other features down the line, if needed.

And there’s a few redis addons on Heroku – one of which I found to be fudging their latency numbers. But we finally settled on RedisGreen after seeing their list of clients, which included Travis CI. In a guest post from Travis CI on the Heroku blog, a team member even mentioned their of use RedisGreen. Looks promising!

One thing we knew we’d have to face when using the service would be the increased latency compared to a redis instance in the same server, rack, or even DataCenter. But considering the app would only initially use it for storing and managing sessions, we weren’t looking to hit 100k op/s.

So how does the service stand up? Quite well, actually! Scroll down for some numbers and details.

Disclaimer: At the time of writing, redis-benchmark doesn’t support password-based authentication. Since RedisGreen instances require AUTH, I compiled a version of redis using the changes found in a pull request on GitHub. Furthermore, the highest 0.01% latency of any benchmark could be outliers caused by my own connection, since these benchmarks were performed from my laptop, and not a Heroku dyno. Your mileage may vary.

16 clients, no pipelining

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
$ ./redis-benchmark -h host -p port -a pass -c 16 -t set,get -n 10000
====== SET ======
  10000 requests completed in 56.54 seconds
  16 parallel clients
  3 bytes payload
  keep alive: 1

...
90.38% <= 93 milliseconds
...
99.11% <= 150 milliseconds
...
99.99% <= 179 milliseconds
100.00% <= 372 milliseconds
176.88 requests per second

====== GET ======
  10000 requests completed in 56.74 seconds
  16 parallel clients
  3 bytes payload
  keep alive: 1

...
91.14% <= 93 milliseconds
...
99.20% <= 150 milliseconds
...
99.99% <= 420 milliseconds
100.00% <= 790 milliseconds
176.25 requests per second

In the benchmark above, I simulated 16 clients, e.g.: 4 dynos, with 1 process per core each. Since the scope of our use is limited to managing session data, I’m only benchmarking SET and GET operations – though I could have thrown in DELETE as well.

The results assume a very naive client that performs no pipelining at all. Many redis clients offer the ability to pipeline commands. One such example is redis-rb. However, most won’t automatically pipeline commands like node_redis

16 clients, 10 request pipeline

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
$ ./redis-benchmark -h host -p port -a pass -c 16 -t set,get -n 10000 -P 10
====== SET ======
  10000 requests completed in 6.25 seconds
  16 parallel clients
  3 bytes payload
  keep alive: 1

...
90.51% <= 119 milliseconds
...
97.91% <= 152 milliseconds
98.41% <= 427 milliseconds
100.00% <= 427 milliseconds
1601.02 requests per second

====== GET ======
  10000 requests completed in 6.05 seconds
  16 parallel clients
  3 bytes payload
  keep alive: 1

...
89.81% <= 120 milliseconds
...
99.21% <= 152 milliseconds
99.91% <= 153 milliseconds
100.00% <= 153 milliseconds
1652.89 requests per second

Here we see some modest pipelining paying off. 1600+ requests per second is enough to satisfy our requirements, and that’s a pessimistic estimate.

16 clients, 100 request pipeline

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
$ ./redis-benchmark -h host -p port -a pass -c 16 -t set,get -n 10000 -P 100
====== SET ======
  10000 requests completed in 2.11 seconds
  5 parallel clients
  3 bytes payload
  keep alive: 1

...
91.01% <= 95 milliseconds
...
98.01% <= 112 milliseconds
99.01% <= 378 milliseconds
100.00% <= 378 milliseconds
4732.61 requests per second

====== GET ======
  10000 requests completed in 1.94 seconds
  5 parallel clients
  3 bytes payload
  keep alive: 1

...
91.01% <= 94 milliseconds
...
98.01% <= 111 milliseconds
99.01% <= 127 milliseconds
100.00% <= 127 milliseconds
5162.62 requests per second

Since node_redis makes an assumption that all redis commands will be executed and replied to in order, it automatically pipelines all requests by simply not waiting for replies. In the above benchmark, I tried to simulate a potentially longer pipeline that could be seen with this sort of client behaviour. The impact is very noticeable.

Just for fun, local redis

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
$ redis-benchmark -c 16 -t set,get -n 10000 -P 100
====== SET ======
  10000 requests completed in 0.02 seconds
  16 parallel clients
  3 bytes payload
  keep alive: 1

1.01% <= 1 milliseconds
11.01% <= 2 milliseconds
34.01% <= 3 milliseconds
87.01% <= 4 milliseconds
100.00% <= 4 milliseconds
434782.59 requests per second

====== GET ======
  10000 requests completed in 0.02 seconds
  16 parallel clients
  3 bytes payload
  keep alive: 1

1.01% <= 1 milliseconds
16.01% <= 2 milliseconds
98.01% <= 3 milliseconds
100.00% <= 3 milliseconds
555555.56 requests per second

Isn’t redis great?


Comments