Using Redis For An Ethereum Mining Pool

Redis [redis.io] is an in-memory data structure storage database. Redis supports data structures such as hashes, lists, sets, sorted sets with range queries. Redis also provides a PUB/SUB messaging system.

I’ve been using Redis to store and process the data required for the BeefPool.net Ethereum mining pool running on a Ubuntu 14.04.4 LTS on a virtual server. As mining is a calculation competition between miners, every millisecond counts. Using Redis to record data sent to and from the miners adds minimal delays to the communications between the pool and the miners.

The mining pool back-end has been developed using the Go [golang.org] programming language and uses the godis [github.com/simonz05/godis] Go Redis client module.

I’m in the process of writing the mining pool front-end to perform the accounting calculations and make the automated payments. This is written in Java and uses the lettuce [github.com/mp911de/lettuce] Java Redis client module.

Notes on the installation and usage of Redis on the Linux platform follow.

In summary, Redis is easy to install and use. It is fast enough for my requirements, does not use much CPU or disk resources and can be accessed from various programming languages as well as the command. You will have to spend some time to design your data models for easy storage and access of the data. Although I have not used this functionality, Redis databases can be clustered.

---

Installation Of Redis On Ubuntu Linux

It’s simple to install Redis from the Ubuntu repository:

Here’s a benchmark of Redis to check it is running:

Remember to set a password in /etc/redis/redis.conf to access Redis:

And restart Redis server for password to take effect:

The Redis data can be accessed from the command line:

You can also specify the password from the command line:

The Redis installation uses port 6379 on localhost by default:

---

Using Redis In The Go Mining Pool Back-End

Import the godis module into your source code:

Declare a client variable:

Open the Redis connection:

Here’s how I save a miner’s reported hashrate into a Redis sorted set. The sort key is a 64-bit floating point number and I am using a number of millseconds from 01/01/1970 as the sort key:

Here’s how I save a miner’s successful mining of a block into a Redis sorted set:

And here’s how I save a miner’s preferences into a Redis hash:

---

View The Saved Redis Data

Let’s view the data saved by the mining pool back-end. First we will check the keys created:

View the miner’s preference stored in a Redis hash:

Let’s look at the last three reported hashrate by a miner. I have saved this information in a Redis sorted set using using a sort key (called “score” in Redis) of the number of milliseconds since 01/01/1970. Here the ZREVRANGE command to view the latest 3 reported hashrate records:

Let’s retrieve a miner’s reported hashrate between two time ranges:

Let’s look at the data recording the blocks mined by a miner:

This corresponds to the last three blocks mined by this BeefPool.net mining pool. From etherchain.org/account/0xbEeF281B81d383336AcA8B2B067a526227638087#blocks:

---

Accessing The Saved Redis Data from Java

Add the following Maven dependency:

Declare the RedisClient variable:

Instantiate the Redis client variables and connect to Redis:

Retrieve the miner’s preferences from the Redis hash:

Retrieve the miner’s hashrate for the last 20 minutes:

---

Further Redis Features

Saving Data To Disk

By default Redis saves it’s in-memory data to disk after a specified interval if more that a specified number of keys are modified/written. To manually save the Redis data held in memory into a disk file, issue the command:

The data is saved into the the following file by default:

Separate Databases

You can store your data in separate databases within your Redis instance. The number of databases by default is 16 – this can be configured with the ‘database’ key within /etc/redis/redis.conf .

The following snippet demonstrates how to switch databases:

Publish/Subscribe

Start Redis in two separate sessions. Here is session #1 where I have subscribed to the topic “test”.

And here is session #2 where I have published to the topic “test”.
You can also use PSUBSCRIBE to subscribe to a pattern, e.g., PSUBSCRIBE Topic.A*. Here is a new session #1.
And here is a new session #2.

Transactions

Redis commands can be grouped together in transactions so that either all commands will successfully be run or all commands will fail:

Keys With Automatic Expiry

I’ll delete all existing keys, then create a key with a 5 second expiry: