Add a Redis instance
Java Serialized Objects
Use Smaller Keys
Switch to 32 Bits
Upgrade Redis Version
Use Better Serializer
Combine Smaller Strings to Hashes
Switch from Set to Intset for Numeric IDs
Switch to Bloom Filter or HyperLogLog
Shard Big Hash to Small Hash
Convert Hashtable to Ziplist for Hashes
Convert to a List instead of Hash
Compress Field Names
Enable Compression for List
Avoid Dynamic Lua Script
Reclaim Expired Keys Memory Faster
Compress Strings Before Storing them in Redis
Redis and clients are typically IO bound and the IO costs are typically at least 2 orders of magnitude in respect to the rest of the request/reply sequence. Redis by default does not compress any value that is stored in it, hence it becomes important to compress your data before storing in Redis.This will help in reducing the payload which will in return give you higher throughput, lower latency and higher savings in your cost.
How to Compress Strings
There are several compression algorithms to choose from, each with it’s own tradeoffs.
- Snappy by google aims for very high speeds and reasonable compression.
- LZO compresses fast and decompresses faster.
- Others such as Gzip are more widely available.
GZIP compression uses more CPU resources than Snappy or LZO, but provides a higher compression ratio. GZip is often a good choice for cold data, which is accessed infrequently. Snappy or LZO are a better choice for hot data, which is accessed frequently.
Compressing strings requires code changes. Some libraries can transparently compress objects, you would only need to configure your library. In other cases, you might have to compress the data manually.
Compressing strings can save you anywhere between 30-50% memory. By compressing strings, you also reduce the network bandwidth between your application and redis databases.
Compressing/Decompressing requires your application to do extra work. This tradeoff is usually worth it. If you are concerned about additional CPU load, switch to a faster algorithm like snappy or LZO.
When to Avoid Compression
Compression should not be followed blindly, there are times when compression will not help you reduce your memory, rather will increase your CPU utilization. There are few cases when compression should be avoided:
- For shorter Strings it’s likely a waste of time. Short strings generally don’t compress much, so the gain would be too small.
- When the data isn’t well structured, compression should be avoided. JSON and XML are good at compression as they have repeated characters and tags.