Why did I use redis cluster and how should I use it?

To Nha Notes | Jan. 27, 2021, 10:34 p.m.

The problem is, AWS RDS MySQL server is high in DB load due to using too many transaction locks by using selec_for_update method of Django framework. To solve this problem, I switched to use redis for locking by using library python-redis-lock, and the high-speed of redis could solve this problem.

But then I went to another problem, that is when the redis is reached to its max capacity in peak access, and I could not scale it up to next stronger one. Thus, I switch to use redis cluster so that I can scale out to multiple shards based on traffic of access.

And the result when using redis cluster is not as my expected yet. I was trying two patterns of redis cluster as following:

• 3 shards of instance cache.t3.micro
• 2 shards of instance cache.m6g.large

But the second patterns is worse than the first one. And both of these patterns is worse than single node of cache.m6g.xlarge of using non-cluster redis.

So what is the cause of this bad result, I am considering below potential ones:

• The vCPU is not high enough (2) ?
• Number of shards is not enough
• Network performance of communication between nodes is not good
• Redis cluster architecture is not specific for high-speed purpose.

What do you think?