Today I added a new server to the site. I will call this the "Task" server, but the server's host name is "Perlea".
We were running with a two server setup App server(Ewa) and Database server(Kate).
The DB server is running a bit more than just the DB, it is also running the Elastic search server and now, a Redis server.
The new Redis server will be used for three purposes:
- As a task queue, I was using RabbitMQ before to queue the asynchronous tasks, which was running on the application server.
- As a Key/Value store for generated thumbnails cache. I was using a database table for this, but it was doing too many hits competing with other reads I needed.
- Denormalized storage, I had a small Redis instance running on the application server which was doing some API counting and such, this same usage will not go to the Redis instance running on the DB server.
So now, I am using one same Redis instance to replace many moving pieces the site used to have.
There have also been changes on the application server. It was doing too many things:
- Running asynchronous tasks (recalculations, thumbnail generation, etc)
- Doing email sending
- Running scheduled tasks
- Generating daily backups
- Running application server for main website, API and dev's site.
So now I added a third server, task server, that will leave the application server only running the websites and it will take care of doing all the other items on the list. So now the app server has free CPU and memory to run the web requests.
Adding a third piece to the equation really changes the game as now the communication is not one line anymore, but a triangle, which requires a better code organization and tasks separation. In the future, adding a 4th server or more should not be difficult. The next step is to have automatic deployment of servers since each server setup is fairly manual, albeit, quite simple.