What is caching ?
In simpler language, caching is used for speeding up access to certain set of frequently accessed data. A cache is a layer b/w the client and the server. Each request that the client makes to the server, first goes through the cache. This serves two purpose :
- Cache Hit: If the data is already present in the cache, it is called a cache hit. In this case, it is sent directly to the client without querying the main server. This reduces the request time and also reduces the load on the server. This is because, if the data is already there in the cache, the cache will not query the main server again.
Example of a cache hit:
- Cache Miss: If the data is not there in the cache, it is called a cache miss. In this case, the data is fetched from the main server , stored into the cache and sent to the client. This will result in cache hit for subsequent access (See point 1)
Example of a cache miss:
Let’s see how the above concept is applied using a real world example.
Example
Let’s say, you are trying to access the current page. The process of accessing the current page can be broken down into the following steps as can be seen in the following diagram.
This is how our server configuration would look like without caching:
Out of all the above steps, there are two steps which take the most amount of time and are responsible for most of the latency that comes while loading this page :
- Internet connection speed: Faster your internet, more quickly the web page loads. This is beyond our control and we cannot optimize it.
- Time it takes to generate the html for this page.(Step4 and Step 5) : This is in our control and we can optimize it. Let’s say 100 different people are accessing this page, then we need to do steps 4 and 5, 100 times which takes up a lot of latency as well as increases the load on our database and application servers. Since, this is a static page (doesn’t change often), we can store the final generated html of this page in some place and “cache” it. This way, when a request comes for this page, we will be fetching it from the cache (which is much faster) and we will not query the database. This means that the latency introduced due to Step 4 and Step 5 is completely gone! and our page is much faster.
This is our architecture diagram with caching. A cache is placed b/w load balancer and the application server.
In the above diagram, we have introduced a cache in between the load balancer and the application servers. This will reduce the load on the application server for frequently accessed pages and decrease the latency for frequently accessed pages.
See the below GIF to get an idea on how the request is handled when we use a cache:
Cache Invalidation
In the above example we saw how cache makes the page access faster by storing a copy of the page and returning that when any request comes.
But what if, I change the contents of the page in the database ? In that case the data in the cache would become stale and we wouldn’t want our readers to be served stale data!
To solve this problem, we need to define the Cache Invalidation policy for the cache. This policy determines how the cache behaves if the data in the cache is modified/written. There are three popular cache invalidation strategies that are used frequently :
Write Through Cache
In write through cache invalidation policy, the data is directly written in both the cache and the database. The write succeeds if both the writes to the cache and the database succeed as shown in the diagram below. The main advantage of this policy is that, there is complete consistency b/w what is written into the cache and the actual data present in the database. The disadvantage is that since write needs to be done at two places, the writes become much slower.
Write Back Cache
In write back cache invalidation policy, fresh copy of the data is written directly in the cache instead of the database. The write succeeds as soon as data is written in the cache. Data written in the cache is written back to the database on a periodic basis and may not be immediate. The advantage of this cachive invalidation policy is that writes are much faster compare to write through approach, since only one write needs to be done and that too in the cache. Disadvantage is that this policy might lead to data loss (in case of a crash/power failure), since the only copy of the data is in the cache.
