Caching (part 3)

Level 3 - Pre-population

Taking everything we’ve learned so far into account, it’s possible to never serve a request dynamically. You might end up with a lot of 503s though, so pre-population of the cache can be helpful there.

Pre-population is generation of cache files by an application rather than by a request sent by a browser. It’s less proscribed than other points in this series, because it’s all about how you generate the requests. Something like a queueing daemon can help out here (like Apache ActiveMQ or IronMQ). The process is something like this :

  1. Generate a list of all your content
  2. Put URLs on the queue
  3. An application / script reads these URLs and visits them, triggering your caching code.

Conclusion

Hopefully now you have a resiliant and scalable caching system, but here’s the summary :

  1. Use memcache.
  2. Use a filecache, but make sure it’s kept up to date.
  3. 503s are a good way to give you some breathing space for generating pages dynamically.
  4. Pre-population should mean you never need to serve a page dynamically.