Part 6 of 10 · 2 min read
Make it fast
A cache that spares the database, and a worldwide network that serves files from nearby.
With more web servers and copies of the data, the next problem is speed. Two tools help.
Cache. A cache keeps the answer to an expensive or common request in fast memory, so the next request does not repeat the work.
The pattern is simple. A web server gets a request. It checks the cache. If the answer is there, it returns it. If not, it asks the database, stores the answer in the cache and returns it.
def get_menu(shop_id):
key = f"menu:{shop_id}"
value = cache.get(key) # 1. look in the cache first
if value is not None:
return value # 2. found: fast path
value = database.load_menu(shop_id) # 3. not found: ask the database
cache.set(key, value, ttl=300) # 4. keep it for 300 seconds
return value
Points to decide when using a cache:
- When to use it. For data read often and changed rarely. Memory is wiped on restart, so a cache is not the place for data that must survive.
- Expiry. Give every entry a lifetime. Too short, and the database is hit again and again. Too long, and the data turns stale.
- Consistency. Keep the cache and the database in step. The two are updated separately, so they can disagree.
- Failure. A single cache server is a single point of failure. Use several.
- Eviction. When the cache is full, an old entry must go. A common rule is least recently used (LRU): remove the entry that has not been used for the longest time.
Content delivery network. A content delivery network (CDN) is a network of servers spread across the world that keep copies of static files, such as images, style sheets and scripts. A user is served by the nearest copy.
How it works: a user asks the CDN for an image. If the nearest CDN server does not have it, it fetches it from the origin (the main storage), keeps a copy for a set time called the time to live (TTL), and sends it on. The next user nearby gets the copy at once.
Things to decide: the cost (CDNs charge for traffic), the right expiry, a plan for when the CDN is down, and how to remove a stale file early. One way to refresh a file is to change its name, such as image.png?v=2.
Now static files no longer burden the web servers, and the database does less repeated work.
Time to live
Time to live, or TTL, is how long a stored copy may be used before it must be discarded or refreshed.
Example: a TTL of 300 seconds means the copy of a menu is used for five minutes.
Check yourself:
- In the code above, what happens the first time a menu is requested?
- Why does a very long TTL cause trouble for data that changes?