CSE Roadmap

Core tier · about 4 hours · 10 parts

Part 6 of 10 · 2 min read

Make it fast

A cache that spares the database, and a worldwide network that serves files from nearby.

With more web servers and copies of the data, the next problem is speed. Two tools help.

Cache. A cache keeps the answer to an expensive or common request in fast memory, so the next request does not repeat the work.

The pattern is simple. A web server gets a request. It checks the cache. If the answer is there, it returns it. If not, it asks the database, stores the answer in the cache and returns it.

def get_menu(shop_id):
    key = f"menu:{shop_id}"
    value = cache.get(key)          # 1. look in the cache first
    if value is not None:
        return value                # 2. found: fast path
    value = database.load_menu(shop_id)   # 3. not found: ask the database
    cache.set(key, value, ttl=300)  # 4. keep it for 300 seconds
    return value

Points to decide when using a cache:

  1. When to use it. For data read often and changed rarely. Memory is wiped on restart, so a cache is not the place for data that must survive.
  2. Expiry. Give every entry a lifetime. Too short, and the database is hit again and again. Too long, and the data turns stale.
  3. Consistency. Keep the cache and the database in step. The two are updated separately, so they can disagree.
  4. Failure. A single cache server is a single point of failure. Use several.
  5. Eviction. When the cache is full, an old entry must go. A common rule is least recently used (LRU): remove the entry that has not been used for the longest time.

Content delivery network. A content delivery network (CDN) is a network of servers spread across the world that keep copies of static files, such as images, style sheets and scripts. A user is served by the nearest copy.

How it works: a user asks the CDN for an image. If the nearest CDN server does not have it, it fetches it from the origin (the main storage), keeps a copy for a set time called the time to live (TTL), and sends it on. The next user nearby gets the copy at once.

Things to decide: the cost (CDNs charge for traffic), the right expiry, a plan for when the CDN is down, and how to remove a stale file early. One way to refresh a file is to change its name, such as image.png?v=2.

Now static files no longer burden the web servers, and the database does less repeated work.

Time to live

Time to live, or TTL, is how long a stored copy may be used before it must be discarded or refreshed.

Example: a TTL of 300 seconds means the copy of a menu is used for five minutes.

Check yourself:

  • In the code above, what happens the first time a menu is requested?
  • Why does a very long TTL cause trouble for data that changes?