./ahmedhashim

Cache is almost always going to be some flavor of Map<Key, Value>. When you’re caching something that’ll be hit concurrently, it’s more helpful to have a Map<Key, Promise<Value>>. This will prevent you from having to worry about most stampede bugs.

The problem

If you had a social network that was fetching users’ profiles that were popular, you could end up with something like this:

const cache: Map<number, UserProfile> = new Map();

const getProfile = async (userId: number): Promise<UserProfile> => {
  if (!cache.has(userId)) {
    cache.set(userId, await db.getUserProfile(userId));
  }
  return cache.get(userId);
};

As you can see, there’s a delay between checking the cache and awaiting it. During this time your cache won’t be populated and will fire another db query. As you can imagine, this becomes a problem as the number of requests increases.

The fix

Instead of caching the value, we can cache the promise:

const cache: Map<number, Promise<UserProfile>> = new Map();

const getProfile = async (userId: number): Promise<UserProfile> => {
  if (!cache.has(userId)) {
    cache.set(userId, db.getUserProfile(userId));
  }
  return cache.get(userId);
};

This way, when we have multiple calls to the same function, they’ll all await the same promise, meaning only 1 query to the db will happen per key.

Why it works

When you create a promise in JavaScript, it’ll start executing immediately. By awaiting that promise multiple times, it’ll return the same value. So once your promise is resolved, it’ll act like caching the value.

Error handling

One thing to keep in mind is if your promise rejects, you’ll want to clear it from the cache. Otherwise, all future calls will reject. You can do this by clearing it from the cache when it fails:

cache.set(
  userId,
  db.getUserProfile(userId).catch((error) => {
    cache.delete(userId);
    throw error;
  }),
);

You’ll also want to consider how much memory you want to allow your cache to grow. Your PromiseMap will grow indefinitely until you start evicting from it. So for larger sets of data, you’ll want to implement some type of TTL/LRU map.

It’s simply changing your map to store promises instead of values.