Getting started Connecting Node.js Python PHP Go Guides Laravel Express.js

Guides

Scaling Express.js with Memcached: A Caching Guide

1. Intro

A simple CRUD application usually starts falling over as traffic grows. Even the most basic applications usually require a full round trip to the database, even when the data there barely changes. Additionally, if some of those pages ever need to write to the database, a request that otherwise would have been a read-only path may face bottlenecks such as lock contention or waits due to disk I/O. And, although these writes don't seem like a big issue at first, when the traffic scales, the added latency due to them also increases, making the database itself a bottleneck and potentially increasing cloud costs due to higher resource usage.

To illustrate this issue, and show how these kinds of issues can be fixed, in this guide we'll build a simple artist catalog application with data from MusicBrainz.org. For this example we'll keep the data model deliberately simple, as it will consist of just two models:

  • An Artist model, holding an artist's own details (a name, a type (i.e. "Person" or "Group"), gender, begin_date & end_date, free-text area and genres fields), as well as a view_count column we'll come back to later.
  • An ArtistUrl model, holding a type string (for example, "youtube" if the URL points to that artist's YouTube channel), the actual url and a relation pointing back to the artist the URL is associated with.

This catalog application will have just two kinds of pages:

  • A homepage, showing an ordered list of artists (either the result of a search, or a default listing when no search term is given); and
  • Individual artist pages, showing additional details about each artist such as its associated area, as well as links to other websites and social networks.

To make listings more useful, it's worth showing some of those additional details for each artist too, such as their country, their birth/foundation year, or their genres. Also, we may think that a user browsing a listing may expect to first see artists whose type is known (i.e. "Person", "Group", etc.) and/or whose area is set, which requires additional computation in the database for sorting records (we'll come back to this shortly).

Regarding artist pages themselves, in addition to the information already present in listings (artist name, type and area, genres and years active), they will also show a list of links from the ArtistUrl relation. Additionally, we will keep a counter of the number of times an artist has been accessed and show it as a rough measure of its popularity.

For this application we will be using Prisma as our ORM with the following schema:

model Artist {
  id         String      @id @db.Char(36)
  name       String      @db.VarChar(512)
  sort_name  String      @db.VarChar(512)
  type       String?     @db.VarChar(255)
  gender     String?     @db.VarChar(255)
  begin_date String?     @db.VarChar(255)
  end_date   String?     @db.VarChar(255)
  area       String?     @db.VarChar(255)
  genres     String?     @db.Text
  view_count Int         @default(0) @db.UnsignedInt
  created_at DateTime?   @db.Timestamp(0)
  updated_at DateTime?   @db.Timestamp(0)
  urls       ArtistUrl[]

  @@index([name], map: "artists_name_index")
  @@map("artists")
}

model ArtistUrl {
  id         BigInt    @id @default(autoincrement()) @db.UnsignedBigInt
  artist_id  String    @db.Char(36)
  type       String    @db.VarChar(255)
  url        String    @db.Text
  created_at DateTime? @db.Timestamp(0)
  updated_at DateTime? @db.Timestamp(0)
  artist     Artist    @relation(fields: [artist_id], references: [id],
                                 onDelete: Cascade,
                                 map: "artist_urls_artist_id_foreign")

  @@index([artist_id, type], map: "artist_urls_artist_id_type_index")
  @@map("artist_urls")
}

On the next sections, we'll start by briefly showing how the data used by the application is retrieved from the database and the potential problems we may find as it scales. Then, we'll see how to add caching in order to make listings and individual artist pages faster. Finally, we'll see how to leverage Memcached for logging each page view and batch updates to counters, avoiding issuing a separate database update per page visited.

2. The problem, concretely

The initial version of our application is a single router with two handlers (one for the root URL (/) and one for showing an artist (/:id)), where the first one (shown below) either triggers a database lookup by search term (via artistRepository.search(term, page)) or lists the artists in the database (by calling artistRepository.defaultOrder(page)). 

// src/routes/ArtistRouterFactory.js
router.get('/', async (req, res) => {
  const q = typeof req.query.q === 'string' ? req.query.q.trim() : '';
  const page = Number.parseInt(req.query.page, 10) || 1;

  const artists = q
    ? await artistRepository.search(q, page)
    : await artistRepository.defaultOrder(page);

  res.render('artists/index', { artists, q });
});

Previously, we briefly discussed how to sort artists in listings, and decided that those with a known type or area should come first. However, that should be combined with the most-common sorting criteria, that is, sorting by name. But "sorting by name" is more complicated than we may first think.

For sorting artists by name, a naive approach would be to just use an ORDER BY name DESC clause, but since that follows the standard ASCII ordering, it would make the list first show artists whose name starts with symbols (such as '^_^' or ¯\_(ツ)_/¯)), or obscure artists whose name starts with numbers (like 800 Cherries). In order to avoid that, we need an extra ordering clause that pulls names starting with a letter ahead of everything else.

Combined with the sorting conditions regarding artist types or areas we previously discussed, the ordering expression ends up needing three parts: a condition to put artists with a type, begin date and area set first, then a condition that sorts those further putting those with a name starting with letters from A to Z first, and finally a tiebreaker based on the standard ASCII ordering.

To make Prisma use a raw SQL expression for the ordering clause, we may add a helper method to construct the query with our custom ordering SQL as follows: 

// src/repositories/ArtistRepositoryFactory.js
const { Prisma } = require('@prisma/client');

const PAGE_SIZE = 5;
const LISTING_COLUMNS =
  Prisma.sql`id, name, type, gender, begin_date, end_date, area, genres`;
const ORDER_TAIL = Prisma.sql`
  (type IS NOT NULL AND begin_date IS NOT NULL AND area IS NOT NULL) DESC,
  name REGEXP '^[A-Za-z]' DESC,
  name ASC`;

function createArtistRepository({ prisma }) {
  async function paginate(whereSql, orderSql, page) {
    const currentPage = Math.max(1, page);
    const offset = (currentPage - 1) * PAGE_SIZE;
    const [items, totals] = await Promise.all([
      prisma.$queryRaw`SELECT ${LISTING_COLUMNS} FROM artists
          ${whereSql} ORDER BY ${orderSql}
          LIMIT ${PAGE_SIZE} OFFSET ${offset}`,
      prisma.$queryRaw`SELECT COUNT(*) AS total FROM artists ${whereSql}`,
    ]);

    const total = Number(totals[0].total);

    return {
      items,
      page: currentPage,
      perPage: PAGE_SIZE,
      total,
      lastPage: Math.max(1, Math.ceil(total / PAGE_SIZE)),
    };
  }

  function defaultOrder(page) {
    return paginate(Prisma.sql``, ORDER_TAIL, page);
  }

  //...
  return { defaultOrder /*, ...*/ };
}

module.exports = {
  PAGE_SIZE,
  createArtistRepository
};

Note how this requires three ordering clauses in ORDER_TAIL, where one of them consists of three nested clauses by itself. As we keep adding conditions, you can see how queries can quickly grow in complexity, which in turn makes it harder for the database to generate result listings fast enough (especially if the database holds hundreds of thousands or even millions of rows).

Similarly, when ordering search results themselves (that is, when there's a user-provided term), we may use a similar technique: show first results whose name exactly match search terms, then apply the same additional ordering clauses from the default ordering as tiebreakers: 

// src/repositories/ArtistRepositoryFactory.js
// ...

function escapeLike(term) {
  // MySQL/MariaDB's LIKE defaults to backslash as its escape character
  return term.replace(/[\\%_]/g, (char) => `\\${char}`);
}

function createArtistRepository({ prisma }) {
  // ...
  function search(term, page) {
    // Reuses the same ordering tail with an extra clause to make
    // an exact match to beat everything else.
    const like = `%${escapeLike(term)}%`;
    const whereSql = Prisma.sql`WHERE name LIKE ${like}`;
    const orderSql = Prisma.sql`name = ${term} DESC, ${ORDER_TAIL}`;
    return paginate(whereSql, orderSql, page);
  }

  //...
  return { search, defaultOrder };
}
// ...
Artist catalog homepage showing a paginated, sorted list of search results.

Regarding individual artist pages, the application will just show basic information (directly taken from the artists table) as well as their URLs (which the schema links to the Artist model under its urls attribute). However, we also wanted to track how many times each artist page is accessed in order to show it as a rough measure of its popularity. To do so, when serving an artist page, we need to both update the counter in the database and increase the in-memory value (so that the page shows the correct amount). 

// src/repositories/ArtistRepositoryFactory.js
// ...
function createArtistRepository({ prisma }) {
  // ...
  async function findById(id) {
    return prisma.artist.findUnique({
      where: { id },
      include: { urls: true },
    });
  }
  async function incrementViewCount(id, count = 1) {
    await prisma.artist.update({
      where: { id },
      data: {
          view_count: { increment: count }
      },
    });
  }
  //...
  return { search, defaultOrder, findById, incrementViewCount };
}
//...
// src/routes/ArtistRouterFactory.js
router.get('/:id', async (req, res) => {
  const artist = await artistRepository.findById(req.params.id);
  if (!artist) {
    res.status(404).send('Artist not found');
    return;
  }

  // Update the DB
  await artistRepository.incrementViewCount(artist.id);

  // Update the in-memory counter before rendering instead of re-reading
  // it from the DB with the new value (this will miss views recorded by
  // concurrent page requests, but is enough to get an approximate value).
  artist.view_count += 1;

  res.render('artists/show', { artist });
});
Artist detail page showing the artist's basic information and view count.

While the approach discussed so far may work in low-traffic scenarios, it may present problems as the application scales:

  • Although most data in the database barely changes, every page view does a full round trip to the database.
  • Every artist page view issues a write on a read path: Although executing UPDATE artists SET view_count = view_count + 1 WHERE id = ? concurrently is safe (the database would normally grant an automatic brief lock on the row being updated, guaranteeing that two simultaneous views of the same artist will serialize their increments instead of losing one), it still means that every page view would need to wait for a write to complete, including I/O overhead, even though the artist data itself rarely changes.

To solve these two issues, we need two different techniques, namely read caching and write batching, as we'll see next.

3. Connecting Express to Memcached

In order to improve the application's performance, the next step will be to connect the Express application to your Memcached instance. To do so, if you are using Velstash Cache, go to your cache instance page, where you'll find the host and port to connect to at the top.

Velstash dashboard: instance details page showing the host and port to connect to.

If you don't have the credentials to connect to your cache instance at hand (or if you want to create dedicated credentials for this application), go to the credentials section and create a new username & password pair by providing a name for your new application.

Velstash dashboard: creating a new set of credentials for an application.

First, you would need to edit your .env file and add the following contents (replace the host and port with those shown in the dashboard for plain connections, and the username and password with the credentials you previously generated for your application). 

MEMCACHED_HOST=mc1.public.gra2.ovh.velstash.io
MEMCACHED_PORT=11211
MEMCACHED_TLS=false
MEMCACHED_TLS_SERVERNAME=
MEMCACHED_USERNAME=username
MEMCACHED_PASSWORD=password

Those credentials will be passed to a small factory method that takes care of instantiating a Memcache client and, if provided, sending the authentication credentials. For this application, we will be using the memcache-client Node.js package, which also supports TLS connections. 

// src/cache/MemcachedClientFactory.js
async function createClient({ host, port, tls = false, tlsServername,
                              username = null, password = null }) {
  const client = new MemcacheClient({
    server: `${host}:${port}`,
    ...(tls ?
            { tls: { servername: tlsServername || host } } :
            {}
    ),
  });

  if (username) {
    await client.set('_auth', `${username} ${password}`);
  }

  return client;
}
module.exports = { createClient };

Worth noting that Velstash doesn't implement SASL for authentication (which requires using the deprecated binary protocol), but uses the standard authentication mechanism built into the Memcached protocol: issuing a SET with the credentials to any key (_auth here) right after connecting. Even though sending the authentication credentials this way won't result in an actual item being stored, the server response will follow the same semantics as if that would have been the case: a successful STORED response for correct credentials (which would make the awaited Promise to resolve successfully) or a CLIENT_ERROR on failure (i.e. invalid credentials, which will throw an Error in the code above to be handled by the caller).

With that in place, a small standalone script is enough to confirm the connection works: 

// scripts/cache-ping.js
const MemcachedClientFactory = require('../src/cache/MemcachedClientFactory');

require('dotenv').config();

async function main() {
  const client = await MemcachedClientFactory.createClient({
    host: process.env.MEMCACHED_HOST,
    port: process.env.MEMCACHED_PORT,
    tls: process.env.MEMCACHED_TLS === 'true',
    tlsServername: process.env.MEMCACHED_TLS_SERVERNAME,
    username: process.env.MEMCACHED_USERNAME || null,
    password: process.env.MEMCACHED_PASSWORD || null,
  });

  console.log('connected, setting ping=pong...');
  await client.set('ping', 'pong', { lifetime: 600 });

  const result = await client.get('ping');
  console.log('get ping ->', result.value.toString());

  client.shutdown();
}

main().catch((err) => {
  console.error('FAILED:', err);
  process.exit(1);
});
$ node scripts/cache-ping.js
connected, setting ping=pong...
get ping -> pong

If you are using Velstash Cache, you can also confirm the round trip worked by checking your instance's dashboard, which should show the new item (the UI may take a minute to update).

Velstash dashboard: instance details page showing the newly stored item accounted for.

Additionally, if you want to connect to your cache instance using SSL (for example, if connecting from a datacenter different from the one your cache instance runs in), you can do so by setting MEMCACHED_TLS to true, and setting the host and port to the values provided in the dashboard for TLS connections (normally, the host will be the same as the one used for plain connections, while the port will be 11212 instead of 11211). Additionally, if the dashboard shows a different host for TLS connections (that will be publicly accessible through Internet) but your application is located in the same datacenter as your cache instance, you may want to set MEMCACHED_HOST to the private host while having MEMCACHED_TLS=true and MEMCACHED_PORT=11212 for extra security and, in that case, you would need to set MEMCACHED_TLS_SERVERNAME to the TLS, public host, which is then used to verify the TLS certificate. 

MEMCACHED_HOST=mc1.public.gra2.ovh.velstash.io
MEMCACHED_PORT=11212
MEMCACHED_TLS=true
MEMCACHED_TLS_SERVERNAME=mc1.public.gra2.ovh.velstash.io
MEMCACHED_USERNAME=username
MEMCACHED_PASSWORD=password

Once the TLS credentials are set, you may use the same scripts/cache-ping.js script used to test non-TLS connections to verify that the encrypted connection works.

4. Caching database reads

Now that we have the client factory in-place, we may first add a small callback-based wrapper (similar to cache-manager's wrap(key, fn) method), allowing us to use a familiar read-through pattern instead of explicitly reaching for the raw memcache-client on every call site: 

// src/cache/CacheFactory.js
function createCache(client) {
  async function remember(key, ttlSeconds, fn) {
    const cached = await client.get(key);
    if (cached && cached.value !== undefined) {
      return JSON.parse(cached.value.toString());
    }

    const value = await fn();

    // Only persist non-null/non-undefined results: a miss keeps calling
    // fn() on every request rather than caching the absence of a value.
    if (value !== null && value !== undefined) {
      await client.set(key, JSON.stringify(value), { lifetime: ttlSeconds });
    }

    return value;
  }

  async function forget(key) {
    try {
      await client.delete(key);
    } catch (err) {
      // Deleting a key that isn't cached rejects with NOT_FOUND -- that's
      // still "not cached anymore", so treat it as a no-op rather than
      // an error.
      if (!/NOT_FOUND/.test(err.message)) {
        throw err;
      }
    }
  }

  return { remember, forget };
}

remember(key, ttlSeconds, fn) follows the same shape you'll find in most read-through caching helpers: return the cached value under key if there is one, otherwise call fn, store what it returns for ttlSeconds, and return it either way. By following this pattern, callers don't need to care whether they got a cached value or a freshly computed one.

4a. Caching single artist reads by ID

Now that we have a cache factory in place, we'll update the ArtistRepositoryFactory in order to receive the cache instance when building the repository, and to actually use it in the implementation of said repository. As a first step, we'll make the required changes to the findById method, which just consist on wrapping the pre-existing call to Prisma's findUnique with the cache's remember method: 

// src/repositories/ArtistRepositoryFactory.js
// ....
function createArtistRepository({ prisma, cache }) {
  //...
  async function findById(id) {
    return cache.remember(`artist:id:${id}`, 60 * 60, async () => {
      const artist = await prisma.artist.findUnique({
        where: { id },
        include: { urls: true },
      });

      if (!artist) {
        return null;
      }

      // Only type/url are ever shown for a link, so narrow down to just
      // those rather than caching the full row.
      return {
        ...artist,
        urls: artist.urls.map(({ type, url }) => ({ type, url }))
      };
    });
  }
}

cache.remember() takes three arguments, and it's worth being explicit about what each one does, since we'll be using this same method throughout the rest of this guide:

  • The key ("artist:id:${id}") is what identifies this particular cached value among everything else Memcached is holding, for this or any other application. It has to uniquely encode everything the cached result depends on (here, it just carries the artist's ID).
  • The TTL (60 * 60) is how long that value is allowed to stick around before Memcached will discard it (requiring future requests to recompute it). cache.remember() passes this value as the lifetime value for memcache-client's set() method, and omitting it falls back to a 60-seconds default.
  • The callback (async () => { ... }) is a function that is only run on a cache miss once cache.remember() has found out no value was previously set for the given key. If the key had a value in the cache, the callback call is skipped altogether, and its cost is avoided (in this case, the most significant cost is the DB query). On a miss, this callback is run, its return value gets stored under the provided key for the given TTL, and is then returned as the result for the cache.remember() call, so callers don't need to care whether they got a cached value or a freshly computed one.

Once the factory has been updated to use the cache, we need to update the point where the factory itself is called. Assuming we have a helper createContainer() function to build all app dependencies when the application boots, this may look like this: 

async function createContainer() {
  const memcachedClient = await MemcachedClientFactory.createClient({
    // ...
  });

  const cache = CacheFactory.createCache(memcachedClient);
  const prisma = PrismaClientFactory.createPrismaClient({ cache });
  const artistRepository = ArtistRepositoryFactory
      .createArtistRepository({ prisma, cache });

  return { memcachedClient, cache, prisma, artistRepository };
}

With that in place, refreshing an artist page a second time should be noticeably faster, since the database round trip is skipped entirely on a cache hit. In our tests, an artist page went from 10 ms to 1.9 ms once its cache entry was warm.

Artist detail page loading noticeably faster once its data is served from cache.

It is worth noting that a lookup for an ID that doesn't exist resolves to null, and remember() only persists non-null, non-undefined results; therefore, a bad ID keeps hitting the database on every request rather than caching a 404 that might become valid data seconds later.

There's another issue worth flagging here: the cached artist instance can drift from the database between requests; specifically, the view_count increment that the artist page handler does by calling artistRepository.incrementViewCount() performs writes straight to the DB, so a cached page can display a view count that's already behind reality until the one-hour TTL expires. We'll come back to invalidation more generally in 4d.

4b. Caching artist lookups by name

Previously, we implemented artist search under the same route handler as the default listing. Specifically, the handler detected whether a search was requested and wired that to a call to the repository search method, falling back to its defaultOrder method instead if no query was provided: 

// src/routes/ArtistRouterFactory.js
router.get('/', async (req, res) => {
  const q = typeof req.query.q === 'string' ? req.query.q.trim() : '';
  const page = Number.parseInt(req.query.page, 10) || 1;

  const artists = q
    ? await artistRepository.search(q, page)
    : await artistRepository.defaultOrder(page);

  res.render('artists/index', { artists, q });
});

Similar to how we cached single artist reads by ID, and given that createArtistRepository() already receives a cache instance as a parameter, we can also keep the router as it is and add caching to the repository methods instead. We will start with the search method.

Two things make the cache key for this call less obvious than the single-artist one, though:

  • First, a single search term paginates across multiple pages and each page is a different result set, so the page number needs to be included in the cache key; otherwise every page of a given search would collide on (and overwrite) the same cache entry.
  • And second, the search term itself is free-form user input, so it should be normalized somehow before using it as part of a cache key. In this case, we may normalize it by hashing the lowercase version of the search terms, which serves both to fold together searches that only differ by case, and to keep arbitrarily long or unusual query strings from turning into an oversized or malformed cache key (as Memcached won't allow keys with whitespaces or control characters, nor keys longer than 250 bytes).

Regarding Artists' view counts, note those are not used in result listings and they are subject to change frequently, so we were already excluding them from the data fetched for listings (i.e. view_count was already missing from LISTING_COLUMNS in ArtistRepositoryFactory).

Taking all into account, the search method ends up like this:

// src/repositories/ArtistRepositoryFactory.js
// ....
function createArtistRepository({ prisma, cache }) {
  //...
  function search(term, page) {
    const like = `%${escapeLike(term)}%`;
    const whereSql = Prisma.sql`WHERE name LIKE ${like}`;
    const orderSql = Prisma.sql`name = ${term} DESC, ${ORDER_TAIL}`;

    const keyHash = crypto.createHash('md5')
        .update(term.toLowerCase()).digest('hex');

    return cache.remember(
        `artists:search:${keyHash}:page:${page}`,
        5 * 60,
        () => paginate(whereSql, orderSql, page)
    );
  }
}

Here we pass the same three cache.remember() arguments as before (see 4a for what each one does in general). In this case, the cache key needs a second component to hold a page number, since page 1 and page 2 of the same search are different result sets that would otherwise collide on (and overwrite) the same cache entry.

Note also that in this case we've deliberately given search results a much shorter TTL than the one used for each artist page in the prior section (5 minutes here, one-hour TTL for each artist page). That's because search terms, being user input, are far more varied than artist IDs, and each distinct term/page combination is its own cache entry: even though the list of results for an individual page is cheap to cache and keep around (and prevents an expensive text lookup against the DB), many will never be requested a second time, and maintaining each search term entered in the search box will become expensive in terms of size, causing frequent evictions for items from a comparatively small, frequently-revisited set of individual artist pages looked up by artist ID. A short TTL keeps the cache from filling up with one-off searches nobody repeats, while still absorbing bursts of the same search term if queried over and over again (such as terms trending on social media, a shared link, a slow crawler hammering the same query, etc.).

Finally, note also paginate() was already only fetching the columns included in the LISTING_COLUMNS constant. Since that constant did not include the column for each artist view count, that means views for listings don't receive a count value they don't need but, more importantly, prevents saving to the cache a value that would be difficult to keep updated in this case and of no use anyway.

4c. Caching the default listing

The default listing (when browsing the list of artists instead of performing a search) follows the same shape as previously, but this time we don't need to include a search term into the cache key since only the page number varies. Also, given that this time the results in each page never vary due to different search terms (every user should see the same items on every page in the listing if not performing a search), we may choose to keep them in cache for longer, since we expect a higher hit rate for them (here we are using 30 minutes). 

// src/repositories/ArtistRepositoryFactory.js
function defaultOrder(page) {
  return cache.remember(
      `artists:list:page:${page}`,
      30 * 60,
      () => paginate(Prisma.sql``, ORDER_TAIL, page)
  );
}

At this point, if you repeat the check previously done for individual artist pages and measure the time taken to load a page from the listing (either with or without search terms), you should see that the time savings should be even more dramatic when visiting the same page a second time (both while browsing the listing normally and while browsing search results). In our tests, search result pages that previously took around 5 seconds went down to numbers between 1-50 ms once served from the cache.

Artist listing page loading dramatically faster once results are served from cache.

4d. Keeping the cache correct when data changes

So far we've only cached reads, but nothing yet stops the single-artist cache introduced in 4a from going stale if an artist's data is ever updated or deleted, for example, through an admin interface we may introduce in the future. Under that scenario, the application would keep serving the cached copy for up to an hour, regardless of what's now in the database.

In order to solve this issue, Prisma allows to define query extensions, which run for every call to a given model method regardless of where in the app that call comes from. Therefore, we can use that to intercept when a model has been updated and deleted, and in turn trigger an appropriate cache update for it. In the next example, we just delete the corresponding cache item, but we could also make it update the cache in response to model updates as well, in case the application was not updating the cache itself when updating the database. 

// src/PrismaClientFactory.js
function createPrismaClient({ cache }) {
  return new PrismaClient().$extends({
    query: {
      artist: {
        async update({ args, query }) {
          const result = await query(args);
          await cache.forget(`artist:id:${result.id}`);
          return result;
        },
        async delete({ args, query }) {
          const result = await query(args);
          await cache.forget(`artist:id:${result.id}`);
          return result;
        },
      },
    },
  });
}

After this change, every call to update() or delete() in the prisma.artist object, from anywhere in the app, will now forget that artist's cache entry automatically, without those needing to know anything about caching at all.

Note this is deliberately not extending the search/listing caches from 4b and 4c: since those are keyed by search term and page number instead of by artist ID, there's no single, predictable key to delete from the cache when one artist's row changes (even though that artist may appear on any number of listing pages or search results). Additionally, Memcached itself has no concept of tags or secondary indexes at all, and works purely as a key-value store, with no way to query "which keys reference X".

In order to solve this, we'd need a real reverse index, which would have to be built by hand; for example, maintaining artist:{id}:pages => [page cache keys] entries and keeping them updated whenever a new search results' page is added to the cache. And still, that would still produce inaccurate results: since Memcached may evict any cached key under memory pressure, the list of cached listing pages referencing a given artist ID could get dropped from the cache at any time, which would prevent the system from removing the cached data for listings that referenced that artist (making the whole approach unreliable).

5. Counting artist views without writing on every page load

Up to this point we've seen the typical usage of a cache that stores data both for listings and individual items. However, when we described pages for individual artists, we wanted to track the number of times each is accessed in a counter within the artist table itself.

In a high-traffic application, keeping that counter updated in real-time may pose some performance issues:

  • Incrementing the counter implies writing to the database for each page view.
  • Even if data is cached, updating the counter on each access means the cached data is immediately invalid right after the first time it is used (since the counter is now stale).

In high-traffic applications, incrementing a database counter may introduce contention: since each request for a given ID now needs to update the same row, it forces other requests to wait until that write completes. That means requests accessing the same artist would end up being served sequentially; and, unlike a normal update, incrementing a value requires locking the row to prevent other threads from writing to it (because next increments would require to know what value was settled by the transaction that first requested incrementing the counter).

And, even if we ignore the fact a page view now requires accessing the database (which is exactly what a cache is meant to avoid), ensuring that only the most up-to-date counter value ends up in the cache would require additional mechanisms (such as distributed locks) to prevent two requests (potentially served by different physical servers or server processes) from updating the cache for the same artist at the same time.

To solve this issue, we may leverage Memcached append operations in order to maintain an append-only list, which would serve as a log of what IDs require their counters to be updated. Then a background process could add all page views that happened recently back to the database in a single update, therefore avoiding the performance problems associated with individual requests updating counters concurrently in real time.

The implementation for this approach would live in a new service exposing two public methods: one used by route handlers to signal a new page view (so new views can be logged), and another one used by a scheduled command to update the DB with the new views (which at that point are only held in Memcached). The service instance would be returned upon calling a new factory method, createArtistViewBatcher, and conceptually resembles to a service appending items to a list and then flushing that list to a backing storage: 

// src/service/ArtistViewBatcherFactory.js
function createArtistViewBatcher({ memcachedClient, artistRepository }) {
  async function recordView(artistId) {
    await appendToList(artistId);
  }
  async function flush() {
    await flushList();
  }
  // ...
  return { recordView, flush };
}

Once this service skeleton for its public interface is defined, the changes required in the route handler are reduced to just calling recordView(): 

// src/routes/ArtistRouterFactory.js
function createArtistRouter({ artistRepository, artistViewBatcher }) {
  // ...
  router.get('/:id', async (req, res) => {
    const artist = await artistRepository.findById(req.params.id);

    // ...

    // Record this view in the cache instead of hitting the DB on every
    // page view; another script will then periodically apply the accumulated
    // totals to the DB.
    await artistViewBatcher.recordView(artist.id);

    // The DB row won't reflect this view until the next flush, so we need to
    // increase the in-memory value before it reaches the view. Note this
    // doesn't touch the DB, it only makes the number shown here account for
    // the view we just recorded above.
    artist.view_count += 1;

    res.render('artists/show', { artist });
  });

  return router;
}

As we've seen in the implementation for the public methods in the new service, recording a view is turned into an operation to append an item into a list, while recording them back to the database is conceptually modeled as flushing that list back to the database.

However, although the code above depicts recording a page view as adding items to a list and flushing them as some sort of atomic write-back to the database, that would be an oversimplification, since having just one shared list would mean there's a possibility of having both page views and the flush job trying to access it at the same time (either for appending views or for clearing the list once everything has been flushed). In order to avoid that, we actually need several lists of IDs, and some kind of non-blocking mechanism to make sure the web server never appends to a list of IDs once the background process has started to flush it to the database.

Specifically, we need an item in the cache telling which list we are currently appending to, and use a predictable pattern for naming the items in Memcached holding individual lists. Then the operation to append an item to the list may be implemented as two high level operations:

  1. Get the cache key for the current list based on an index key, initializing it to zero if missing.
  2. Actually append the ID of the entity for which we are recording a view for.

The implementation for this method is included below. Note this works directly against memcache-client's raw add & append methods rather than the remember & forget cache helper from section 4, since the latter were read-through caching operations, while this needs the protocol's real atomic primitives (add, to store an item only if the key doesn't already exist yet; and append to atomically add data after whatever's already stored for a key, failing if the key is missing). 

// src/service/ArtistViewBatcherFactory.js
const INDEX_KEY = 'artist-counter-pending-flushes-index';
const LIST_KEY_PREFIX = 'artist-counter-pending-flushes-';

function createArtistViewBatcher({ memcachedClient, artistRepository }) {
  //...
  async function appendToList(artistId) {
    await addKeyIfMissing(INDEX_KEY, '0');

    const index = await memcachedClient.get(INDEX_KEY);
    const listKey = `${LIST_KEY_PREFIX}${index.value.toString()}`;

    await addKeyIfMissing(listKey, '');
    await memcachedClient.append(listKey, `${artistId} `);
  }
  async function addKeyIfMissing(key, initialValue) {
    try {
      await memcachedClient.add(key, initialValue);
    } catch (err) {
      if (!/NOT_STORED/.test(err.message)) {
        throw err;
      }
    }
  }
  // ...
}

Once we have a method for appending the IDs, we need another one for actually collecting which IDs need their counters updated and by which amount. The mechanism this time is a bit more complex:

  1. Initialize the key used to track what is the current index name, in case it was not previously set (which also makes use of the same addKeyIfMissing(INDEX_KEY, '0') method used above).
  2. Increment the index so that new writes would now go into a different cache key from that point onwards. Given that Memcached returns the value resulting from an increment operation, this also serves to know what was the previously-used index.
  3. Fetch the list of pending IDs we are about to flush, which is stored under the key associated with the index we fetched on the previous step. Once fetched, delete it so a future flush doesn't double-count the same views again, and so the key doesn't stick around taking up space once it's no longer needed.
  4. Count how many times each ID is included in the list, additionally filtering out empty values in case there is any.
  5. For each ID, issue an update against the database and drop any cached data for the associated artist, which will now reflect the updated count once a new page view fetches the artist.

Note that the code below uses the repository's incrementViewCount for updating the counter, and that triggers the update hook we previously set on createPrismaClient in chapter 4d. Therefore, flushing the counters to the database also makes Prisma delete stale data from the cache associated with each artist being updated. 

// src/service/ArtistViewBatcherFactory.js
const INDEX_KEY = 'artist-counter-pending-flushes-index';
const LIST_KEY_PREFIX = 'artist-counter-pending-flushes-';

function createArtistViewBatcher({ memcachedClient, artistRepository }) {
  //...
  async function flushList() {
    await addKeyIfMissing(INDEX_KEY, '0');

    const newIndex = Number(await memcachedClient.incr(INDEX_KEY, 1));
    const oldIndex = newIndex - 1;
    const listKey = `${LIST_KEY_PREFIX}${oldIndex}`;

    const retired = await memcachedClient.get(listKey);

    try {
      await memcachedClient.delete(listKey);
    } catch (err) {
      if (!/NOT_FOUND/.test(err.message)) {
        throw err;
      }
    }

    if (!retired) {
      return 0;
    }

    const ids = retired.value.toString().trim().split(' ').filter(Boolean);

    const counts = {};
    for (const artistId of ids) {
      counts[artistId] = (counts[artistId] || 0) + 1;
    }

    await Promise.all(
        Object.entries(counts).map(([artistId, count]) =>
                artistRepository.incrementViewCount(artistId, count)
        )
    );

    return Object.keys(counts).length;
  }
  // ...
}

The flush job itself is now a thin wrapper for artistViewBatcher.flush(), which encapsulates all the logic to issue the database writes (rotate the list, read and delete the retired list, and write the new counter values back to the database). 

// scripts/flush-views.js
require('dotenv').config();

const ContainerFactory = require('../src/ContainerFactory');

async function main() {
  const { memcachedClient, prisma, artistViewBatcher } =
          await ContainerFactory.createContainer();

  const flushed = await artistViewBatcher.flush();
  console.log(`Flushed pending views for ${flushed} artist(s).`);

  memcachedClient.shutdown();
  await prisma.$disconnect();
}

main().catch((err) => {
  console.error('FAILED:', err);
  process.exit(1);
});

Wiring the flush logic to actually run on a schedule is also straightforward if using node-cron: 

// scripts/schedule.js
async function main() {
  const { artistViewBatcher } = await ContainerFactory.createContainer();

  console.log('Scheduler running, Ctrl+C to stop.');

  cron.schedule('* * * * *', async () => {
    const flushed = await artistViewBatcher.flush();
    const now = new Date().toISOString();
    console.log(`[${now}] Flushed pending views for ${flushed} artist(s).`);
  });
}
main().catch((err) => {
  console.error('FAILED:', err);
  process.exit(1);
});
$ node scripts/schedule.js
Scheduler running, Ctrl+C to stop.

While developing locally, you can verify if this works by manually running node scripts/schedule.js to run the scheduler in the foreground, which would check every minute for any due updates (enough to watch it write back to the DB shortly after browsing a few artist pages).

6. Wrap-up

In this tutorial, we have explored how to use Memcached as a backing cache for Node.js and Express and then how to leverage that cache for improving application performance on different scenarios of increasing complexity: caching single-model reads by ID, caching lookups by name and caching paginated results when no query to search for is provided.

Then, we explored how to keep the cache updated when data is updated in most common scenarios by evicting cached data by ID when a model is saved or deleted, and surfaced common problems to keep cached listings updated when individual items are updated.

Finally, we introduced a more complex scenario where we detected a write-constrained page on a supposedly read-only path, and then leveraged Memcached's speed for turning that into a deferred write. Specifically, we prevented several requests from updating the same database table concurrently under the hot path by adopting an in-memory write log, so writes could be deferred and then grouped based on the row they refer to instead, removing them from the read path.

This guide was built and tested against Velstash Cache. You too can provision a Memcached instance to speed up your Express application in no time by signing up here.