Only the Crawlers Get the Server

For about a year, every Freebo checkout link that got pasted into iMessage, Slack, WhatsApp, or Facebook previewed as the exact same thing: a generic “Freebo — Professional Booking Platform” card with a tiny Freebo logo. Didn’t matter whose booking it was. Didn’t matter what product. Didn’t matter that the actual page, once a human clicked it, had the operator’s name on the header, their photos in the gallery, their colors on the buttons. The preview was ours. The page was theirs.

That’s not a brand problem for me. That’s a brand problem for the operator who’s paying for a product so they can run a booking flow under their own name. Pasting their link should look like pasting their link.

The reason it didn’t is the oldest story in SPA-land. Freebo’s checkout surface is a React app served out of a single index.html. All the real content — the operator name, the product cards, the first photo of the trip — gets filled in by JavaScript after the page loads. iMessage and Slackbot and Facebook’s facebookexternalhit crawler don’t run JavaScript. They read the HTML head and move on. So they all read the exact same head and showed the exact same card.

The common answer here is “do SSR.” Migrate to Next.js, or React Server Components, or rewrite the shell with hydration, or stand up a Node server that renders the React tree per-request. All of those are correct answers if you’re starting fresh. None of them are proportional answers when the thing you actually need is a different <title> and three <meta> tags on exactly the paths that get pasted around.

So the thing I shipped is smaller. One file, 170 lines, no new framework. The whole premise:

The pipeline

Everything hangs off a single Express app.get('*') handler at the bottom of the SPA server. Pseudocode:

app.get('*', async (req, res) => {
  if (isPreviewCrawler(req.get('user-agent'))) {
    try {
      indexHtml ??= fs.readFileSync(path.join(distDir, 'index.html'), 'utf8');
      const html = await sharePreview.render(indexHtml, {
        pathname: req.path,
        search: req.originalUrl.slice(req.path.length),
        pageUrl: `${req.protocol}://${req.get('host')}${req.originalUrl}`,
      });
      if (html) {
        res.setHeader('Vary', 'User-Agent');
        return res.type('html').send(html);
      }
    } catch {
      // Any failure serves the plain page below.
    }
  }
  res.sendFile(path.join(distDir, 'index.html'));
});

Three things are important in that block, and they’re the three things that make this cheap enough to ship:

1. Shoppers never enter the if. Real humans are not Slackbot, so their request skips the API call entirely and gets the stock SPA. The crawler-only path is the only one that waits on anything.
2. Every failure falls through. API down, path we don’t recognize, slug we can’t resolve, 2.5s timeout, parse error — any of those just serves the untouched index.html. The preview looks like it did last year. Nothing breaks.
3. The Vary: User-Agent header matters. Without it, a CDN or proxy can cache the crawler response and hand it to a real browser, which would be mortifying.

The user-agent net

The hardest part of this isn’t the regex, it’s trusting the regex. iMessage, surprisingly, doesn’t send its own crawler — it borrows Facebook’s. Twitter still fires Twitterbot even inside X. Slack rotates between Slackbot-LinkExpanding and Slack-ImgProxy. Reddit sends a browser UA half the time and redditbot the other half.

The whole net lives in one line so it’s easy to audit:

I put a loose fallback — plain bot, crawler, spider, preview — at the end of that regex so any new unfurl bot picks it up without a redeploy. The downside of a loose fallback is that an occasional headless automation gets the server-rendered path and pays a 2.5s tax. The upside is that the next WeChat clone doesn’t have to be special-cased.

The preview itself

Once you have the API payload, building the preview is just picking the right strings and the right image. Two rules that mattered more than I expected:

• The image has to be a still, not a video. The checkout cards animate — product MP4s play as silent looping photos — but the OG image has to be a static URL, so the preview picks the first non-video media item in the operator’s own ordering.
• The title has to feel like the operator’s, not Freebo’s. For a single-product link, the preview reads Trip Name | Operator Name. For a multi-product booking page, it reads Book online with <Operator>: Product A, Product B and more. Freebo as a brand only appears if the API resolve failed, in which case the whole crawler path quietly skipped anyway.

That firstPhoto helper is doing the hidden-but-important work: it sorts media by the operator’s own sequence, falls back to array index, and filters out videos. The non-video filter is one line and it is the whole difference between a preview card that shows the trip and a preview card that shows a thumbnail-less black rectangle because an iPhone decoded the MP4 as the OG image.

Why this shape instead of the “right” shape

I spent a weekend before this seriously considering a Next.js migration. I bounced off two things:

• The SPA works. The real checkout flow is a single React tree with its own router, state, and payment lifecycle. Rebuilding that under a server-rendered framework for the sake of three meta tags is a huge diff that touches every page, every test, and every deploy, and the only win is link previews.
• The problem is 100% on the paths that get pasted. Nobody shares /dashboard. Nobody pastes /admin/products/edit. The surface area that needs a preview is small, finite, and already behind a clean public API. That’s a cue to not do a framework migration — it’s a cue to write a route table.

Big migrations are fun to design and expensive to live inside. Little migrations — single files, one regex, one fallback that silently keeps the old behavior — are boring and ship next week.

What I’d steal from this for other projects

If you have an SPA and the only reason you’ve been eyeing SSR is link previews, this trick is probably enough:

• Keep the SPA. Don’t touch React.
• Add an Express (or Hono, or Fastify) catch-all before the static handler.
• UA-sniff for crawler. One permissive regex plus a loose bot|crawler|spider|preview fallback.
• Call the same public API the SPA calls on load. Cache aggressively — unfurl bots retry, and iMessage, Slack, and WhatsApp will all fetch the same URL within seconds of a paste.
• Regex-replace the <title> block in index.html with your OG + Twitter tags. Always set Vary: User-Agent.
• Any failure falls through to the unchanged file. Nothing you add here is allowed to break the real page.

The whole thing is a server-side rendering of exactly one tag on exactly the paths that need it. Everyone else — the people actually paying you — gets the fast, static, cache-friendly SPA they already had.

The crawlers get a server. The humans get the app.