willianpinho.com Blog
Cover image for The API layer that was fake all the way down
  • architecture
  • api
  • integration
  • testing

The API layer that was fake all the way down

A generated client with a baseUrl nothing ever reads, and no transport anywhere. I wrote the retry and error layer the usual advice asks for, ran it against that boundary and then against a real one, and counted what the switch turned on: three dead branches, one that was wrong in a way nothing local could reveal, and a timeout that cancels nothing. Then six classifiers instead of one, because the first three defects might have been mine.

I was reading a generated API client in a codebase I had just been handed. It looked like every generated client you have ever integrated against:

export class InventoryApiClient {
  private baseUrl: string;

  constructor(baseUrl: string = "/api/mock/inventory") {
    this.baseUrl = baseUrl;
  }

A configurable base URL. Async, typed methods. Two clients, nine methods, 185 lines. Everything about the shape says "this talks to a network."

Then the method body:

  async getInventory(
    sku: string,
    location?: string
  ): Promise<InventoryItem | null> {
    // In a real implementation, this would make an HTTP request
    // ...
    const { getInventory } = await import("../../mocks/inventory-api");
    return getInventory(sku, location);
  }

A dynamic import of a local module, and a function call in the same process. That is the entire implementation. Every method in both clients is that shape.

To be clear about provenance: this was starter code I was given, not code I wrote. It was honest about itself, in the comment above that import, which matters for the point.

Open the file before trusting the interface: obvious, and everyone already agrees. The question starts after you open it. Knowing the boundary is fake, what do you build against it? The usual answer is to build for the real network now, since it is coming anyway. I think that is worse than building nothing. What follows does not settle that comparison, because it never builds the nothing arm. What it does is price one side of it: I wrote the layer that advice asks for, ran it against the fake boundary, then ran the same layer against a real transport and counted what changed.

What is actually absent

Not "mocked out". Absent.

grep -nE "fetch|http[s]?:|XMLHttpRequest|axios|Request\(|new URL" src/generated/*/client.ts
-> no matches

And the baseUrl is not merely defaulted, it is dead. Six occurrences across both files: declaration, parameter, assignment. Zero reads. Pass any string you like; nothing will ever look at it.

That dead parameter is the argument in miniature: configuration written for a network call that does not happen, sitting in the repository being read by nothing.

What is there instead

One directory over sit the modules those dynamic imports resolve to. Nine exported functions, and every one of them opens the same way:

export async function getInventory(sku: string, location?: string): Promise<InventoryItem | null> {
  // Simulate API latency
  await new Promise((resolve) => setTimeout(resolve, 100));

The declared sleeps are 100, 150, 100 and 50 milliseconds for inventory, and 100, 120, 100, 100 and 100 for shipments. That is the whole simulation. Of everything a network does to a caller, the one property these modules reproduce is the one that carries no failure.

I ran them to see what a caller could learn. 900 calls, 100 per method, through the generated clients rather than around them:

method                                    fail      min      p50      p99      max
InventoryApiClient.getInventory              0    99.47   101.56   105.03   108.97
InventoryApiClient.searchInventory           0   150.07   152.39   170.26   174.05
InventoryApiClient.getLowStockItems          0    99.75   102.64   107.32   108.83
InventoryApiClient.getLocations              0    50.44    52.37    56.82    58.83
ShipmentApiClient.getShipment                0    99.62   102.41   105.59   107.38
ShipmentApiClient.getShipmentsBySku          0   120.09   122.38   127.35   135.47
ShipmentApiClient.getDelayedShipments        0    99.78   102.36   104.63   106.37
ShipmentApiClient.getShipmentsByStatus       0   100.36   102.36   104.74   107.00
ShipmentApiClient.getActiveShipments         0   100.48   102.35   106.00   106.60

Zero failures. Every p50 lands between 1.56ms and 2.64ms above its declared constant, and the worst p99 is 1.12 times its own p50. There is a distribution in those numbers, but it is not a service's: the mock declares a constant, and what varies around it is the scheduler of whatever machine you ran it on. An earlier run of the same script on the same laptop put the same two figures at 1.45ms to 1.96ms and 1.20. Calibrate a latency budget from either run and you have calibrated it to a setTimeout.

Then I tried to make the calls fail: 40 of them, eight hostile arguments across the five methods that take one. Twenty threw. Every one was the same thing:

InventoryApiClient.getInventory <- null: Cannot read properties of null (reading 'toLowerCase')
ShipmentApiClient.getShipment <- number: trackingNumber.toLowerCase is not a function

Not one of the twenty resembles a network failure. They are type violations, reachable only by lying to the compiler. And the fifth method never threw at all: fed null, 42, [], undefined and "", getShipmentsByStatus returned {"shipments":[],"total":0} every time, which is a wrong answer wearing the shape of a right one.

What makes the absence easy to miss is that these files do model failure, in the wrong layer. The shipment records carry statuses delayed and exception, and notes like 'Weather delay - winter storm affecting route'. The domain's bad days are represented in detail. The transport's do not exist.

The layer written blind, run on both sides

So I wrote the layer the counter-position asks for, using only what that boundary can tell you. A 250ms timeout, comfortably above every p99 in that table. Three attempts, 50ms of linear backoff. A six-branch error taxonomy: ok, timeout, server_error, client_error, connection_refused, unknown. Ordinary code. I would have written it.

Against the fake boundary, thirteen calls through the generated clients:

branch hits: ok=11  timeout=0  server_error=0  client_error=0  connection_refused=0  unknown=2

Four of the six branches never execute. The two that do are the happy path and the bucket the TypeErrors fall into.

Then the same wrapper, unchanged, over fetch against a local HTTP server I can make answer 200, 500, 404, hang past the timeout, or refuse the connection:

200 fast                    ok                       attempts=1    14.08ms
500 upstream                FAILED as server_error   attempts=3   157.35ms
404 not found               FAILED as client_error   attempts=1     2.21ms
hangs past the timeout      FAILED as timeout        attempts=3   914.67ms
connection refused          FAILED as unknown        attempts=1     3.26ms
      error seen: fetch failed

branch hits: ok=1  timeout=3  server_error=3  client_error=1  connection_refused=0  unknown=1
work that landed after its timeout: 3

The script prints an error seen: line under every failure; three of the four are elided above, and the one left in is the one the next paragraphs turn on.

Three things came out of that, and only the first is the one I went looking for.

Three branches went from never executed to exercised in a single commit. timeout, server_error and client_error had not run once. It took a transport that could fail to run them at all, and I had to build that transport deliberately. Nothing here ran in production. That is the point: if nobody builds the failing transport on purpose, the first thing that runs those branches is whatever you deploy onto.

One branch never fired on either side, because it was wrong. The classifier tests err.code === "ECONNREFUSED". A refused connection out of fetch is a TypeError whose message is fetch failed and whose code is undefined; the string is on err.cause.code. So the refusal fell into unknown, unknown is not in the retryable set, and it got attempts=1. A refused connection is the textbook case for a retry, and it is the one case that got none. Nothing the fake boundary can produce would have shown me that. The runtime's own documentation would have, if it had occurred to me to doubt the property name.

The timeout does not cancel anything. These methods take no signal parameter and there is nothing to abort inside a function call in the same process, so the timeout you write is a race against a timer. For these particular functions, which sleep and then return, abandoning the work costs nothing. That is a property of the mocks and not of locality: local work that holds a lock or mutates state would not be so forgiving. Over a real transport the same code abandons the request without stopping it, and three of them completed after the wrapper had already reported them gone. The caller's latency is bounded, which may be all the timeout was ever for. The connections are not.

But that is my design

My own first reaction to those three was that they might belong to the wrapper rather than to the boundary, and that a better engineer would have written it differently. So I wrote six classifiers instead of one, each a shape that turns up in ordinary Node code: the code read off the error, the code read off err.cause, the node-fetch name check, a substring match on the message, "any TypeError is the network", and status codes with no transport branch at all. Then I ran all six against both boundaries, with two extra real failures added: a socket destroyed mid-body, and a 200 whose body is HTML.

On the fake side, five of the six are indistinguishable. Every scenario that boundary can produce gets the same verdict from all five: four calls ok, two type violations unknown. The sixth differs, and it differs by being wrong in the direction the fake boundary rewards, labelling those local type violations connection_refused.

On the real transport:

named the refused connection correctly: 2 of 6 (code-on-cause, typeerror-is-network)

Four of the six cannot name a refused connection at all. Of the two that can, one reads err.cause.code; the other calls every TypeError a refusal, which is why that same classifier also labels a socket reset and both local type violations as refusals. The socket destroyed mid-body is unknown to five of the six. The 200 that fails to parse is unknown to all six.

Status codes are the easy part, and all six get 500 and 404 right. Everything that is actually about the transport is where they diverge, and the fake boundary produces none of it. Nothing that boundary can produce tells these six apart, and telling them apart is the entire job. You can still get there another way, by reading the runtime's documentation or by building something that fails on purpose. Neither happens by default, and a boundary that never fails gives you no reason to start.

The objection also has an answer from outside my own code. On 15 August 2026, GitHub code search for the two strings error.code === 'ECONNREFUSED' and await fetch(, filtered to TypeScript, reported 940 matches. That total is GitHub's, it moves, and I have audited none of it. The three files I opened say something more interesting than "everyone gets this wrong". In f/git-rewrite-commits, src/providers/ollama.ts:69, that check sits in a file whose line 1 is import fetch from 'node-fetch', where the thrown FetchError really does carry code. It is correct. In Finsys/dockhand, src/routes/api/registry/tags/+server.ts:161, the identical comparison sits in a SvelteKit route. That repository has no node-fetch anywhere and never assigns globalThis.fetch; what it does do is call undici's setGlobalDispatcher, which changes routing and not the shape of the error. So fetch there is the runtime global, error.code is undefined when the registry refuses a connection, and the intended 503 Could not connect to registry does not return. The handler falls through to its generic branch and answers 500 with the message fetch failed. The comment in that repo's own DNS dispatcher says its IPv4 pinning "guarantees fetch failed" and cites three of its issues, which is the same TypeError this whole section is about. And in danshapiro/freshell, scripts/precheck.ts:197, someone writes both: error.code === 'ECONNREFUSED' || error.cause?.code === 'ECONNREFUSED'.

So the shape is not mine, and it is not carelessness either. It is a check that is right for one widely used library and wrong for the implementation the runtime now ships, and which side of that you are standing on is decided by an import line you may not have written.

The argument was already in the repository

I did not have to price the morning that layer costs, because part of it had already been spent. One directory up from the clients, the request handler wraps its work like this:

try {
  const { message } = await request.json<{ message: string }>();
  const result = await this.processMessage(message);
  return Response.json(result);
} catch (error) {
  return Response.json(
    { error: 'Failed to process message', ... },
    { status: 500 },
  );
}

processMessage is what reaches those generated clients. So the branch reads as "the call downstream failed" while the call downstream is a function invocation in the same process. Two inputs reached it in front of me: a malformed request body and a type violation inside a tool arriving as Cannot read properties of null. A catch that wide can certainly see others, and I am not claiming an inventory. I am saying neither of the two I saw is what the message says it is for.

The rest of the morning buys more of the same shape:

  • a latency budget derived from a constant plus 2ms of scheduler noise
  • a retry policy for a call whose only reachable error is a TypeError, which retrying cannot fix
  • an error taxonomy in which four of six branches are unreachable
  • a cache in front of something already in-process
  • load tests that measure your own import graph

The first three are what I measured. The last two I did not build, and they are on the list as the obvious next items rather than as findings.

Wasted work is the optimistic reading. The bill arrives the day the transport becomes real: the scaffolding switches from useless to load-bearing, and its transport branches have never run. The happy path ran eleven times and the catch-all twice; the three branches that exist for the network ran zero times. In this run the switch turned on all three at once, shipped a misclassification that disabled retries for refused connections, and left three abandoned requests in flight. A test could have reached the 5xx branch, as my own local server proves. Nothing in the repository reached it, so nothing did. What you have is error handling with months of age and zero evidence. In review it reads like error handling that survived months of production, unless someone thinks to ask whether any of it has ever run.

That is worse than an empty file. An empty file is honestly missing, and someone notices on the first timeout. This is a control everyone believes exists.

What the experiment does not show

Every layer here is mine. The wrapper is the code I would write against that boundary, not code recovered from someone else's repository, and the six classifiers are shapes I have seen and written rather than a sample drawn from anything. Six is not a survey, so "four of six" is a statement about those six. I name those three public files because I read them, not because I sampled them. I opened a handful out of 940 search hits and picked the three that showed the distinction, which makes them an illustration and not a rate.

The local server is not a network either. It can destroy a socket mid-body, which is one of the seven scenarios above, but it cannot produce packet loss, a TLS handshake failure, DNS, or a proxy. Every failure in that table is one I chose to produce, which makes three broken branches a floor and not a count.

And zero failures across 900 calls is evidence about the substitute, not about the service it stands in for. It does not even establish that the local functions are total; the twenty TypeErrors above are the counterexample. It says those inputs did not make them fail, and nothing about what the real API does under load, during a deploy, or behind something that drops connections.

The counter-position, in the form that survives

The seam is right here and the context is loaded, so build the port now. I would take that, and the run above changed my terms for it. Write the port either way: the shape costs almost nothing and makes the seam explicit. Then there are two honest options for the policy behind it. Leave it empty, and it is at least honestly missing. Or write it, and stand up something that actually fails so you can exercise it before you believe it. The scripts here took an afternoon, so the second is affordable, and I no longer think you have to wait. What is not on the list is the thing the usual advice actually produces: a policy written against a boundary that cannot fail, shipped, and trusted.

The check

Before you write the retry, name the failure it handles and where you saw it. If you cannot, you are not adding resilience, you are adding a claim.

Whether there is a failure mode to handle at all starts with one grep for the transport. It is a cheap check and a partial one: it will not see an aliased import or a wrapper module, and it speaks only for the files you point it at. When it comes back empty on a file that calls itself a client, that file is not an HTTP client. It is a naming convention.

And if you are going to write the layer anyway, run it against something that can actually fail before you believe it. Three scripts and a local HTTP server found three defects in code I was confident about, and then showed that of the six classifiers I wrote to check whether those defects were my fault, exactly one got the case the first one missed without mislabelling another scenario I ran. All of that was invisible from the side of the boundary where the code was written.