Gateways, proxies and service meshes

A mesh moves the concerns of service-to-service communication out of the application and into a proxy deployed beside each instance. Retries, timeouts, circuit breaking, mutual TLS, traffic splitting and per-call telemetry become configuration rather than library code, and crucially they become uniform across services written in different languages by different teams.

That last point is the honest argument. In a single-language estate, a shared client library does most of this, and a library is simpler to operate than a fleet of proxies. In a polyglot estate, a library means implementing and maintaining the same behaviour four times, which is where the sidecar's per-instance cost starts looking cheap.

The features that are genuinely hard to get any other way are mutual TLS everywhere with automatic certificate rotation, consistent traffic shifting for canaries at the network level, and telemetry that is identical across services because it is emitted by the same proxy. Each of those is possible without a mesh and each is fiddly enough that most teams do not do it consistently.

The cost is a second network to understand. A proxy per instance is CPU, memory and a latency addition on every hop, the control plane is a new dependency in the request path's configuration, and debugging now involves asking whether the problem is the application, its sidecar, the other sidecar or the control plane. That is a genuine increase in the number of places a failure can live.

So the honest test is the number of services and the number of languages. A handful of services in one language does not need a mesh and will feel the cost immediately. Fifty services in four languages, with a platform team to run it, is where the uniformity pays for the complexity. Everything in between is a judgement, and the sidecar-free variants that push the same functions into the kernel are a live attempt to lower the price.

Traffic entering the estate and traffic moving inside it have almost nothing in common, and conflating them produces designs that are wrong at one end. External traffic comes from clients you do not control, over networks you cannot measure, with credentials that may be stolen and volumes that may be hostile. Internal traffic comes from services you deployed, in a network you configured, and every caller has an identity you issued.

So the concerns differ. At the edge: authentication of untrusted callers, per-tenant rate limiting, request size limits, bot mitigation, TLS termination and caching for anonymous traffic. Inside: identity between services, timeouts and retries with budgets, circuit breaking, load balancing that understands long-lived connections, and tracing that spans the whole call tree.

Zero trust is the idea that the interior should not be treated as safe simply because it is the interior. A flat internal network where any service can call any other with no identity means one compromised component reaches everything, which is how a modest breach becomes a total one. Mutual TLS and per-service authorisation make lateral movement a series of separate obstacles rather than one.

The mistake in the other direction is applying edge machinery inside. A heavyweight gateway between every pair of internal services adds a hop, a failure domain and a bottleneck to calls that were fine. Internal calls want the lightest thing that provides identity, timeouts and telemetry, which is the argument for a sidecar or a library rather than for a second gateway.

The practical layering that most estates converge on is a CDN and a public gateway at the edge for untrusted traffic, and a mesh or a shared library inside for identity and resilience between services. Naming which layer a requirement belongs to is usually enough to settle an argument about where a feature should live.