Cloud platforms and cost

Do the arithmetic before choosing anything. Ten thousand visitors a month, with a few pages each, is a handful of requests a minute at peak. A single small server or a static site on a CDN handles that without noticing. Multi-region clusters, autoscaling groups and service meshes answer problems that start several orders of magnitude higher, and the number is easy to compute and rarely computed.

The habit worth forming is converting everything into requests per second and bytes per second. A million requests a day is about twelve per second averaged, perhaps fifty at peak. A hundred gigabytes of storage is one disk. Those numbers deflate most architecture arguments before they begin, and the ones they do not deflate are the ones worth having.

Over-engineering costs money and, far more expensively, costs change. Every additional moving part is something to operate, debug, secure, upgrade and explain to the next person, and it is paid for while the product is still unproven. The right architecture for an unvalidated idea is the one that can be replaced quickly, because most of its decisions will turn out to be wrong.

Modern hardware is also larger than most people's intuition. A single well-configured server with fast storage handles tens of thousands of requests a second for many workloads, and a managed database on a mid-sized instance serves traffic that would have needed a room of machines fifteen years ago. Vertical scaling remains the cheapest scaling available and stops being fashionable long before it stops being effective.

The exception is anything expensive to change later. Data models, tenancy boundaries, identifiers and the public shape of an API are decisions that get more expensive every month, so they deserve thought now. Instance sizes, autoscaling rules and even the choice of platform are reversible in an afternoon and deserve much less. So: build for the traffic you have and design for the traffic you can measure yourself heading towards. Scaling ahead of a forecast is speculating with the only resource an early product genuinely lacks, which is time.

Egress is the most commonly underestimated line on a cloud bill. Data leaving the provider is charged per gigabyte while data arriving is usually free, and serving media directly from object storage costs several times what the same traffic costs through a CDN. Cross-zone and cross-region transfer is charged too, so a chatty architecture spread across zones pays for its own internal conversation.

Idle provisioned capacity is next, and it is the quietest. Instances and databases sized for a peak that happens twice a year run at that size for the other 363 days. Non-production environments running at production size overnight and at weekends are the same waste with fewer excuses, and switching them off on a schedule is usually the fastest saving available.

Then there is the tax on inattention: unattached volumes, old snapshots, forgotten load balancers, log retention set to forever, data sitting in an expensive storage class years after anyone read it. None of these is architecture; all of them are somebody's afternoon and a recurring charge until someone spends it.

Managed services trade money for operational time, and that trade is excellent for a small team and increasingly poor at large volume, which is why big companies eventually build their own. Netflix building Open Connect and Dropbox moving off S3 are the same calculation at different points on the same curve: past some volume the provider's margin exceeds the cost of doing it yourself, and below it managed wins comfortably.

The habits that keep a bill honest are unremarkable. Tag everything so cost can be attributed to a team or a feature, set a budget alert so a surprise is a notification rather than a monthly discovery, and look at unit economics, cost per request or per customer, rather than the total, because the total is supposed to grow.

The last one is cultural. Engineers cannot economise on a bill they never see, and the single most effective cost control in most organisations is showing the people who create the spend what it costs, at the level of the feature they just shipped.

The three options are best understood by what you hand over. With a virtual machine you hand over the hardware and keep the operating system, the runtime, the process supervision and the patching. With a managed container platform you hand over an image and keep the code inside it. With a serverless function you hand over a function and, crucially, you also hand over the idle time, which is the part that changes the bill rather than the architecture.

Scaling to zero is the property serverless is bought for. A function with no traffic costs nothing, and one with a hundredfold spike gets a hundred instances without anybody having configured a scaling group. That suits work whose shape is spiky or unpredictable: a webhook receiver, a scheduled job, an admin endpoint used twice a week, the long tail of internal tooling that would otherwise each need a home.

The cold start is the cost of that, and it is worth being specific about where it comes from rather than treating it as a fixed tax. A request arriving with no warm instance waits for a sandbox to be created, the runtime to boot and the application to initialise, and the last of those three is usually the largest and is entirely yours: a framework that scans the classpath, a dependency tree that runs code at import, a client that opens a connection pool at module scope. Interpreted runtimes typically start in tens to low hundreds of milliseconds; a JVM or .NET application doing real initialisation can take seconds. The mitigations are provisioned concurrency or a minimum instance count, which work by keeping instances warm, which is to say by giving up scale to zero and paying for idle after all.

The cost crossover is a duty cycle rather than a request count, and it is arithmetic anyone can do before choosing. Per-invocation pricing charges memory multiplied by execution time, so a function allocated one gigabyte and busy for a whole month is billed for roughly 2.6 million gigabyte-seconds, which is the same order of magnitude as simply renting a small instance for that month. Below roughly a fifth to a third of full utilisation serverless is comfortably cheaper; well above it, always-on capacity is, and the exact crossover moves with the provider's prices, which is the argument for computing it rather than inheriting an opinion about it.

Then there are the constraints that decide feasibility rather than cost. Execution has a hard ceiling, fifteen minutes on Lambda, so anything genuinely long-running needs a different home or needs breaking into steps. Nothing survives between invocations, so in-process caching is unreliable and any warm state is a coincidence. And instance count follows traffic, which means database connections follow traffic too, so the same property that makes scaling effortless is what exhausts a relational database's connection slots at peak, which is why a pooler is standard equipment rather than an optimisation.

So the decision follows the traffic shape and the work, not the fashion. Spiky, short, stateless and event-driven: serverless, and the operational saving is real. Steady, predictable and continuous: a container on a managed platform, where there are no cold starts and always-on capacity is the cheaper way to buy the same compute. Unusual system requirements, a specific kernel, a GPU, licensed software, a process that must not be restarted: a virtual machine, and accept the operational work as the price of the control. Most real systems contain all three, chosen per workload, which is the correct answer rather than a failure to standardise.