Storage and media

Routing uploads through your API means large files consume your bandwidth, your memory, your request timeouts and your autoscaling budget, to nobody's benefit. A single 2GB upload can occupy a worker for minutes, and a handful of them at once will exhaust a service sized perfectly well for its actual work.

A pre-signed URL removes the problem rather than solving it. Your server authorises the upload and returns a time-limited URL; the client sends the bytes straight to object storage. The signature encodes what may be uploaded, to which key, until when, so the permission is narrow and expiring rather than a credential handed to a browser.

The constraints belong in the signature, not in a check afterwards. Sign a specific key so a client cannot choose where the object lands, set a content length range so nobody uploads a hundred gigabytes, pin the content type, and keep the expiry short, minutes rather than hours. Anything you fail to constrain is something the holder of that URL may do.

Completion has to be discovered rather than trusted. The client telling your API that it finished is a claim, and a malicious or merely buggy client can make it without uploading anything. The reliable signal is an event from the storage service itself, which is also what lets you verify size, type and checksum before treating the object as real.

Two more things worth building in. Multipart upload splits a large file into parts that upload in parallel and resume individually, which is what makes uploads survive a mobile connection; incomplete multipart uploads then linger and are billed, so a lifecycle rule to abort them after a few days belongs in the same change. And validate the content rather than the extension, because a file named image.png that contains something else is the oldest trick in the list.

The result is an API that stays small and fast because the large payloads never touch it. That is the general shape worth recognising: the best way to scale a piece of work is often to arrange for it not to arrive.

An uploaded video is transcoded into a ladder of resolutions and bitrates, then split into segments of a few seconds each, with a manifest listing what exists. The manifest is the whole protocol: the player reads it, decides which rendition to fetch next, and asks for one segment at a time over ordinary HTTP, which is why every CDN can serve video without knowing anything about video.

The player measures throughput and switches renditions between segments, so a connection that degrades produces lower quality rather than a stall. That is adaptive bitrate, and it is the reason streaming survives a train tunnel when a single progressive MP4 does not. HLS and DASH differ in details and agree on this design.

Transcoding is expensive and it is paid once per title against a saving on every playback, which is why it is worth doing carefully. A fixed ladder applied to every video is wrong in both directions at once: it wastes bits on an animation with flat colour and starves a grainy night scene. Per-title and per-shot encoding measure the content and choose the ladder for it, and Netflix has published savings in the region of 30 per cent at equal quality from exactly that.

The startup sequence is what users judge, and it is a specific engineering problem rather than a consequence of good encoding. Start on a low rendition so playback begins quickly, then climb once throughput is known. Keep the first segments short so the first frame arrives sooner. Preload the manifest and the first segment when intent is obvious. Time to first frame is the metric that correlates with abandonment, not average bitrate.

Live is a different problem wearing the same clothes. Segment duration sets the floor on latency, since a segment cannot be served until it exists, so ordinary HLS sits several seconds behind. Low-latency variants deliver partial segments to cut that, at the cost of more requests and more complexity in the packager, and the question to ask is whether the product needs seconds or needs to be honest that it does not. Then storage and rights. Every rendition of every title is another copy, so a ladder of six renditions is six times the storage before any language or subtitle variant. Digital rights management adds licence servers and encrypted segments, and it is a business requirement rather than a technical one, which is worth saying out loud because it is the part that most complicates an otherwise clean pipeline.