Storage is the cheap part

When we started designing how Lethra holds a document, the retention period turned out to be one database column. Everything difficult was somewhere else.

That surprised us less than it surprises the teams we talk to, but only because we had already spent time on the cost model. Most teams approach long-term retention the way you would approach any storage question: pick a bucket, set a lifecycle rule, multiply by cost per terabyte, move on to something interesting. It is a reasonable instinct and it produces a number that looks like it settles the matter.

The number is real. It is also almost irrelevant.

The arithmetic that ends the storage conversation

Take a platform generating enough volume for retention obligations to matter — call it a few thousand documents a month across a handful of partners, averaging a few hundred kilobytes each once you account for the fact that most compliance artefacts are text and metadata rather than media.

That accumulates on the order of tens of gigabytes a month. Sustained across the full obligation window, and assuming nothing is ever deleted early, the steady state lands in the low tens of terabytes.

At European object storage prices, the whole archive costs less per month than a mid-tier SaaS subscription. Applying storage tiering — hot for the recent, warm for the middle, cold archive for the long tail that statistically nobody will ever retrieve — cuts it further, to a figure that is genuinely difficult to care about. Against the revenue the same archive represents, storage runs to roughly two hundredths of one percent of cost of goods.

Storage is not the expensive part of this problem. Compliance is, key custody is, and the engineering to make either defensible is. The bucket is the cheapest thing in the system, and a design that optimises it has optimised the wrong variable.

Four properties that a bucket does not give you

What took the actual work were four properties, none of which storage provides and all of which the obligation implies.

The document has to be unchangeable in a way you can demonstrate

Not we have a policy against editing it. Not only two people hold write access. Both of those are statements about intent and organisational discipline, and both are exactly as strong as the weakest credential in the system on the worst day.

Consider the position honestly. A file sits in a bucket. An engineer with production credentials could overwrite it and nothing anywhere would record that this happened. The file's history rests entirely on trust in that engineer, and however good the engineer is, that is a weak position to defend from — not because they would, but because you cannot demonstrate that they did not.

The distinction that matters is between prevented and detectable. Prevention is hard in a system that has to write the object in the first place. Detection is achievable: if the content is hashed at the moment of arrival, and that hash is committed somewhere append-only, then a later modification is not prevented but it is provable. That turns out to be enough, and it is considerably cheaper than the alternatives.

The history has to be readable by someone outside the system

An audit trail stored in the same database it audits is worth very little to a reader whose entire job is to doubt it. If the party who can modify the records is also the party who maintains the log of modifications, the log establishes nothing beyond what its keeper is willing to have established.

This is not a hypothetical concern about dishonesty. It is a structural point about what evidence means. A record is evidence to the degree that it could not have been produced after the fact by the party benefiting from it. An internal audit log fails that test by construction, regardless of how well it is kept.

So the trail has to be verifiable without access to us and without trusting us, which means committing to something external and doing it on a schedule that cannot be retroactively adjusted.

Destruction has to produce evidence

Deletion in a distributed system is an assertion that propagates, not an event that occurs. Replicas, snapshots, backups with their own retention policies, write-ahead logs, provider-side tiering that moved a copy somewhere cold months ago. Between the delete call and the bytes being genuinely unrecoverable sits a chain nobody outside the storage provider can inspect.

The design consequence is that proof of destruction cannot rest on a delete operation, however reliable that operation is, because the reliability is unverifiable from outside the provider. Destruction has to become something provable locally, which means making the data depend on something small enough to destroy with certainty. We seal each document under its own encryption key in a hardware-backed store, and destroying that key is what makes the remaining bytes meaningless. That mechanism deserves its own article and gets one.

The process has to be written down and kept current

This is the least interesting of the four and the one most likely to be skipped, which is presumably why it accounts for a disproportionate share of the problems teams eventually run into. A system whose behaviour is only documented in its source code is a system whose behaviour cannot be explained to anyone who does not read that source code, and the people asking are frequently not those people.

There is no clever engineering here. There is only the discipline of writing the description down and updating it when the thing changes, which is harder than it sounds because nothing breaks when you skip it.

None of these are storage problems

All four reduce to the same underlying question, which is worth stating plainly because it reframes the whole build:

Can you produce evidence that the thing in the bucket is the thing that arrived, unchanged, on the date claimed — and that the evidence itself was not assembled afterwards to suit the answer?

That question is not answered by any storage product, at any price, from any provider. It is answered by cryptographic commitment: hashing content at arrival, committing those hashes to a structure that cannot be retroactively altered without breaking, publishing the commitments where a third party can see them, and anchoring the whole thing to an external time source so that even the publication schedule is not something you control after the fact.

The technique is not new. It predates every company currently selling into this market. Certificate Transparency has used it to keep the public certificate authorities honest for years. Software supply chain projects use the same pattern to make build provenance checkable. It is well understood, auditor-legible, and — this is the part that took the design work — entirely absent from the tooling available to a platform team who needs it for documents rather than for certificates or builds.

What this means for how you evaluate

If you are assessing options for an obligation like this, the storage question is the one to get out of the way first, precisely because it is easy. Any competent provider will hold the bytes. The bytes are not at risk and they are not expensive.

The questions worth spending evaluation time on are the other four. How is modification made detectable, and by what mechanism rather than what policy. Who can verify the history, and what do they need from you in order to do it. What does destruction produce besides a status code. And is there a written description of the process that someone outside the engineering team could actually follow.

A provider who answers the first question well and deflects the other four has sold you storage with better marketing. That is a real product and there is nothing wrong with buying it, as long as you know that is what you bought.