Audit Logs Are Not Evidence: Proving the Compliance Work Happened

An audit request is almost never "is this vendor compliant."

It is "walk me through how you determined this vendor was compliant in Q3 of last year." Those sound like the same question and they are not. The first has an answer in your system. The second requires the reasoning that produced it, and reasoning is the thing programs discard.

What a log records and what it omits

Most compliance systems have audit logging, and it does what audit logging does. A status changed from pending to verified. A user updated a field. A document was uploaded on a date. A reminder fired.

Reconstruct a compliance conclusion from that and you get a sequence of state transitions with no argument in it. You can see that someone marked a vendor's insurance verified on 14 March. You cannot see which clause set the requirement, what the certificate showed, whether the limits were compared to the contract or to a template, or whether the person doing it had the agreement open.

That gap is invisible right up until somebody external asks, at which point the request escalates from a document pull to a series of interviews with whoever was doing the job at the time - and that person has frequently moved on. The reconstruction is then assembled from inference, which is both expensive and weaker than it needed to be.

Evidence has five parts

A defensible record of a compliance check contains five things, and dropping any one of them breaks the chain.

The obligation, as written, with the clause and page it came from - not a category label like "insurance" but the specific requirement that specific agreement imposes.

The request, if evidence had to be collected: who was asked, when, how many times, and what was asked for.

The artifact that came back, stored as received rather than transcribed into fields, because the transcription is an interpretation and the auditor wants the source.

The comparison: what the artifact was tested against, and on which attributes. A certificate meeting a $5 million per-occurrence requirement is a different finding from a certificate merely being current.

The conclusion and its author, whether that is a person or a system, with the reasoning visible enough that a reviewer can independently agree or disagree.

Store those five and an audit request becomes retrieval. Store status alone and it becomes a project.

The retention problem people discover late

Evidence has to survive longer than the relationship. For Medicare Advantage, 42 CFR 422.504(i)(2)(iv) requires plans to make their first tier, downstream and related entities agree that HHS, the Comptroller General, or their designees may inspect, evaluate and audit through 10 years from the final date of the contract period or from the completion of any audit, whichever is later. That is an inspection right, not a retention mandate. The regulation's own retention requirement, at 422.504(d), covers ten years of accounting records rather than compliance evidence generally, so the ten-year figure everyone quotes is a right first and an obligation only by implication. It is an implication with teeth, and the clock has two ways of restarting: 422.504(e)(4)(ii) extends retention to six years from the final resolution of a termination, dispute or fraud allegation, and (e)(4)(iii) allows inspection at any time where CMS finds a reasonable possibility of fraud.

The clock can restart on a date nobody can predict at signature, and it outlives the contract. A vendor offboarded in 2023 can still generate an evidence request, for an entity that may no longer exist, under an agreement nobody has opened since the offboarding.

Meanwhile the operational instinct at offboarding is to close the record and revoke access. The vendor disappears from the vendor management system, which is exactly the moment the surviving evidence obligations become invisible. If you want a single test of audit readiness, ask what in your stack holds evidence for a vendor you stopped using three years ago.

Why this got harder with automation, and then easier

There is a real risk in automating compliance verification: a system that quietly marks things compliant is worse than no system. It converts an unknown into a false certainty, and it removes the discomfort that would have made someone check.

OIG's Medicare Advantage guidance, published 3 February 2026, points at the same instinct from a different angle. On attestations it says they "may be a way to help mitigate risk, but they should be accompanied by additional compliance-related steps." An attestation is a counterparty's assertion. A green status in a dashboard is your own system's assertion. Neither is evidence, and both feel like it.

The upside is that automation makes the five-part record cheap, where manually it was prohibitive. Nobody was going to hand-document the clause, the request history, the artifact, the comparison and the reasoning across a vendor book of the size Ponemon found in 2019, where the average healthcare provider carried 1,320 vendors, at a dozen recurring obligations each. A system doing the work can emit all five as a by-product, because it had to hold them to do the work at all.

What we built toward

Harper produces the record as a consequence of doing the job rather than as a separate documentation exercise.

Each obligation carries the clause, page and regulatory citation it came from. Evidence requests go to the vendor who holds the document, so the artifact arrives from the source rather than third-hand through forwarded email, and the request history is part of the record. When a document comes back it is compared against the specific clause that required it, and the comparison is shown - clause on one side, document on the other - so the conclusion is checkable rather than asserted. Anything ambiguous escalates to a person, and that decision joins the trail with its author.

Because the trail is immutable and organized by obligation rather than by system event, the answer to "how did you conclude this in Q3" is retrieval. And because obligations persist past termination, the vendor you offboarded in 2023 still has a record when someone asks in 2030.

We counted the obligations in seven real healthcare agreements to show what extraction yields; this is what happens to each one after extraction.

Test your own audit readiness in ten minutes

Pick one vendor and one recurring obligation - insurance, exclusion screening, a BAA provision, anything.

1. Produce the clause that creates the obligation, with the page.
2. Produce the most recent evidence that satisfied it.
3. Produce what that evidence was compared against.
4. Produce who or what concluded it was satisfied, and when.
5. Now do the same for a quarter two years ago.

Step 5 is the real test. Most programs can do steps 1 to 4 for the current period with some effort, and cannot do any of them for a historical one without interviewing people. That difference is the entire distance between having evidence and being able to reconstruct it, and an auditor only ever asks for the historical version.