Skip to main content
Compliance & AI policy

What our SOC 2 Type II audit found that surprised us

A frank summary of the findings that came out of our most recent SOC 2 Type II audit, what we expected, what we did not, and what other behavioral-health AI vendors should be ready for.

We finished our most recent SOC 2 Type II audit cycle in February. This is our second full cycle. The certificate is on the trust center, the gap analysis is closed, and the controls are in place. None of that is interesting. What was useful for me, as the engineer responsible for the controls, was the set of findings that I did not expect going in. Those are the ones worth writing about, because they are the ones a behavioral-health AI vendor reading this can actually learn from.

What follows is a frank summary. I have left out anything specific to our auditor or anything that would compromise control evidence, but the patterns are real, and several of them generalize across the category.

Background, briefly

For readers who are not in this every day: SOC 2 Type II is an attestation that the controls a vendor claims are in place are actually operating, observed across a window of time (90 days minimum, often 6 to 12 months). It covers five Trust Services Criteria, of which security is mandatory and the others (availability, confidentiality, processing integrity, privacy) are scoped in based on what the vendor offers. We scope in security, availability, and confidentiality. Most behavioral-health AI vendors should scope in the same three.

The audit produces a report listing the controls, the auditor’s testing, and any exceptions. Exceptions are findings. A clean report has zero exceptions. A serious report has some.

What I expected to find

Going in, I had a list of areas I worried about. Two stood out.

First, evidence of code-review approvals on AI-related changes. Every code change we ship has at least one approving review documented in GitHub, but the AI components have a faster cadence than the rest of the platform, and the audit window included a stretch where two engineers were doing most of the AI work. I expected the auditor to ding us for self-approvals or thin reviews on AI changes. Reasonable thing to worry about.

Second, the access-review process for production. We do quarterly access reviews; I had spot-checked them and they looked fine. But quarterly reviews on a fast-growing engineering team always have a stale row or two by the end of the quarter, and SOC 2 auditors are good at finding those rows.

What was actually found

Both of the things I expected came up clean. The code-review evidence was complete because we do not allow self-approval and the cadence does not change that. The access reviews were clean because our offboarding process closes access on the last day, not on the next quarterly review.

The findings that came up were the ones I had not put on my list.

The first was about backup restoration tests. We test restores; we have always tested restores. What we did not document well was the result of each restore test: who ran it, when, what the test data was, and whether the restore was bit-for-bit verified. The auditor wanted a logged record per test, not “we test quarterly.” Fair finding. Easy to remediate. We now write a structured log entry per restore test with the verification result and the operator.

The second was about service-account credential rotation. Service accounts that touch the production database had a documented rotation schedule, and the schedule was being followed, but the documentation lived in a runbook that was not in the standard “Policies” folder the auditor asked for. So the rotation was happening; the policy stating that it had to happen was in the wrong place. Findings like this are SOC 2 in a nutshell. The control was operating. The documentation was orphaned. We moved the policy and added a quarterly link-check to the policy index.

The third was the most interesting one, and the most generalizable. Our incident-response runbook covered security incidents, availability incidents, and privacy incidents. It did not have a separate path for AI-output incidents, the case where the model produces output that creates a clinical or compliance problem. The auditor read this as a gap, given the product category. We agreed. We added an AI-output incident path with explicit triage criteria, evidence-collection steps, and customer-notification rules. This change has since become part of our ISO/IEC 42001 management system as well.

What other vendors should be ready for

Three patterns I think generalize:

The auditor will look harder at AI-specific controls in 2026 than they did in 2024. The default audit checklists are catching up. If your incident-response process treats AI-output incidents the same as ordinary bugs, that will be flagged. Add the explicit path now.

Restore-test evidence is going to be requested in detail. Quarterly restore drills are not enough. Log the test, log the result, and log the verification.

Documentation drift is the most common finding. The control is in place; the policy is in the wrong folder. Audit your policy index annually as a self-assessment. Most findings I have seen across vendor reports are documentation drift, not control failure.

Why we publish this

Behavioral-health AI is a category where buyers are doing more rigorous security review than they used to, and the buyers I speak with are tired of the “we are SOC 2 compliant” line that does not say what is actually in the report. I would rather show the work. Our certificate, scope, and most recent attestation date are on the trust center. The findings above are the real ones from the most recent cycle. None of them changed our risk posture in a meaningful way; all of them sharpened the documentation. That is what an audit is for.

If you are a buyer evaluating us or anyone else in this category, ask vendors what their last audit found and how it was remediated. The answer (or the dodge) tells you a lot.

See it on your workflow

Twenty minutes, one mock visit. You leave with a note in your template.

We run a mock session live, draft the note, and walk through what the downstream claim would look like. No slides. No sales deck.

Live in 2 weeks or less BAA signed by default 30-day money back