shipping production AI · since 2026 NAICS 541330 / 541511 / 541512 / 541519  ·  CMMC-aware
Refinery Report / Cybersecurity / post · buyers
CybersecurityVendor RiskThird-Party RiskProcurement

Vendor Security Review Checklist for SaaS Buyers

How to review a SaaS vendor's security before signing: tiering by data and dependency, reading an audit report properly, and the contract terms that outlive the questionnaire.

D
By the DSE practice team
Operator-led practice · how we research & review
July 19, 2026
12 min · 2,653 words

By the DSE practice team · published July 19, 2026 · reviewed July 19, 2026

Most SaaS security reviews are performed at the wrong time, on the wrong artifact, with the wrong outcome. The wrong time is after the business has committed and the review is a formality. The wrong artifact is a completed questionnaire, which records what the vendor says rather than what the vendor does. And the wrong outcome is a pass/fail, when the useful outcome is a set of contract terms and a monitoring cadence proportional to what the vendor can actually do to you.

This checklist covers conventional SaaS and service-provider security review. It is deliberately not about AI vendor governance — model risk, training-data handling, and foundation-model dependency are a separate assessment with different questions, covered in our third-party AI vendor risk assessment checklist for banks and fintechs.

Tier before you assess

Reviewing every vendor at the same depth is how programs die. Tier on two axes and let the tier drive the effort.

Data exposure. Does the vendor hold, process, or gain access to regulated data, customer data, credentials, or your production environment? A vendor with read access to your identity provider is a higher-consequence dependency than a vendor holding a marketing list, regardless of contract value.

Operational dependency. If the vendor is unavailable for a week, what stops? Payroll, order entry, and clinical or client delivery systems are a different tier from a design tool.

Tier Definition Review depth Reassessment
Critical Regulated or customer data, or a process that stops the business Full review, audit report read in detail, contract terms negotiated, named internal owner Annual, plus on any material change
Important Internal data, or a process with a workable manual fallback Questionnaire plus audit report review, standard contract terms Every 18–24 months
Routine No sensitive data, easily replaced Register entry, baseline contract terms On renewal

This mirrors what the framework asks for. NIST CSF 2.0 puts supplier risk in GOVERN under GV.SC, and GV.SC-04 is simply “Suppliers are known and prioritized by criticality.” Criticality first, questions second.

Read the audit report properly

An independent audit report is the strongest artifact most SaaS vendors will give you, and it is routinely mis-read. Four things to check before anything else:

  1. Scope. Which systems and services does the report actually cover? A report scoped to the vendor’s core platform tells you nothing about the acquired product you are buying.
  2. Type and period. A design-only report attests that controls were suitably designed at a point in time. An operating-effectiveness report covers a period. Confirm which one you hold and that the period is current — a report whose period ended fourteen months ago is a historical document. Ask for the bridge or gap letter covering the interval.
  3. Exceptions. Read the testing exceptions section, not the opinion. Exceptions are where the report earns its value; a report with none, in a complex environment, deserves a question rather than relief.
  4. Complementary user entity controls. Every such report contains a list of things the customer must do for the vendor’s controls to be effective — usually access management, configuration, and monitoring on your side. This section is the most consequential and the least read. Assign each item an owner in your organization.

Where a vendor has no independent report, the fallback is a completed questionnaire plus evidence for the handful of controls that matter for its tier — not an assumption that the absence is disqualifying. Early-stage vendors often have good practice and no report. What you do about it belongs in the contract.

The security questions worth asking

Keep the list short enough that the answers get read. For a critical-tier vendor:

For organizations under the FTC Safeguards Rule, this is not optional diligence: 16 CFR 314.4(f) requires taking reasonable steps to select service providers capable of maintaining appropriate safeguards, requiring those safeguards by contract, and periodically assessing providers based on the risk they present. Entities handling protected health information have parallel business-associate obligations under the HIPAA rules at 45 CFR Part 164.

The contract terms that outlive the questionnaire

A questionnaire captures a moment. A contract governs the relationship. For critical-tier vendors, these are the terms worth spending negotiating capital on:

CSF 2.0 states this directly at GV.SC-05: requirements to address cybersecurity risks in supply chains are established, prioritized, and integrated into contracts and other agreements. And GV.SC-08 is the one almost everyone omits — relevant suppliers and third parties are included in incident planning, response, and recovery activities. If your incident plan does not name who calls your critical vendors and what you will ask them for, that subcategory is unmet no matter how good the questionnaire was.

Monitoring after signature, and exit before you need it

The review is not the program. GV.SC-07 asks that supplier risk be understood, recorded, prioritized, assessed, responded to, and monitored over the course of the relationship, and GV.SC-10 asks for provisions covering what happens after a partnership or service agreement concludes. In practice, at mid-market scale:

For deeper structure, NIST publishes a CSF 2.0 Cybersecurity Supply Chain Risk Management quick start guide, and SP 800-161r1 is the full treatment for organizations that need it.

Where this fits

Vendor review is one component of a broader program, and it is usually the component that gets built after an enterprise customer asks how you oversee your own suppliers. Building it as part of a cybersecurity risk assessment — where the vendor register lands alongside the asset inventory and the risk register rather than in a separate procurement spreadsheet — is the version that stays current. Who owns keeping it current is a governance question; see vCISO vs MSP vs MDR: who owns what.

For the paid engagement that runs this across a vendor portfolio, see the vendor and SaaS security review; to baseline your own posture first, the CISA CPG baseline scorecard scores you against the same performance goals the tiering above draws on.

FAQ

What should a SaaS vendor security review include?

Tier the vendor by data exposure and operational dependency, then scale the review to the tier. For a critical vendor: read the independent audit report for scope, type and period, exceptions, and complementary user entity controls; ask a short set of questions covering data handling and retention, encryption and key custody, vendor personnel access, product authentication and audit logging, sub-processors, vulnerability management, incident notification, and resilience; negotiate contract terms; and assign a named internal owner with a reassessment cadence.

How is this different from an AI vendor risk assessment?

Conventional SaaS review addresses data handling, access, availability, and supply-chain dependency. AI vendor assessment adds model governance, training-data handling, model-change and version management, evaluation evidence, and foundation-model concentration — questions that a standard security questionnaire does not ask and that a standard audit report does not cover. They are complementary reviews with different question sets.

What if a vendor has no SOC 2 or equivalent report?

Absence of an independent report is not automatically disqualifying, particularly for smaller or earlier-stage vendors. Fall back to a completed questionnaire plus evidence for the specific controls that matter at that vendor’s tier, and handle the residual uncertainty in the contract — notification commitments, evidence rights, data return, and a documented exit path.

What contract terms matter most?

Breach notification with a defined window measured from the vendor’s discovery and mapped against your own downstream obligations; sub-processor change notice with a right to object; an annual right to current audit evidence; data return and certified deletion on termination with format and timeframe; security requirements written into the contract rather than referenced in a unilaterally editable policy page; and cooperation and log preservation during an incident.

How often should vendors be reassessed?

Critical vendors annually and on any material change; important vendors every 18 to 24 months; routine vendors at renewal. Event triggers matter more than the calendar: a publicly disclosed incident at the vendor, an acquisition of the vendor, a change in what data they hold, or a change in your own regulatory position should each re-open the review.

What this guide is / What it is not

What it is: A practitioner method for conventional SaaS and service-provider security review, anchored to NIST CSF 2.0 GV.SC-01 through GV.SC-10 (NIST CSWP 29, February 26, 2024), the NIST CSF 2.0 C-SCRM quick start guide, NIST SP 800-161r1, the FTC Safeguards Rule at 16 CFR 314.4(f), and the HIPAA rules at 45 CFR Part 164. What it is not: Legal advice, contract drafting advice, an audit, a certification, or an attestation. A vendor review is a point-in-time assessment of information the vendor provides and does not guarantee the vendor’s security, prevent a vendor breach, or satisfy any regulatory obligation on its own. Contract terms should be reviewed by qualified counsel, and whether a specific obligation applies to your organization is a legal determination.

The Bottom Line

Tier first, so effort follows consequence rather than contract value. Read the audit report for scope, period, exceptions, and the complementary user entity controls you are silently agreeing to operate. Spend your negotiating capital on notification windows, sub-processor notice, evidence rights, and data return — the terms that still matter in year three. Then assign a named internal owner per critical vendor and set a reassessment trigger, because a vendor register nobody owns is a document, not a control.

Next step · tier what you already run

Thirty minutes on the vendors that could stop your business.

Bring your vendor list and what each one touches. We will tier them by data and dependency and tell you which three need a real review before the next renewal.

Key facts

Read next · AI Security & Governance

P
Founder · Principal Engineer
Data & AI engineer · 10+ yrs hands-on

Writes most of the long-form here. Lives in the codebase. Active on GitHub and LinkedIn.

§ Next step

Not sure which of these is you?

Tell us what's broken in a paragraph and a principal reads it directly — or walk the ladder from a low-commitment first engagement up to retained work.

One long-form a week. No marketing.

Subscribe to the Refinery Report. Practitioner deep-dives on AI engineering, security, and the realities of running production systems. Unsubscribe in one click.

~12 issues / quarter