Six AI questions every board should ask management

Boards do not need to understand how a model works to oversee how a company uses AI. They need to ask the right questions and recognize a weak answer when they hear one. These six questions cover what directors are accountable for: knowing where AI is used, which uses matter most, what would cause management to stop, whether the systems still work, what vendors do with company data, and what independent assurance actually covers.

Why directors should ask, not build

Leading IT and cybersecurity audits in regulated financial institutions taught me that the most useful questions about technology risk are rarely technical. They are simple questions that management can only answer well if the program underneath is real.

AI is no different. A director does not need to know what a transformer is or how retrieval works. A director does need to know whether management can name its AI uses, explain which ones could harm customers or the company, and show evidence that someone is watching them. The questions below are designed to surface that, in plain language.

1. Where is AI used, and who owns each use?

Why it matters. The board cannot oversee what management cannot list. AI arrives through many routes: internal builds, vendor platforms, features added to existing software, and tools employees adopt on their own. Without an inventory, every other answer is a guess.

What a good answer sounds like. “We maintain an inventory of AI use cases, including AI features in vendor products. Each entry has a named business owner. We refresh it on a set schedule and compare it against other sources, such as procurement and network data, to find gaps.” When I built an enterprise AI inventory, mapping each use case to the NIST AI Risk Management Framework and ISO/IEC 42001 gave us a consistent way to describe them, and that consistency is what lets a board compare one quarter to the next.

Red flags. No inventory. An inventory that covers only internally built models. Ownership assigned to “IT” or a committee instead of a person. A count that has not changed in a year.

2. Which uses are high-stakes, and how are they tiered?

Why it matters. Not every AI use deserves board attention. A writing assistant and a system that influences credit, hiring, or fraud decisions carry very different risk. The board should know that management has a method for telling them apart.

What a good answer sounds like. “We assign each use a risk tier based on the data it uses, who is affected by its output, and how much it influences decisions. High tier uses receive full review, testing, and ongoing monitoring. Here are the high tier uses and their owners.” Management should be able to explain the criteria in a sentence or two.

Red flags. Every use rated the same. No written criteria. High tier uses that bypassed review because they came through a vendor. Uncertainty about whether any uses fall under regulations such as the EU AI Act or, in banking, model risk management guidance like SR 11-7.

3. What would make us stop using an AI system?

Why it matters. This is the question I find most revealing. A program that has defined when it would pause or retire a system has thought seriously about failure. A program that has not is relying on someone to notice and argue for action under pressure.

What a good answer sounds like. “Each high tier use has agreed thresholds, such as error rates, complaint volume, or override rates, and defined triggers like a vendor model change or a new customer population. When a threshold is crossed, the owner must decide whether to continue, restrict, or pause, and that decision is recorded. Here is an example of when we did.”

Red flags. “We would stop if something went wrong,” with no definition of wrong. No example of anything ever being paused or retired. Uncertainty about who has the authority to stop a system.

4. How do we know it still works?

Why it matters. AI systems change after approval. Data drifts, vendors update models, and use expands beyond the original purpose. A system that performed well at launch can degrade quietly.

What a good answer sounds like. “High tier uses are monitored against the measures we agreed at approval. Owners review the results on a set schedule and sign off. Exceptions come to the risk committee. We also re-test when something material changes.” For generative AI, management might reference how it evaluates accuracy and harmful output, which the NIST Generative AI Profile (NIST AI 600-1) addresses.

Red flags. Monitoring that tracks uptime but not output quality. Reports that go to a technical team and stop there. No re-testing after vendor updates. A reliance on “the vendor monitors it.”

5. What do vendors do with our data and our models?

Why it matters. Much of a company’s AI exposure sits with third parties. Vendors may use customer data to train their models, rely on subprocessors, or change the underlying model without notice. For banks, the 2023 Interagency Guidance on Third-Party Relationships makes clear that using a vendor does not transfer the responsibility.

What a good answer sounds like. “We review AI-enabled vendors before approval. Our contracts address whether our data can be used for training, where it is processed, notice of model changes, and audit rights. We reassess when a vendor adds AI features to a product we already use.” I reviewed AI-enabled vendors before approval in my own work, and the questions I would put first are about data use, training, and change notification.

Red flags. Standard vendor reviews with no AI-specific questions. Contracts signed before AI features were added and never revisited. No one able to say whether customer data is used for model training.

6. What does assurance cover, and who provides it?

Why it matters. Boards rely on assurance to know that what management reports is true. With AI, assurance can be uneven: one system validated, another never reviewed, and a vendor report that covers the hosting environment but not the model.

What a good answer sounds like. “Second-line risk reviews high tier uses. Internal audit has included AI governance in its plan and has tested the inventory, tiering, and monitoring. Where we rely on vendor attestations or certifications, we know what they cover and what they do not.” Management and audit should be able to describe gaps honestly.

Red flags. No AI coverage in the audit plan. Heavy reliance on vendor certificates whose scope no one has read. Assurance that covers policies but never tests whether they work in practice.

The six questions at a glance

Question	What a good answer sounds like	Red flag
Where is AI used, and who owns each use?	A current inventory, including vendor features, with a named owner for each entry	No inventory, or ownership assigned to “IT”
Which uses are high-stakes, and how are they tiered?	Written tiering criteria and a list of high tier uses with owners	Every use rated the same
What would make us stop using it?	Defined thresholds and triggers, with a recorded example of acting on one	Nothing has ever been paused
How do we know it still works?	Scheduled monitoring of output quality, signed by owners, re-tested after changes	“The vendor monitors it”
What do vendors do with our data and models?	AI-specific due diligence and contract terms on training, location, and change notice	No one knows whether data is used for training
What does assurance cover, and who provides it?	Second-line and audit coverage of high tier uses, with known gaps	Vendor certificates accepted without reading the scope

Before the next board meeting

Directors who want to start this conversation can do it without a large request.

1. Ask management for the current AI inventory, with owners, at the next risk committee meeting.
2. Request the list of high tier uses and the criteria used to tier them.
3. Ask for one example of a threshold being crossed and what was decided.
4. Ask how vendor model changes are detected and reassessed.
5. Ask internal audit where AI governance sits in the audit plan.
6. Agree on how often AI risk will be reported to the board, and in what format.

Closing

Good board oversight of AI looks a lot like good oversight of any other risk. It depends on clear questions, honest answers, and evidence that the program works. Directors who ask these six questions regularly will learn quickly whether management is in control or hoping for the best.

---

This article reflects my professional perspective and is general information, not legal or regulatory advice. Apply it in light of your organization’s obligations and risk appetite.