The governance gap in the middle
A company with a few hundred or a couple thousand employees can have AI embedded in recruiting, sales, customer support, finance and product engineering before anyone names a program owner. Some tools were bought centrally; others arrived as features in existing subscriptions. Employees may also use public assistants outside approved workflows. The result is not a single AI initiative but dozens of small decisions made across the organization, often without a shared record of what is running or what data it touches.
That is a particularly awkward position for the US mid-market. The organization is large enough to face consequential decisions, vendor complexity and customer scrutiny, but may not have a dedicated AI office or a large legal and model-risk team. Borrowing an enterprise governance manual wholesale is not the answer: if every use case requires weeks of committee review, teams will go around the process. Doing nothing is no better. The workable path is a small, repeatable operating system that makes ownership, risk and evidence visible.
For this article, mid-market is a working segment—roughly $100 million to $1 billion in revenue or 200 to 2,000 employees—not a legal category. The advice applies beyond those bounds. The important characteristics are distributed AI adoption, limited specialist capacity and a need to show customers and leadership how decisions are made. The objective is not to slow useful AI down. It is to know where to accelerate safely and where to insist on deeper review.
Start with visibility, not a policy binder
The first question is simple: where is AI being used, by whom and for what purpose? A useful inventory includes internally built models, general-purpose assistants, AI features inside purchased tools and AI used by suppliers on the company's behalf. List the business owner, technical contact, vendor, intended use, data categories, people affected, deployment status and decision type. A record with a named owner is much more valuable than a long list of tool names with no accountability.
Do not wait for a magical discovery button. Procurement and expense records reveal purchased subscriptions; single sign-on logs show connected applications; security tools may reveal browser extensions or domains; surveys and interviews uncover embedded features and informal workflows. Each source is incomplete. Network visibility, where lawfully and appropriately available, may identify a service without telling you whether it is used for harmless drafting or a sensitive decision. Pair technical signals with a short owner interview before assigning a risk tier.
Make intake easy enough that employees choose it over silence. Ask for five minutes of basic information, acknowledge the request promptly and reserve intensive review for higher-impact uses. Do not punish a team merely for surfacing a previously unknown tool. A disclosure-friendly program sees more of reality than a punitive one. Then connect new purchases, renewals and material feature changes to the register so the first inventory does not become an obsolete spreadsheet six months later.
- 01DiscoverFind tools, features and owners
- 02AssessPrioritize by impact and data
- 03ControlApply proportionate safeguards
- 04ProveReview, document and improve
Tier the use case, then match the controls
The same model can be low risk when summarizing public product documentation and high risk when influencing hiring, credit, health or access to an essential service. Assess the use, not just the technology. Ask: who could be affected, what data enters the system, what decision follows the output, whether a person can meaningfully intervene, how serious an error could be and whether the outcome can be reversed. These questions are intelligible to a business owner and useful to counsel and security alike.
A practical model can use three operating tiers plus a prohibited-use list. Low-risk uses receive registration, acceptable-use guidance and periodic owner confirmation. Moderate-risk uses add data handling review, vendor checks and a documented human review step. High-risk uses require a fuller impact assessment, testing appropriate to the decision, explicit approval, monitoring and an escalation path. Prohibited uses should be stated plainly; leadership needs to decide where the organization will not take the risk, rather than asking reviewers to improvise the boundary case by case.
A tier must change what happens next. 'Human oversight' is not sufficient on its own: name the reviewer, define when review occurs, document their authority to override the output and retain a record. Likewise, 'bias testing' needs a method, relevant populations, a review cadence and a response when performance is unacceptable. Reassess after material changes in data, vendor, population, autonomy or purpose. A use that was low risk at launch can become high risk without the model itself changing.
Use NIST as a backbone, not a compliance shortcut
The NIST AI Risk Management Framework gives US organizations a common vocabulary: Govern, Map, Measure and Manage. Govern establishes responsibility, policies and culture. Map asks what the system does and the context in which it operates. Measure examines performance and risk with methods appropriate to that context. Manage turns findings into decisions, controls and ongoing response. It is a useful structure for an operating program because it connects policy to specific systems and decisions.
NIST AI RMF is voluntary guidance, not a certificate or a substitute for legal analysis. Mapping an inventory record or control to the framework does not automatically establish compliance with a state law. Instead, keep two connected layers: a consistent internal risk workflow built on the framework, and an obligations register maintained with counsel for the jurisdictions, industries and use cases that actually apply. The former makes the program reusable; the latter keeps legal conclusions specific.
This distinction matters when a product promises one-click compliance. A platform can surface likely obligations, route reviews and collect evidence; it cannot decide every factual question about a deployment or replace counsel's interpretation. The strongest workflow helps teams answer the right questions and shows who reviewed the answer, on what date and using which version of the policy. That is far more credible than a green badge with no reasoning behind it.
Understand the patchwork without treating every rule as universal
The US regulatory picture is use-specific and jurisdiction-specific. New York City's Local Law 144, for example, addresses covered automated employment decision tools and includes requirements around bias audits, public information and notices. It does not govern every AI tool used by every New York employer. California's AB 2013 concerns disclosures about training data for certain generative AI systems made publicly available to Californians; that is a different question from whether an employee uses a third-party assistant at work. California privacy rules concerning automated decision-making require their own applicability review.
Colorado's AI Act focuses on certain high-risk systems used in consequential decisions and has its own definitions, duties and effective-date history. State requirements can change through amendments, rulemaking and litigation. FTC enforcement under existing consumer-protection authority is another category: unsupported or deceptive AI claims and unfair practices deserve attention, but a general warning about 'AI washing' is not a universal checklist of technical obligations. Employment, privacy, sector-specific and contractual duties may overlap. A national footprint does not make every law apply to every use.
Create an obligations matrix with the triggering activity, relevant geography, responsible owner, counsel's applicability determination, control and evidence. Start with actual high-impact uses: applicant screening, employee monitoring, consumer decisions and uses involving sensitive data. Record why a rule applies—or why it does not—and revisit when the product or law changes. This is a manageable legal workflow. A giant undifferentiated list of states and acronyms is not.
Treat shadow AI as a workflow and data problem
'Shadow AI' is often described as an employee problem. It is also a design problem: if approved tools are slow to obtain, unclear to use or unsuitable for real work, employees will find alternatives. An acceptable-use policy should answer practical questions. Which tools are approved? Which classes of data may be entered? Can customer information or source code be pasted into a prompt? What should someone do if they already shared data in the wrong place? Short, concrete examples beat a blanket instruction to 'use AI responsibly.'
Security and privacy teams should agree on data boundaries for each approved tool, including retention, model-training settings, access controls, subprocessors and deletion options. Where supported, configure enterprise controls rather than relying only on awareness training. But discovery tools should be deployed proportionately and with appropriate employee privacy review; seeing that a service was accessed is not proof of a policy breach. Triage signals, confirm context and offer a safe alternative before treating every unknown domain as an incident.
Give people a clear reporting path and a fast route to approval. For a low-risk drafting use with public information, a lightweight registration may be enough. For a workflow involving confidential financial files or applicant data, security, privacy and the business owner should review the use together. This differentiation makes enforcement more credible. It also gives leadership a real measure of demand: which teams need AI, what work they want to do and where the approved catalog is failing them.
Bring vendor AI inside the boundary
A mid-sized company can have more AI exposure in purchased applications than in anything it built. An HR platform may add screening or ranking; a CRM may introduce predictive features; a support tool may summarize conversations. Even if a vendor develops the model, your organization still needs to understand what the feature does in your context. Add AI questions to existing vendor review rather than launching a disconnected diligence program.
Ask which features are enabled, whether they can be disabled, what data the vendor and its subprocessors receive, whether customer data is used for training, how performance is tested and how material changes will be announced. Match the depth of diligence to the decision's impact. A meeting-note summary tool and an applicant-ranking system should not get the same questionnaire. For higher-risk uses, document the vendor's answers, the contractual commitments, any independent evidence and the gaps you accept or remediate.
The critical moment is often after procurement. A feature can change during the contract term, or an existing vendor can enable AI where none existed during the original review. Build triggers into renewals and change notices. Link each vendor feature to an internal system owner and risk record; 'the vendor handles it' is not an ownership model. Keep an exit or disablement plan for uses that cannot tolerate an unreviewed change.
Make evidence a by-product of work
A policy tells people what should happen. Evidence shows what did happen for a particular system on a particular date. For each control, define the artifact that proves it operates: a completed assessment, approval decision, vendor response, test result, monitoring review or incident record. Assign an owner and a review date. A control with no observable proof is difficult to operate and harder to defend when a customer, insurer or board asks for specifics.
Design the workflow so the artifact appears naturally. When a high-risk use is approved, the decision record should capture who approved it, conditions imposed and the evidence reviewed. When an owner confirms a system record, save the date and what changed. When a finding is opened, link the remediation task to the control and the system. This is not document hoarding: collect the smallest set of material needed to reconstruct a decision and show that safeguards remained in place.
Make freshness visible. An assessment from launch may no longer describe today's data or workflow; a vendor response may predate a product update. Set review dates appropriate to risk and flag stale evidence. A board report should show not only how many systems are registered, but how many high-impact uses have current approvals, unresolved findings and overdue reviews. Trends over time are more useful than a reassuring one-time snapshot.
A 90-day path to an operating program
In the first 30 days, name an accountable executive and a small working group from legal, security, technology and the business. Publish a simple intake form and acceptable-use interim guidance. Pull together a first inventory from procurement, SSO, expense records and owner interviews. Do not aim for perfect completeness; identify the systems most likely to affect people, sensitive data or critical processes. Write down what remains unknown.
In days 31 through 60, establish tiering criteria and a short prohibited-use list. Review the highest-impact systems first. Assign owners, record the decision each system supports and connect the relevant vendor and data information. Have counsel identify the laws and contractual duties that apply to those uses. Define approval steps and the minimum evidence for each tier. If a high-risk system cannot meet the standard immediately, record the gap, interim safeguard and decision maker rather than pretending the issue has vanished.
In days 61 through 90, put the loop on a cadence. Set recertification dates, connect procurement and change management to intake, and produce a short leadership report showing inventory coverage, high-risk approvals, open remediation and evidence freshness. Test the process with one new use case and one changed vendor feature. Ask the requesters where it felt slow or confusing, then remove unnecessary steps. The goal at day 90 is not a finished manual; it is a routine the organization can continue next month.
What to measure—and what to avoid
The best early metrics show whether the program sees and handles reality: percentage of identified systems with a named owner; percentage of high-risk uses with a current assessment and approval; median intake time by tier; overdue reviews; open findings by age; and proportion of priority controls backed by current evidence. Define each metric before publishing it. A rising inventory count may signal better discovery, not deteriorating governance. A zero-incident report may mean no one knows where to report incidents.
Avoid buying complexity before establishing ownership. A platform can unify records, route reviews and make reporting easier, but no dashboard can decide risk appetite for leadership or resolve ambiguous legal facts. Similarly, do not promise automatic detection of every employee prompt or one-click compliance across all states. Start with a workflow that people trust, then automate repetitive handoffs and reminders. If teams can register a use, understand its tier, get a decision and find the supporting evidence, the foundation is working.
The mid-market advantage is the ability to move with less ceremony. A focused program can be rigorous without being heavy: one inventory, one tiering model, clear ownership, specific legal mapping and a repeatable evidence trail. That combination turns AI governance from an annual presentation into a living business practice—and gives teams more confidence to use AI where it genuinely helps.
Sources & further reading
Regulatory requirements depend on the facts and may change. Consult counsel for applicability to your organization.
Put a practical AI governance program in motion
GovernIQX brings inventory, assessments, controls and evidence into one operating view. Explore the platform or talk through where your organization should begin.
