AI Applications
How to Set Up an AI Centre of Excellence (Without It Becoming a Slide Deck)
A practical build guide for an AI Centre of Excellence: who sits on the team, which operating model to choose, what to ship in the first 90 days, and the governance that keeps it alive without slowing it down.
CSEWhy ·
A practical build guide for an AI Centre of Excellence: who sits on the team, which operating model to choose, what to ship in the first 90 days, and the governance that keeps it alive without slowing it down.
The short answer, then the reasoning
An AI Centre of Excellence is a small central team, usually five to nine people, that owns AI standards, shared tooling, vendor decisions and a ranked pipeline of use cases for everyone else in the organisation. You set one up in four moves: name one accountable owner with a real budget line, pick three narrowly scoped use cases that each have a named business sponsor, ship one of them to production inside 90 days, and write the governance handbook only after that. Do it in that order.
Most CoEs are built in reverse. A steering committee forms, a charter gets drafted, a vendor shortlist appears, and nine months later there is an impressive document and no working system. The organisation concludes AI is overhyped. What actually happened is that nobody was ever on the hook for a number.
The unit of credibility in year one is a shipped workflow that a business owner will defend in a budget review. Not a pilot. Not a proof of concept sitting in a sandbox. Something people use on a Tuesday because the old way is now slower.
Who is actually on the team
Headcount is where most CoE plans go wrong in both directions. Twelve people is a department that will spend its first quarter defining itself. Two people is a side project. Five to nine, with roughly half of them part time from existing functions, is the shape that works for a company between 200 and 5,000 employees.
The lead should come from the business side, not from IT. This is counterintuitive and it matters. An IT-led CoE optimises for platform consistency and ends up building infrastructure nobody asked for. A business-led CoE optimises for a P&L line and pulls IT in when it hits a wall, which is the correct direction of pull.
| Role | What they own | Commitment |
|---|---|---|
| CoE lead (from the business) | Use case pipeline, budget, exec reporting, saying no | Full time |
| AI / solutions engineer | Builds, prompts, evaluations, integrations | 1 to 2 full time |
| Data engineer | Access, pipelines, data quality, retrieval layers | Half to full time |
| Risk, legal or compliance partner | Risk tiering, DPDP and contract review, audit trail | Embedded, roughly a day a week |
| Enablement lead | Training, documentation, adoption tracking | Half time |
| Business sponsor per use case | The number the use case is supposed to move | A few hours a week each |
Pick an operating model before you pick a tool
Three models exist in practice, and the choice is mostly determined by how centralised your data and your decision-making already are. Pretending to be federated when every data request still routes through one team will stall you for months.
| Model | Works when | How it fails |
|---|---|---|
| Centralised | You are starting from zero, data is messy, under ~1,000 employees | Becomes a bottleneck and a queue; business units route around it |
| Hub and spoke | You have 3+ functions with distinct workflows and some technical depth | Spokes drift into incompatible tools unless the hub owns standards hard |
| Federated | Mature data platform, strong engineering culture across units | Duplicated spend, no shared evaluation standard, compliance gaps |
The first 90 days
Treat this as a delivery plan, not a discovery phase. The single most useful filter for a first use case: high volume, low variance, tolerant of a human check before anything leaves the building. Invoice processing, support ticket triage, RFP first drafts, claims pre-screening, sales call summarisation into CRM fields. Boring and measurable beats visionary and vague.
One thing worth being blunt about: your first use case should not be customer facing. Pick something internal where an error costs an hour, not a customer.
- Days 1 to 15: name the lead, confirm the budget, interview eight to twelve process owners, and list every repetitive workflow they describe.
- Days 16 to 30: score the list on volume, error cost, data availability and sponsor enthusiasm. Pick three. Kill the rest publicly so people know the pipeline is real.
- Days 31 to 60: build one. Set an evaluation set of 50 to 100 real examples before you write a line of integration code, so you can tell improvement from vibes.
- Days 61 to 80: run it in parallel with the existing process. Measure handling time, error rate and how often humans override the output.
- Days 81 to 90: ship it to one team, publish the numbers internally including what went wrong, and open the intake form for the next round.
The skills gap that quietly sinks the whole thing
Here is the pattern I keep seeing. The CoE ships its first workflow, the numbers look good, and then adoption flatlines because the twenty people who actually have to use it were handed a tool and a PDF. Meanwhile the two engineers who built it become a permanent queue, because nobody else in the organisation can build anything.
A CoE is a capability-transfer function, not a build shop. If your enablement plan is a one-hour onboarding session, you have built a dependency rather than a centre of excellence. The fix is unglamorous: a working group of fifteen to thirty people across functions who learn to scope, prompt, evaluate and automate their own processes, with the CoE reviewing their work rather than doing it.
That is usually the point where external help earns its keep, because internal teams rarely have time to design a curriculum while also shipping. Mentor-led formats work better than recorded courses here, which is why our AI Masterclass sessions are built around participants bringing their own live workflows rather than sample datasets. If the constraint is build capacity instead of skills, the faster route is to have the first two or three workflows built with you through AI Automations while your team learns the pattern on a real system. If you are not sure which of the two your organisation actually needs, send us your first three candidate use cases and we will tell you which are worth building and which are not.
Governance that does not strangle the thing it protects
Write the handbook after your first production system, because only then do you know what the real risks look like. Keep it to four artefacts: a one-page intake form, a three-tier risk classification, a model and tool register, and a human-review rule for anything that reaches a customer or a regulator. That is enough for year one.
Under India's DPDP Act, personal data flowing into a third-party model is a processing decision with consequences, so your register needs to record what data each system touches and where it goes. Vendors will tell you their enterprise tier does not train on your inputs. Get it in the contract.
Four things kill CoEs, and all of them are visible early: no budget of its own, no business sponsor with a number attached, governance written before anything shipped, and a pipeline that nobody is allowed to say no to. If your CoE has approved every request it has received, it is a service desk.
The test at month twelve is simple. Can a function head point at a process that runs differently now and tell you what it saved? If yes, you have a Centre of Excellence. If the answer is a roadmap, you have a committee.
FAQs
1. How much does it cost to run an AI Centre of Excellence in the first year?
For a mid-sized company, most of the cost is people: two to three dedicated salaries plus part-time time from existing staff. Tooling and model spend for early internal use cases is usually small by comparison, often a few thousand dollars a month, and the bigger hidden cost is integration work into existing systems.
2. What is the difference between an AI CoE and a data science team?
A data science team builds models and analysis; an AI CoE owns adoption, standards, vendor choices and the pipeline of business use cases across functions. Many organisations already have the second skill set sitting inside the first team, but without a mandate or budget to change how other departments work.
3. Should the AI Centre of Excellence report to IT or to the business?
Reporting into the business, typically to a COO, CFO or strategy function, works better in year one because the CoE is judged on process outcomes rather than platform stability. IT should be a mandatory partner on security, data access and integration, with veto rights on risk but not on the use case pipeline.
4. How many people do you need to start an AI CoE?
Three dedicated people can start it: a lead with budget authority, one engineer who can build and evaluate, and one person who owns data access. Everyone else can be part time from existing functions until the first production workflow proves the model.
5. What should the first AI use case be?
Something internal, high volume, low variance and tolerant of a human check before output leaves the organisation, such as document extraction, ticket triage or meeting-to-CRM summarisation. Avoid customer-facing systems first, because an early public failure costs more trust than the win is worth.
6. When should a company shut down its AI Centre of Excellence?
When individual business units can scope, build and govern their own AI workflows to the same standard, the central team should shrink into a small standards and risk function. A CoE that still exists in its original form after four years is usually protecting its own headcount rather than transferring capability.