Caribbean AI Association
Sailing boats beside Kingston’s waterfront warehouses, with mountains behind the town.

AI Practice
Ratings.

How well does your organisation use AI when someone depends on the result? CAIA investigators follow the work, from the first decision to the final check.

Apply for assessment
Kingston waterfront, Jamaica · c. 1901

What your grade tells you

A board might see how many staff have AI licences. A customer sees the answer those tools helped produce. Our review tests what happens between the two.

You receive a score out of 100, an A–F grade and a breakdown of six areas. Use the findings to choose your next investment, brief your board or show partners exactly which part of your organisation CAIA assessed.

Inside the assessment

Investigators interview staff, observe live tasks and check dated records. Open an area to see the three things they score.

AI fluencyCan staff spot a plausible wrong answer?
  • Staff can choose an appropriate AI task. Ask staff in different roles to explain when they would use AI and when they would stop.
  • Staff verify outputs. Observe a worker checking a name, number or claim against a trusted record.
  • Staff escalate mistakes. Give staff a suspect output. Record whom they contact and what they preserve.
AI adoptionWhich tools still earn their place after the pilot?
  • Live use has an owner. Match the use register to a task observed during the review.
  • The team measures actual use. Compare active use with the eligible staff or cases over a defined period.
  • The workflow improves an outcome. Compare time, cost, quality or service against a stated baseline. Check for work shifted to another team.
AI governanceWho can approve a system, change it or stop it?
  • Decision rights are assigned. Trace an approval to the named person responsible for the use.
  • Purchases and changes receive review. Sample a supplier or model change and inspect its approval record.
  • Leaders oversee exceptions. Follow a recorded exception through a review meeting to a decision and a due date.
Risk managementWhat happens when a risk becomes an incident?
  • The team identifies material risks. Check a risk register against actual uses, data and people affected.
  • Controls operate in practice. Select a high-risk use. Inspect the control, its owner and a recent test.
  • Incidents lead to action. Trace an incident or exercise from report to escalation, correction and retest.
Safety and alignmentDoes the system behave as intended for the people it affects?
  • The team defines acceptable behaviour. Inspect intended use, prohibited outcomes and measurable limits.
  • The team tests foreseeable failures. Review representative cases, including local language or user groups where relevant.
  • People can intervene and obtain a remedy. Observe an override or rollback, and trace how a person can challenge a harmful result.
InnovationWhich experiment produced a result worth repeating?
  • Experiments test a specific claim. Check the hypothesis, comparator and success measure recorded before a trial.
  • Results include costs and trade-offs. Inspect measured benefits, failure cases, review time and operating cost.
  • The team acts on what it learns. Trace a decision to adapt, stop or expand a trial and inspect the follow-up result.

From field notes to a grade

Each of the 18 checks earns 0–4 points. All checks carry the same weight. Divide the points earned by 72 and multiply by 100. For each area, divide its three-check total by 12 and multiply by 100. Use the full score for thresholds and ranking. Round to one decimal for display.

Read the scoring rules
  1. No practice found. The team had the opportunity to demonstrate it and the review confirmed its absence.
  2. Designed. A named owner and a dated plan or specification exist.
  3. In use. The assessor verifies a current operating example.
  4. Measured. Dated records show repeated use, a measure and a review of the result.
  5. Improved and retested. The team acted on the results and verified the effect in a later sample.

Use the highest level supported by the evidence, including the requirements at earlier levels. A field note records the source, date, sample and rationale for each check. If assessors cannot obtain enough evidence to decide a check, they mark it unverified. The organisation remains unrated until the evidence is sufficient.

A90–100Sustained practice
B80 to under 90Established practice
C70 to under 80Working practice
D60 to under 70Uneven practice
E50 to under 60Early practice
FBelow 50Major gaps

For example, 59 of 72 points gives 81.9 and grade B. A weak safety score can still prevent the organisation from earning a Cay Star.

In the March register, completed assessments appear in descending order of the full score. Equal scores share a position. Each entry shows the sector, size, assessed unit, six area scores, grade and Cay Stars. Compare entries with similar scope. An assessment of one department covers that department only.

Try the assessment scorecard

The Cay Star

Four compass points recall navigation between Caribbean islands. A square cut from the centre represents a chip. The same mark repeats for each distinction and works in a single colour.

One Cay Star

A verified workflow

Investigators verify one live workflow, its owner and a measured result.

Overall 70+ · Controls 60+ each

Two Cay Stars

Repeatable practice

Investigators verify results in two distinct live workflows. Each has its own evidence trail.

Overall 80+ · Controls 75+ each

Three Cay Stars

Practice others can repeat

Meet the two-star requirements, show results in two separate review periods and demonstrate that another team or setting has reproduced the practice.

Overall 90+ · Controls 85+ each

“Controls” means governance, risk management, and safety and alignment. Each must meet its own minimum. An unresolved critical harm blocks every star. This includes a live use causing or posing an immediate risk of serious harm without an effective safeguard. A completed assessment can receive zero stars.

Stars apply to the named scope and edition. Each year we reassess the evidence. An organisation can gain, retain or lose stars, or earn all three in its first assessment if earlier operating records meet the time requirement.

“Can we follow one decision from the AI output to the person who checked it?”
Field question from the CAIA assessment
Shopping district, San Juan · December 1941

From application to publication

Agree the scope and price

Tell us the size of your organisation and which teams, systems and locations to include. CAIA quotes for investigator time, evidence review and visits. You decide whether to proceed after you receive the scope, fee and publication terms.

Follow the work

The assessor selects examples from the agreed scope, including higher-risk work and exceptions. Staff demonstrate tasks, controls and measurable outcomes. A small team can show evidence from several workflows without having separate departments.

Check the decision

A second reviewer checks the evidence and scoring. A panel resolves disagreements and records its reasons. Staff who negotiate fees have no vote on the result. A reviewer with a conflict steps aside.

Review facts and publish

Before publication you receive the draft and can flag factual errors or submit dated evidence for reconsideration. A reviewer outside the original assessment considers any disputed finding. CAIA publishes the final result and verified innovation highlights at the end of March. Changes after publication carry a dated correction.

“What changed for the person doing the work, and how did you measure it?”

Innovation interview question · CAIA field guide

What informed the method?

Fitch publishes criteria that identify rating drivers and the limits of its analysis. Moody’s describes how committees use scorecards and keeps fee discussions separate from analytical decisions. Michelin uses three annual levels and reassesses recipients. CAIA applies these design principles to workplace AI. CAIA owns the criteria, scoring rules and Cay Star.

Fitch Ratings criteria · Moody’s methodologies · Moody’s analytical independence · MICHELIN Guide criteria

Apply for assessment

Send an expression of interest for your organisation. There is no charge to enquire. CAIA will contact your representative to agree the scope and quote for the paid assessment.

Use business contact details. Describe the scope briefly and keep client records and internal evidence for the assessment.

Photographs · Kingston waterfront, Detroit Publishing Co., Library of Congress · San Juan shopping district, Jack Delano, Library of Congress