
The Monthly Intelligence Report
What gives an AI decision its evidence?
Caribbean reporting on disaster alerts, AI usage and the checks a business can use before buying a tool.
A road closure notice gives you somewhere to start. Before an AI assistant tells a resident to use another route, someone needs to check when the notice changed, which road it covers and who can confirm a diversion. A hotel testing automated booking replies faces a similar question. What record gives the assistant enough evidence to offer an answer the business can act on?
A warning starts with a record someone can verify
What the disaster discussion asked of AI
On 23 September, UNESCO reported on a Caribbean disaster-resilience discussion that it joined with the University of the West Indies Disaster Risk Reduction Centre on 22 July. Participants considered using AI to map risk, organise information for responders and improve early warnings. They raised a prior question about whether communities have current data, trust the channel that carries a warning and can act when it arrives. “Do you even need AI?” asked Leona Verdadero, a UNESCO programme specialist working on digital policy and transformation. UNESCO’s account of the discussion.
Take a road closure as a test case. Start with a dated notice from the responsible authority. Keep the issuer and the affected stretch of road attached when the assistant produces a plain-language update. Then give it an older notice and a report from a resident that conflicts with the official one. Check whether staff can see the conflict without combing through the conversation. If they can’t, the tool may speed up the work of writing a notice while making it harder to check the decision behind it.
For the drill, ask the person who sends public messages to approve the alert. Also ask someone who lives on the route whether the instructions name a road they recognise and give them a way to check for a newer notice. The agency should test how it will issue an update if the tool or its usual message channel is unavailable.
The usage index measures a particular kind of attention
What the scores tell a buyer
The Caribbean AI Model Usage Index covered January to August 2026 across 21 territories. It combined sources such as search interest and app-store rankings. ChatGPT received 58 of the index’s 100 points and Gemini received 30. The scores give a buyer one way to compare tools. For household usage, paid subscriptions or the quality of work a tool produces, the buyer needs other measures. Caribbean AI Model Usage Index 2026.
A buyer can use the ranking to shortlist products, then ask staff which ones they use. A purchasing decision still needs a test on the buyer’s own work. A hotel can give each shortlisted assistant a room-change request, a copy of the rules on the booking and an old rate card. The reviewer checks whether the draft names the current rule, asks about missing details and stays within the refund terms the hotel has approved. Save the inputs and the reviewer’s corrections. Use the record to decide which assistant fits the front desk’s work.
Give one business task a finish line
Measure the work after the first draft
At the Manchester Chamber of Commerce on 10 September, CAIA President Adrian Dunkley presented Northstar AI, a talk about finding a business use for AI and measuring what happens after adoption. It gives firms a place to start with a bounded task and a way to look at the cost of bringing it into daily work. Northstar AI presentation and practical plan.
For a small firm, choose one kind of customer email that already has a clear answer in the company’s records. Time the work from the moment the email arrives to the moment an approved reply goes out. Include the minutes staff spend finding the source, correcting the draft and dealing with any follow-up the reply creates. Use a comparable set of cases for the assisted trial. Mark which source the reviewer opened and which edits changed the answer customers received.
If drafting becomes faster, check what happened to the queue. A staff member may have gained time on each reply and still have more unanswered emails at the end of the day because the approval step filled up. Give the person who handles the queue authority to slow down the trial or change the sample if errors start to reach customers.
Get the supplier to show how a decision is recorded
Ask for one difficult case during a demonstration
At a procurement demonstration, give the supplier two versions of a quote, each with a different date or currency. Ask them to show how the tool selects a record and where the reviewer can see that choice. Then remove a document and check whether the draft names what is missing. A response that looks finished may conceal a gap the buyer needs to know about.
Require the supplier to export a usable record of the test. It should identify the version of the tool, the records it used and whether a person changed the decision. Ask which staff role can stop the process if a price, quantity or payment term is wrong. This creates something a second team can inspect before it uses the same system with a different supplier. NIST’s risk-management profile for generative AI offers a reference for documenting tests and assigning responsibility. NIST Generative Artificial Intelligence Profile.
What belongs on your next review agenda
Bring a case someone else can retrace
Start with the last case that made a reviewer hesitate. Bring the version of the source the assistant used, the correction made by staff and the time that correction took. Ask a colleague who didn’t work on it to find the same evidence. If they have to ask the original reviewer where to look, fix the record before expanding the trial.
For public agencies, bring the person authorised to issue or withdraw a message into that discussion. For a business, bring the person who can hold an order or change a booking. Give them the first choice of which error to put into the next drill. They will have to answer for that decision when the tool misses it.