A Citizen's Checklist for Evaluating an AI Medical Tool

Note from the President

The CAIRA Summit closed in Bridgetown on the eleventh. Two thousand six hundred and fourteen attendees, fifty seven sessions, the public adoption of the regional AI Charter by twelve CARICOM member states with the remaining three to follow in November, and a closing keynote from Prime Minister Mottley that has been reprinted in three regional newspapers. The Association is, today, in a different position than it was when I wrote you last month. I owe a longer reflection on what the Summit produced, and what it changed in our work, and I will write it for the November issue.

This month I asked Kwame Stevens to write the feature, because the most consistent request I heard at the Summit, from members in every country, was for practical materials that ordinary Caribbean citizens could use to evaluate the AI tools now showing up in their lives, particularly in healthcare. October is Breast Cancer Awareness Month, and the practical question of whether a particular AI tool is actually helping the patient in front of you, or marketing itself as such, is now a question that any Caribbean family can be asked to answer. Kwame has produced a checklist. It is the kind of thing every CAIRA chapter should print and hand out at the next public meeting.

Adrian Dunkley Founder and President, Caribbean AI Association

---

Feature

A Citizen's Checklist for Evaluating an AI Medical Tool

By Kwame Stevens

This piece is structured as a citizen's checklist, because the audience for it is not the consultant cardiologist at the University Hospital, who has been trained to evaluate medical claims. The audience is the parent in Kingstown whose mother has been told that her hospital uses AI to review mammograms, the nurse in Castries who has been asked to incorporate an AI assistant into her triage workflow, the small business owner in Port of Spain whose insurance company has begun citing AI in its underwriting decisions for her family policy. The question this checklist answers is, in plain words, how do I know whether to trust the thing.

A short note on framing. There is no single test that distinguishes a useful AI medical tool from a marketing exercise. There is, instead, a set of questions whose answers, in combination, will give you a defensible picture. The questions below are the ones I teach my secondary school students to ask, adapted for an adult audience. None of them require technical knowledge. All of them require asking.

Question one. What does this tool actually do.

A surprising amount of AI marketing in healthcare obscures, rather than clarifies, what the tool does in the clinical workflow. The answer should be specific and short. Examples of good answers. "This tool reads digital mammograms and flags areas the radiologist should examine more closely. The radiologist makes the final reading." "This tool predicts the probability that a patient with these symptoms will be admitted in the next twenty four hours, to help the triage nurse prioritize. The nurse decides." "This tool monitors blood glucose data from a patient's continuous monitor and alerts the patient and the clinic when readings fall outside a configured range. The clinic responds." Examples of bad answers. "This tool uses artificial intelligence to improve patient outcomes." "This tool leverages machine learning to support clinicians." If the tool's vendor or your hospital cannot describe the tool's specific clinical role in one sentence, you do not yet know what it does, and neither do they.

Question two. Who decides.

The clinical AI tools currently being marketed in our region fall into two broad categories. The first is decision support tools, which produce information that a human clinician interprets and acts on. The second is decision making tools, which produce an output that drives, directly, what happens to the patient. The difference matters. A decision support tool can be useful even when it is imperfect, because the clinician absorbs the input and acts on their own judgment. A decision making tool that is imperfect can directly harm a patient before any human is in the loop.

You are entitled to know which category any tool used in your care falls into. If your hospital uses a tool that determines, without human review, whether your case is or is not flagged for follow-up, that tool is a decision making tool by another name, and the standard you should ask about its performance is higher than the standard for decision support. The right answer to this question, in nearly every Caribbean clinical setting today, is that a human clinician retains the final decision. If that is not the case, ask why.

Question three. Where was it validated.

Every clinical AI tool that has been responsibly developed has, somewhere, a published or available record of how well it performed on the populations it was tested on. The question for the Caribbean patient is whether the populations it was tested on resemble the population that includes you.

The simple version of this question, which any patient can ask their clinician or any hospital can ask their vendor, is "what is the largest study showing this tool's accuracy on patients who look like the patients we see here." A good answer cites a study with a defined population, a sample size in the thousands, and a publication in a journal a clinician can name. A weaker answer cites a vendor white paper. A bad answer is "trust us." Most clinical AI tools currently deployed in our region have, in honesty, weak answers to this question, because the validation work on Caribbean populations has largely not been done. That is not a reason to refuse care. It is a reason to ask the question and to listen carefully to the answer.

Question four. What does it do when it is wrong.

Every AI tool is wrong some of the time. The well-designed ones tell you when they are uncertain. The badly designed ones produce a confident output regardless of how strong the underlying evidence is. The question for the patient or the citizen is how the tool's mistakes show up, and what protections are in place against them.

The framing I teach my students is to ask "what does this tool look like when it is failing, and how would I or my clinician know." Good clinical AI tools have published false positive rates, false negative rates, calibration data, and known failure modes. They also have an alert mechanism for clinicians when the tool's confidence is below a threshold. If your tool's literature does not contain these answers, the tool may still be useful, but the burden of catching its errors falls more heavily on the clinician, and the clinician should be staffed and time-resourced to bear that burden. If they are not, the deployment is not safe.

Question five. What does it do with your data.

The question of where your medical data goes when an AI tool processes it is now a regulatory question in our region. The Charter Andre wrote about in July establishes a set of standards for this. The version a patient can practically use is shorter.

Ask, where is my data processed when this tool is used. The answer should be one of three things. On the device or system inside the hospital, with no external transmission. On servers within our country or within CARICOM. On servers in a specific named foreign country. Each of these has implications for who has access to your data, what laws govern that access, and what happens if something goes wrong. The right standard is not always local processing. There are cases where the best available tool is hosted abroad and the trade-off is worth taking. The wrong situation is one where the patient was not told, and where the institution itself does not know the answer.

Question six. Who is accountable when it goes wrong.

This is the question hospitals, vendors, and regulators most want to avoid, and the one the patient most needs answered. If an AI tool used in your care contributes to a clinical error, who is responsible. The clinician who used it. The institution that deployed it. The vendor that built it. The regulator that approved it. The honest answer in most jurisdictions today is that the accountability chain is partially settled and partially contested, and that the patient's primary legal recourse remains against the clinician and the institution. The Charter has begun to clarify the vendor and regulator responsibilities. The clarification is not complete.

For a patient, the practical implication is to ensure the institution's complaint mechanism is one you can actually use. Ask, before any consequential AI assisted procedure, what the complaint mechanism is, who reviews it, and how long it takes. If the institution cannot tell you, the institution is not yet ready to deploy the tool. That is not a reason to refuse necessary care. It is a reason to escalate the question to the institution's leadership and to the CAIRA Working Group, which is collecting reports of exactly these gaps as part of our regional implementation work.

Question seven. What does the alternative look like.

The final question is the one I find most often ignored, and it is the one I would urge every Caribbean patient and every citizen to keep at the front of their mind. AI tools in healthcare are being introduced into a system that already has practices for what was done before. The question is not, in the abstract, whether the AI tool is good. The question is whether the AI tool, in your specific context, is better than the alternative practice it replaces or augments.

The alternative might be a radiologist reading without AI assistance. It might be a nurse making a triage decision based on training and experience. It might be a clinic following its existing diabetes monitoring protocol. The honest comparison is not AI versus no care. It is AI assisted care versus the standard of care that already exists. In many cases the AI tool meaningfully improves on the alternative, and in some cases it does not. The patient or citizen who is told that AI is being introduced is entitled to ask what was happening before, what changed, and what is expected to change in their outcomes.

A word for the people who deploy these tools.

If you are reading this from the position of a hospital administrator, a ministry of health official, an insurance executive, or a vendor representative, I want to address you directly. The patient is going to start asking these questions. The patient should ask these questions. The institutions that have prepared good answers will be trusted. The institutions that have not will lose trust, in some cases fairly and in some cases unfairly. The fair losses you can prevent by doing the work in advance. Publish your AI deployment register. Train your front-line staff to explain the tools in plain language. Set up the complaint mechanism. Pay for the validation work on Caribbean populations. Be the institution that the patient can ask without feeling foolish.

And a word for parents.

October is Breast Cancer Awareness Month, and several CAIRA member countries are running screening campaigns this month. If you have not had a mammogram in two years and you are forty or older, book one this month. If the women in your family have not, talk with them. The most consequential AI question in Caribbean breast cancer outcomes, in 2025, remains the question of whether women are getting screened in the first place. Everything else, including the questions in this checklist, comes after.

---

Kwame Stevens teaches computer science at a secondary school in Spanish Town and coordinates the CAIRA Learning Working Group.

Originally published in The Monthly Intelligence Report, October 2025.