Picture yourself in a GP appointment with your parent on one side and your teenager on the other, trying to remember what the doctor just said. Plenty of people now go home and ask an AI chatbot to explain it. So how do we know whether those chatbots give safe, sensible answers? That is the question OpenAI’s HealthBench tries to answer.

The problem with exam-style tests

For years, medical AI was judged like a student sitting a multiple-choice exam. A model might score well on a medical licensing test and still struggle when a real person types something vague like “I just feel off”.

Real health conversations are messy. People leave things out, change their story and ask the wrong question. A good answer often starts with a question back, or a clear “you need to be seen today”.

What HealthBench is

HealthBench is a set of tests published by OpenAI, the company behind ChatGPT, in May 2025. Instead of exam questions, it uses 5,000 realistic conversations between an AI and a patient, carer or health worker. Many go back and forth over several turns, and they cover many languages.

It was built with 262 doctors who have practised in 60 countries. For each conversation, doctors wrote a checklist of what a good answer should and should not do. Across the whole set there are more than 48,000 of these checklist points.

The conversations cover themes such as:

  • knowing when to send someone to emergency care;
  • answering safely when the facts are uncertain;
  • asking for missing context before answering;
  • pitching the answer to the person, whether a worried parent or a nurse.

There is also a harder subset, HealthBench Hard, of 1,000 conversations that leading models still find difficult.

How did AI models score?

In OpenAI’s paper, scores rose sharply between generations. GPT-3.5 Turbo scored 16%, GPT-4o scored 32%, and OpenAI’s o3 model reached 60%. On HealthBench Hard, the best model scored only 32%. OpenAI also said a small, cheap model, GPT-4.1 nano, beat GPT-4o at a fraction of the cost.

So the models are improving quickly, but there is still a long way to go, especially on the hardest cases. OpenAI itself says the weakest areas are seeking context when a question is vague, and reliability in the worst cases.

Who marks the AI’s homework?

Here is the part I think matters most. The answers are not marked by doctors one by one. They are marked by another AI model, GPT-4.1, using the doctors’ checklists. OpenAI checked this grader against doctors’ own judgements and found the two agreed about as often as doctors agree with each other.

That is reassuring up to a point. But it is still a company marking tests it designed, for models it sells, using its own AI as the examiner. Independent checking matters.

To its credit, OpenAI has published the test and its data openly, so researchers and companies elsewhere can run it, find faults and hold it to account.

That openness has already paid off. David Gilbertson, writing on the tech blog ITNEXT, used AI tools to hunt for mistakes in the doctors’ checklists. He found examples where points looked wrong, such as credit for suggesting herbal teas for a two-month-old baby. He thought only a few hundred of the roughly 400,000 points were affected, so the overall effect was small. But it shows why outside checking matters.

Why should you care?

If you are a parent, a carer or just trying to keep your family well, AI in health care will reach you, whether through a chatbot, an app or a tool your GP practice uses. It could help explain a test result or a hospital letter.

But only if it is built and tested properly. HealthBench is not perfect, and it is not a safety certificate. It is a useful step towards judging AI on the kind of conversations real people have.

For now, treat any chatbot’s health answer as a starting point, not a diagnosis. My guide to symptom checkers explains how to use online tools safely, and when to go to the right NHS service instead.

Further reading

Update, October 2026

  • Newer models, OpenAI’s own claims. When OpenAI launched GPT-5 in 2025, it said the new model did much better on HealthBench than GPT-4o, and made far fewer factual errors on health questions. These were OpenAI’s own figures, not independent tests.
  • ChatGPT Health. In January 2026 OpenAI launched ChatGPT Health, a separate space in ChatGPT for health questions that can use medical records and fitness app data. OpenAI said it is not meant for diagnosis or treatment. At launch it was not available in the UK, the European Economic Area or Switzerland.