Improving AI-Assisted Medical Diagnosis and Triage by the General Public

Description

This study is a randomized controlled trial (RCT) investigating whether access to a new LLM interface can improve medical triage and diagnostic accuracy for laypeople compared to access to a standard LLM interface. It addresses previous findings where laypeople using standard LLMs performed worse than those using conventional methods (e.g., web search) due to incomplete symptom sharing and poor interpretation of AI advice. To address this, the research tests a structured LLM system that proactively asks clinical history questions before providing a standardized, easy-to-read diagnostic output.

Interventions

• new GPT-4o Interface (Device)
  Participants can access any assistance methods they would typically employ (e.g., web search or health portals) in addition to a new LLM (GPT-4o) interface to complete medical scenarios. The new LLM interface uses a fixed system prompt that (a) instructs the model to ask targeted clarifying questions before providing any diagnostic or triage suggestions, and (b) requires all final responses to follow a structured template listing: possible conditions, approximate likelihood of each, and a recommended triage with brief reasoning.
• Control (Other)
  Participants use any assistance methods they would typically employ at home (e.g., Google or health portals) in addition to a standard LLM (GPT-4o) to complete medical scenarios. AI-overview in web searches will be disabled via an extension. They would not be allowed to access any LLMs other than the standard LLM interface.

Eligibility criteria

Conditions

Locations

• Lahore University of Management Sciences — Lahore, Punjab Province, Pakistan