- AI should not be seen as a substitute for human expertise
- AI-informed referrals require more human support, not less
RedArc has seen a significant rise in the number of people who have accessed AI-generated health information over the past year. It finds that these individuals are often overwhelmed or “ need expert clinical guidance to challenge misconceptions and provide balanced, evidence-based guidance. It believes, somewhat ironically, that greater access to online and AI-sourced health information is increasing, rather than reducing, the need for expert human support.
With the public’s accelerating use of artificial intelligence (AI) tools for everything from symptom-checking to guidance on potential diagnoses and treatment, RedArc believes that the judgement, empathy and expertise of experienced healthcare professionals are needed now more than ever.
Christine Husbands, commercial consultant, RedArc said: “One of the many challenges with health information is that people don’t know what they don’t know, so often don’t know the right questions to ask. While AI has made information more accessible, it can also take people down unhelpful paths, creating confusion rather than providing clarity. It certainly cannot yet provide the personalised care, reassurance and clinical perspective that people usually need when a significant health matter arises.
“Human expertise remains essential in ensuring people receive the correct type of support for them. Human qualities, such as empathy and judgement, together with the huge information-collation capabilities of AI is a very powerful combination.”
The risks of individuals relying on AI for health information
While RedArc recognises the benefits that AI can bring to healthcare, particularly in encouraging engagement on health matters, it believes four key areas warrant caution:
- Accountability: Unlike healthcare professionals, AI systems are not accountable for the information they provide and are not bound by professional standards, codes of conduct or clinical oversight. By contrast, nurses, such as those who operate at the heart of RedArc’s support, continue to be recognised as Britain’s most trusted profession, according to the latest Ipsos Veracity Index.
- Recycling of low-quality information: AI-generated health information is only as reliable as the content on which it is trained. As increasing volumes of AI-produced material are incorporated into online health information sources, there is a risk of inaccuracies and lower-quality content being repeatedly recycled.
- Interpretation of information: Effective support is about more than information. It requires an understanding of an individual’s circumstances, emotions and needs, a context that AI is not able to understand.
- Echo-chamber: AI responses can be shaped by the way questions are framed and can vary significantly between AI systems. In some cases, this can create a reinforcing cycle that amplifies existing concerns, without the benefit of an experienced third party to provide context, reassurance and a balanced perspective.
RedArc believes these concerns are compounded by the fact that, unlike traditional search engines, AI tools are designed to engage users in conversational exchanges that can create the impression of personalised, human-led guidance. As a result, many people, and particularly those who are vulnerable, may place undue trust in the information provided.
Christine Husbands concluded: “Increasingly, organisations need to recognise that many people will have already extensively researched their symptoms, explored possible diagnoses and even formed views on treatment options before speaking to a professional.
“It is important not to dismiss this information, but to help people understand it in the context of their own circumstances. Therefore, effective support has always been, and will remain, fundamentally human. However, as AI becomes more widely used, organisations must ensure that they are, or their chosen support provider is, equipped to help people interpret the information they find and guide them towards the most appropriate support.”