Using AI to Guide AI

AI-based tools have become highly effective at detecting cardiovascular diseases, but providers still lack guidance on when to use them.

We have many effective traditional tests in medicine, but we don’t use them for everyone. Instead, we look at the patient in front of us, consider their history, symptoms, and goals of care, and determine which tests are appropriate. The same principle applies to AI tools.

However, AI models are still very new, and as a field, we have not evolved our thinking about when and how to use them. Therefore, we think there needs to be an AI-informed layer that can help clinicians identify when it is appropriate to use that AI.

In a real-world setting, where most people are healthy, overusing AI models will create many false positives, which result in unnecessary downstream costs and testing, as well as unnecessary patient anxiety.

We need to navigate this surge of AI tools in a data-driven way to make sure that we have useful guidance that best directs AI resources.

Large language models (LLMs) are trained to learn word patterns. Given a sequence of words, what is the most likely next word? That is how LLMs generate text that sounds like it was written by a human.

We do something very similar, but instead of focusing on words, we think about events. Just like any story, the sequence matters.

By analyzing deidentified data from a patient’s health record, we trained TARGET-AI to recognize patterns that help us determine whether a patient’s clinical history trajectory suggests they are on a path that increases the risk of a particular diagnosis. We developed and validated this algorithm across the Yale New Haven Health system and validated its performance externally in datasets from the U.S. as well as the U.K. This is critical to show that the model doesn’t simply learn patterns specific to our population, but works across different patient demographics and system characteristics.

We hope that health systems will begin using TARGET-AI to help determine when and how to deploy AI throughout their systems. Our goal is to help build the guardrails to maximize the precision of other AI detection tools.

This space is moving very fast. We always need to think outside the box, or our thinking becomes very reactive.

I always recommend that researchers look at their own clinical practice or laboratory and identify what could be streamlined or optimized. Those are real issues that could benefit from research.

There is also much more we need to learn about the partnership between humans and AI, including how they work together and where the pain points lie. This is the next phase of AI research.