The next generation of lit reviews
By Kathy Belk, Nicole Parker, Jan-Willem van Doorn, and Jonathan Wert
This is the first in a series of three articles by HCG thought leaders that will be appearing in Med Ad News exploring the impact and application of artificial intelligence in a number of areas touching healthcare and healthcare communications. The next article, discussing AI and clinical practice, will appear in February.
A systematic literature review is one of the more time- and resource-intensive undertakings in pharma. It requires teasing out all the articles on a particular subject that might be lurking in the many nooks and crannies of an organization, or externally, or both. Then, if done properly, it requires two separate human screeners with highly specialized knowledge and experience reviewing each and every one of those articles, each of them deciding, “Do we include this article or not?” for every one of dozens or hundreds of articles. Then, if the two screeners ever disagree, which they inevitably do, an arbitrator, another highly specialized human, must make a final decision on inclusion. And then, another highly specialized human must review all the approved articles to tag any information or data or conclusions relevant to the question at hand – “These are the patients in this age group,” or, “These are the patients with these particular comorbidities.” And then, another highly specialized human, or several, have to actually extract the relevant tagged data, analyze it, and summarize it in some usefully descriptive quantitative or qualitative way; “This percentage of studies showed this, and this percentage of studies showed that.” And all that just to answer one question. Want to answer another question? Go do the whole thing again. For all the extraordinary advances we’ve seen in the business and science of pharma over the past 50 years, doing a literature review isn’t all that much different today than it was for our ancestors in the 1970s.
Sweet relief, though, may be in sight. At Lumen Value & Access, a Healthcare Consultancy Group company, we’ve been able to introduce artificial intelligence and machine learning as well as automation technology into the literature review process. It’s the early days yet, surely. But we’ve had data scientists train an AI algorithm in that platform to screen articles and then used that AI process to replace one of the two screeners in the traditional review process. We’ve also used natural language processing in that same platform to facilitate tagging and extracting data. Manual intervention is still required in those processes, but they’ve become substantially more efficient, to the tune of a roughly 40 percent decrease in human time required to complete a full review. The outputs we’ve seen so far are as good if not better than might be expected from a fully human traditional review. Yes, the AI makes mistakes and needs to retrained periodically. But what the AI doesn’t do is suffer the consequences that a human might after reading through 1,200 highly specialized research articles filled with abstruse technical language while hunting for needles. AI doesn’t get heavy eyelids or fuzzy brain no matter what you throw at it. Its judgment isn’t impacted by the length or difficulty of the process. The 1,200th article gets the same treatment as the first.
Introducing AI and natural language processing as well as technology enabling interactive outputs to the literature review process has helped bring other improvements to the literature review process besides plain efficiency and a reduction in heavy eyelids. Your typical pharma organization has a long list of what might be called centers of knowledge. The HEOR folks are doing reviews (the “R” stands for research, after all!) and the med affairs folks are doing reviews, and the marketing folks are doing reviews, and maybe other departments are doing reviews as well. But when someone in HEOR does a review, they are likely only reviewing a limited chunk of the literature available to answer a very targeted question and sharing the results only with fellow HEORs; and when someone in med affairs … you see what we’re saying. Silo, pharma’s favorite four-letter word. But of course computers don’t care about artificial walls between departments, and they also don’t care how big the datasets are. So in the process of integrating AI into the literature review process, we also made sure that the outputs included all of the organization’s literature resources, including posters and grey literature, and that the outputs could be dynamically filtered to suit the specific needs of any potential user or department. So now when someone in HEOR orders a review, they are able to look at that knowledge base to answer many different questions, and the results are accessible and useful to everyone, which doesn’t sound like a big deal to an outsider but is in fact a substantial innovation over how things were done in the world of literature reviews pretty much forever.
Another benefit we’ve discovered is the benefit of being dynamic. In the traditional model, when somebody orders a review in January, the review might answer their question in January, but that answer grows less and less comprehensive and trustworthy the further from January you get. Medical studies and literature don’t stop appearing, after all, just because you’ve finished a review, so you might find yourself ordering the same exact review in October that you did in January. But AI platforms care just as little about time as they do about size, and they can be built to continue to update reviews as new information is entered into their various data sources. With such technology available, literature reviews can be transformed from the equivalent of books on a shelf –static and never-changing – to live, near real-time interactive tools that can be accessed by anyone at any time.
It would be difficult to overstate how valuable a dynamic capability in literature reviews could be during the course of a product’s life cycle. Early in the R&D process you might want a review to better understand your product’s potential position in the market and assess readouts from trials. As it moves closer to launch you might want to use a literature review to demonstrate comparative effectiveness and to develop inputs for economic models to show budget impact for payers. After launch literature reviews can evaluate your product’s efficacy in a real-world setting and demonstrate its value compared with old and new competitors as the market changes around you. Being able to do all this on an ongoing automated basis rather than just at one or two or three static moments in time will mean better- and faster-informed brand teams, payers, HCPs, and, best of all, patients.
A related benefit is that AI doesn’t have to be trained again the next time you ask a similar question. The first time you do a literature review to investigate, say, comparative effectiveness of diabetes drugs in a specific population, yes, you’ll need a specialized data scientist to train the AI properly to recognize exactly what you are looking for. But the following month, when you want to know about, say, relevant comorbidities in that same population, well, that AI is already trained now. It might need some brushing up or updating, but the investment has been made. Which means that each subsequent literature review on a similar subject will be quicker and more efficient than the one before.
We’ve also managed to use AI to begin to automate, and “dynamize,” (if that’s a word!) the process of summarizing and displaying the conclusions of literature reviews. When humans have to create the summary of a complex literature review, it requires a tenuous and manual process to summarize the included studies, never mind any attempts at meta-analysis of that data. But with the help of AI and automated data dashboards, one can rapidly create dynamic summaries that give the user the ability to dig deeper with the click or two of a mouse. Take a look at the visuals in Figure 1 to see what we mean. In the first, each little segment of that sunburst contains more data, more comparisons, more analysis based on a literature review of balloon guide catheter studies – in this case, different populations, different ages, different comorbidities, different outcomes, all the data right there complete with confidence intervals and odds ratios for easy comparison, any possible interpolation or combination just a click away. In the second, just a few clicks away from the first, you can see forest plots of odds ratios for the relevant studies, showing the odds of IV-tPA treatment for non-BCG patients versus BCG patients. In the old days, to drill down to that level of specificity in a summary you’d have to plow through who knows how many pages and indexes, and some poor human would’ve had to have done a mountain of calculations to even put it on that page in the first place.
The next step? As AI technology gets faster, smarter, and better-trained, eventually it’ll start asking the questions itself before we humans even think of them. AI can be constantly combing our databases of literature, constantly analyzing and summarizing, so that the answers, the comparisons, the relationships that we might not have seen or even known to look for will be right there to discover at a glance. With human support and guidance, they’ll have the power to discover gaps in the research, unmet needs, contradictions, missing pieces that need to be filled in, underserved or under-researched patient populations, undiscovered economic inequalities or opportunities. The analyses they produce and the discoveries they make will be available outside the walls of pharma companies – to regulators (some of whom are already making strides in this direction), advocacy groups, researchers, perhaps even HCPs and patients themselves. And that will mean better research, better assignment of priorities, and – most importantly of all – better outcomes for patients. One of the great challenges in pharma, even today, is that we don’t know what we do know. The information available, even on a relatively limited subject, is more than any number of humans can handle unaided. But the expanding use of AI technologies, underpinned by humans able to direct those technologies, is changing that paradigm, which will mean better and faster outcomes for pharma’s literature reviewers – and for patients, too.
Kathy Belk is VP, strategic operations and innovation; Nicole Parker, Ph.D., is senior VP, scientific services; Jonathan Wert, M.D., is senior VP, clinical strategy; and Jan-Willem van Doorn, M.D., Ph.D., is chief transformation officer, Healthcare Consultancy Group.