A Research Agenda to Advance the Study of Implementation Mechanisms

Transcript

• 00:01And so on.
• 00:03We're very happy to have
• 00:04you here as well as
• 00:05all of us from the
• 00:06United States.
• 00:07I'm I'm Donna Spiegelman. I'm
• 00:09director of our Center on
• 00:10Methods for Implementation
• 00:12and Prevention Science.
• 00:14I'm professor of
• 00:17Public Health.
• 00:18And I'm very pleased to
• 00:20have as our, center's,
• 00:22guest,
• 00:23for our seminar today, doctor
• 00:25Carol Lewis,
• 00:26who is deputy director of
• 00:28the Center for Translational
• 00:30Research and Implementation Science, otherwise
• 00:33known as CITRIS, at the
• 00:34National Heart, Lung, and Blood
• 00:36Institute at the National Institutes
• 00:38of Health of the United
• 00:40States.
• 00:41As CITRIS is,
• 00:43going to be offering
• 00:45a a full day of
• 00:47of, activities at the upcoming
• 00:50dissemination and implementation
• 00:52of science re conference in
• 00:54December, and so we encourage
• 00:56all of you to attend
• 00:57that as well.
• 00:58Doctor Lewis has a PhD
• 00:59in clinical psychology with a
• 01:01research focus on advancing pragmatic
• 01:04and rigorous measures and methods
• 01:06for implementation science and practice.
• 01:09She has pushed the field
• 01:10to focus on measurement quality
• 01:12to improve the field's research
• 01:14rigor and generalizability.
• 01:16Her work has led the
• 01:17field in considering nuances
• 01:20of health systems, organizations, and
• 01:22individual change
• 01:23in how evidence based interventions
• 01:26can be more effectively
• 01:28implemented and sustained
• 01:29to benefit patients and the
• 01:31general public.
• 01:33Doctor Lewis' expertise
• 01:34in implementation
• 01:35science is evidenced through national
• 01:38and international
• 01:39responsibilities.
• 01:41Ally, can you please keep
• 01:42an eye on having everyone
• 01:43be muted?
• 01:44Her national and international
• 01:46responsibilities,
• 01:47including
• 01:48serving as president of the
• 01:49Society for Implementation
• 01:51Research Collaboration, otherwise known as
• 01:54CERC, which is how I
• 01:55first met her when she
• 01:56invited me to give a
• 01:57talk at the SERC
• 01:59meeting maybe five or six
• 02:01years ago. She's also cofounding
• 02:03editor in chief of the
• 02:04journal Implementation
• 02:06Research and Practice.
• 02:08She's a lecturer for implementation
• 02:10science training institutes,
• 02:12a scientific adviser domestically and
• 02:14abroad, and coeditor of Practical
• 02:16Implementation
• 02:17Science.
• 02:18And in twenty twenty two,
• 02:19she was recognized
• 02:21by Clarivate as one of
• 02:22the most highly cited social
• 02:24scientists in the world.
• 02:26Doctor Lewis has been a
• 02:27driving force in elevating the
• 02:29importance
• 02:30of implementation
• 02:31mechanisms,
• 02:33emphasizing how studying mechanisms of
• 02:35change may improve tailoring, optimizing,
• 02:38and scaling
• 02:39of effective implementation
• 02:41strategies,
• 02:42a topic that's of great
• 02:43interest to us here in
• 02:44our center, both, in terms
• 02:47of,
• 02:48advancing methodology
• 02:49for mediation analysis
• 02:51and using this sort of
• 02:52information to improve
• 02:54the optimization
• 02:55of implementation studies as they
• 02:57go through the design logo
• 02:59that we're working on. So
• 03:00I'm very pleased to turn
• 03:02the,
• 03:03meeting over today to doctor
• 03:05Lewis who's gonna talk about
• 03:07a research agenda to advance
• 03:09the study of the
• 03:11mechanisms.
• 03:12And, well, doctor Lewis is
• 03:14open to people,
• 03:16speaking up if they have,
• 03:19a clarifying question while she's
• 03:21talking,
• 03:22and then you can put
• 03:23sort of more opinion and,
• 03:26questions of a of a
• 03:28sort of far ranging
• 03:30nature in the chat, and
• 03:31then we'll go through them
• 03:32at the end of the
• 03:33talk along with any questions
• 03:35that any of you have
• 03:36at the end of the
• 03:37talk. So with that, I'd
• 03:39like to turn things over
• 03:40to doctor Lewis.
• 03:43Thank you, doctor Spiegelman, for
• 03:45the kind invitation
• 03:46and introduction.
• 03:48And, absolutely, I'm hoping that,
• 03:51I can address any clarifying
• 03:52issues that might arise. It
• 03:53sounds like we have a
• 03:54really diverse,
• 03:55set of backgrounds in our
• 03:57audience today. I see some
• 03:58old and new friends. So
• 04:00thanks for joining us. I'm
• 04:02delighted. This is the first
• 04:03time
• 04:04we'll have, a significant amount
• 04:06of minutes to actually unpack
• 04:08the process that led to
• 04:09and then the results of
• 04:11developing a research agenda to
• 04:12advance the study of implementation
• 04:14mechanisms.
• 04:16And so importantly,
• 04:18I don't have anything to
• 04:19disclose and that the contents
• 04:21of the presentation today are
• 04:22not to be construed as
• 04:24representing the official position of
• 04:26NIH or the Department of
• 04:27Health and Human Services here
• 04:28in the United
• 04:29States.
• 04:31I thought it was important
• 04:32to remind us all
• 04:35why we're here in terms
• 04:36of thinking about implementation science
• 04:38and,
• 04:39implementation
• 04:40mechanisms. And so,
• 04:43folks might recall back in
• 04:45two thousand and one when
• 04:46the institutes of medicine in
• 04:47the United States made this
• 04:48urgent call for fundamental change
• 04:51to close the quality gap.
• 04:52And,
• 04:53the book featured on the
• 04:54screen on the left here
• 04:55recommended a sweeping redesign really
• 04:57of the American health healthcare
• 04:59system
• 05:00and provided four overarching principles
• 05:02for specific direction.
• 05:04One of those four principles
• 05:05were key steps to promote
• 05:07evidence based practice,
• 05:09integration, and strengthening clinical information
• 05:12systems.
• 05:13And then in two thousand
• 05:14and eight, the Institute of
• 05:15Medicine
• 05:16put, together this, also this
• 05:19publication of To is Human,
• 05:20where they broke the silence
• 05:22around medical errors and their
• 05:23consequence and set forth a
• 05:25national agenda to improve,
• 05:27patient safety through the design
• 05:29of a safer health health
• 05:31care system. So these are
• 05:32some of the inspirations
• 05:34for this formal field of
• 05:36implementation
• 05:37science, coming to be. In
• 05:39addition to the publication that
• 05:41and colleagues put out in
• 05:43two thousand and nine where
• 05:44they,
• 05:45feature this leaky pipeline of
• 05:47research that helped to explain
• 05:49a very commonly cited,
• 05:52statistic that,
• 05:55a small percentage of our
• 05:56research takes,
• 05:58up to seventeen years for
• 06:00effective interventions to become routinely
• 06:02available to patients. And you
• 06:04can see some of the
• 06:05reasons for that time lag
• 06:07and then the leak in
• 06:08this pipeline on the image
• 06:10on your screen.
• 06:12I'm delighted to say that
• 06:13we are approaching twenty years
• 06:15of implementation
• 06:16science at the National Institutes
• 06:18of Health.
• 06:20What you see on this
• 06:21slide is,
• 06:22what is fondly referred to
• 06:24as the PAR twenty two
• 06:25one zero five, which,
• 06:27is a funding mechanism for
• 06:29independent investigator
• 06:30initiated applications,
• 06:33both in the United States
• 06:34also with foreign components, but
• 06:36also for foreign investigators.
• 06:38And you can see just
• 06:39how many of the institutes,
• 06:41centers, and offices at NIH
• 06:43have signed on to this
• 06:44program announcement. And later in
• 06:45my talk, you'll see that
• 06:46this week, we've renewed this
• 06:48for another,
• 06:49another term, which I'm really
• 06:50excited about.
• 06:52But,
• 06:53just to make sure we're
• 06:54on the same page, implementation
• 06:56science is the scientific study
• 06:58of the use of strategies
• 07:00to adopt and integrate,
• 07:02evidence based health interventions into
• 07:04clinical and community settings to
• 07:06improve individual outcomes and benefit
• 07:09population health.
• 07:11Now I have permission from
• 07:12the gentleman in the right
• 07:13hand corner of my screen,
• 07:15doctor David Chambers, who's the
• 07:17deputy director for implementation of
• 07:18science at the National Cancer
• 07:20Institute,
• 07:20to share how he characterized
• 07:22our field when he and
• 07:23I did a plenary session
• 07:25at SERC,
• 07:26several years ago now. And
• 07:28so, briefly, in the two
• 07:30thousands,
• 07:31right on the heels of
• 07:32that, funding opportunity,
• 07:35we saw a great deal
• 07:37of emphasis on trying to
• 07:38explain and understand the gap
• 07:41between what we know from
• 07:42scientific evidence and then what
• 07:44we see in practice.
• 07:46And, one of the seminal
• 07:48articles that came out during
• 07:50that decade of implementation
• 07:52science is featured at the
• 07:53bottom here,
• 07:55published by Laura Damschroeder and
• 07:56colleagues.
• 07:57This is referred to oftentimes
• 07:59as the CFIR, the consolidated
• 08:01framework for implementation research.
• 08:03And what they did was
• 08:04they took together these determinant
• 08:06frameworks or frameworks that characterize
• 08:09barriers and facilitators to implementing
• 08:11evidence based interventions,
• 08:14and they organized them into
• 08:15five domains. And I think
• 08:16it was forty seven constructs
• 08:18including sub constructs. And these
• 08:20are things like
• 08:21inner setting, culture, and climate
• 08:23of a health system or
• 08:24even an educational setting if
• 08:26that's the target of an
• 08:27evidence based in, intervention
• 08:30integration effort,
• 08:32to outer setting characteristics such
• 08:34as policies, but also, of
• 08:35course, characteristics of individuals,
• 08:38the process, and the intervention
• 08:39characteristics themselves.
• 08:41So we saw a lot
• 08:42of these publications
• 08:43on characterizing,
• 08:46contextual factors that might help
• 08:48or hinder the integration of
• 08:49an evidence based intervention.
• 08:52Fast forward ten years or
• 08:53so,
• 08:54implementation
• 08:55scientists started grappling with, well,
• 08:57what works? We know what
• 08:58gets in the way. What
• 09:00are the methods or techniques
• 09:01that we need to deploy
• 09:03in order to either resolve
• 09:05those barriers or leverage those
• 09:06facilitators
• 09:07and have success in terms
• 09:09of fidelity to an evidence
• 09:11based intervention?
• 09:13And here, I saw that
• 09:14doctor Powell is with us
• 09:15today. So if you have
• 09:16any questions about implementation strategies,
• 09:18direct them to him.
• 09:19But he was important in
• 09:21leading the charge on a
• 09:23compilation of, strategies for implementing,
• 09:26clinical inner innovations in health
• 09:28and mental health and subsequently
• 09:30in twenty fifteen updated this
• 09:32compilation that folks might be
• 09:34familiar with, and referred to
• 09:36as ERIC, the experts for
• 09:37implementing,
• 09:40interventions recommending change. The mark
• 09:41of a good acronym is
• 09:42you can't remember what it
• 09:43stands for. But so ERIC
• 09:45is on the street, and
• 09:46a lot of people use
• 09:47this as a menu for,
• 09:50selecting strategies to overcome these
• 09:52obstacles for implementation.
• 09:55And now in the twenty
• 09:56twenties, we're starting to see
• 09:57more of and and we
• 09:58hope to inspire you today
• 10:00and in other spaces
• 10:02to be
• 10:03critically evaluating
• 10:05why, how are these implementation
• 10:08strategies working or not.
• 10:10And
• 10:12answering this question of what
• 10:13works best for whom, when,
• 10:15and under what conditions. And
• 10:17I'll talk a little bit
• 10:18more about the systematic review,
• 10:20that we led
• 10:21a few years ago in
• 10:22just a moment.
• 10:24But before we get into
• 10:25that, I thought maybe another
• 10:27way to break this down.
• 10:28What are the factors that
• 10:29we've been exploring? What are
• 10:30we really interested in? What
• 10:32should we be studying?
• 10:33So in this table on
• 10:34the right hand side, we're
• 10:36featuring implementation outcomes. This is
• 10:38not the host of them.
• 10:39These are just some examples.
• 10:41There are at least eight
• 10:42of them that we seek
• 10:43to achieve, but we're here
• 10:45because we want adoption of
• 10:46evidence and evidence based practices.
• 10:48We want to see that
• 10:49they're acceptable, appropriate,
• 10:51feasible to be done in
• 10:52the intended
• 10:54and that they're sustainable,
• 10:56and that they're spread.
• 10:57So this is this is
• 10:58why we're here, implementation outcomes.
• 11:01But then, as you saw,
• 11:02that first decade, we really
• 11:04started to unpack what are
• 11:05the determinants, what are those
• 11:06barriers, what are those facilitators.
• 11:08And, again, these are just
• 11:09a a handful of examples,
• 11:11and they range from the
• 11:13intrapersonal,
• 11:14so in the individual provider,
• 11:16for example, all the way
• 11:17to system level,
• 11:19types of determinants.
• 11:22And then
• 11:23what are the implementation strategies?
• 11:25What are the activities we
• 11:26should be doing? What are
• 11:27the,
• 11:28methods that we should be
• 11:29deploying in order order to
• 11:31overcome
• 11:32those determinants
• 11:33of practice?
• 11:34And then what the focus
• 11:35of today is is on
• 11:37the mechanisms.
• 11:38How are those strategies,
• 11:40working through what process to
• 11:42overcome those determinants to help
• 11:44us achieve those outcomes?
• 11:46And I have permission from
• 11:47this gentleman here featured on
• 11:49your screen, doctor Alvin Gang,
• 11:51to
• 11:52retell,
• 11:53a re
• 11:55a a a retelling
• 11:56of the story of vitamin
• 11:58c and how it surfaced
• 11:59as a cure for scurvy.
• 12:00So,
• 12:01thanks to Elvin. When I
• 12:02heard him describe,
• 12:04in this way,
• 12:06the the retelling of the
• 12:08vitamin c discoveries,
• 12:10I I, was so compelled
• 12:12that it helped us,
• 12:14motivate the study for mechanisms
• 12:16to advance implementation science.
• 12:18Implementation science, so I wanted
• 12:19to share it briefly with
• 12:20you. So as you all
• 12:21know, scurvy was this devastating,
• 12:22disease that killed perhaps two
• 12:24million sailors before the twentieth
• 12:26century, often claiming half of
• 12:27a crew.
• 12:29And as we now know,
• 12:30vitamin c contributes to the
• 12:31formation of the collagenous tissues,
• 12:33a material that keeps your
• 12:34blood vessels, joints, and gums
• 12:36intact, and its absence can
• 12:38be fatal. So in seventeen
• 12:40forty seven, James Linda Surgeon
• 12:42tried an experiment that then
• 12:43made history. Right?
• 12:46So his experiment,
• 12:48went something like this. He
• 12:49got twelve,
• 12:50sailors sick with scurvy and
• 12:52put them into six groups,
• 12:53two in each, and gave
• 12:54each group the candidate treatments
• 12:56that you see here. The
• 12:57two that got citrus recovered
• 12:59dramatically while the others languished.
• 13:01And this final finding is
• 13:02hailed as an early, if
• 13:04not the earliest successful clinical
• 13:05trial, and Lind has gone
• 13:07down in history as a
• 13:08scientific hero.
• 13:10However, the results were not
• 13:11successfully or consistently taken up
• 13:13for fifty to a hundred
• 13:14years, and admissions for scurvy
• 13:16at this typical hospital in
• 13:17Britain continued to be high
• 13:19for at least one hundred
• 13:20years after Lynn's experiment.
• 13:22So here we have a
• 13:23fabulous example of what we
• 13:24now refer to as this
• 13:25no do gap. So it's
• 13:27clear that the citrus theory
• 13:28had far less of an
• 13:29effect than it could have,
• 13:30and this is where implementation
• 13:32science can really come into
• 13:33play.
• 13:34And as with everything that
• 13:35we undertake in implementation
• 13:37science, the truth is far
• 13:38more complicated. So even though
• 13:40Lynn found that citrus led
• 13:42to recovery, no one knew
• 13:43what the mechanism,
• 13:44knew the mechanism, vitamin c.
• 13:47It's tragic in a way.
• 13:49As you can read, Lynn
• 13:50can, himself confesses in his
• 13:52final writings on the topic.
• 13:53He doesn't understand what he
• 13:55observed. And so some ships
• 13:56boiled lemon juice to concentrate
• 13:58it, but that inactivated vitamin
• 14:00c and sailors got scurvy
• 14:01and,
• 14:02casting doubt on the theory.
• 14:04And so in Elvin's words,
• 14:05this is not the story
• 14:06of a man with evidence,
• 14:08but the failure failure of
• 14:09a system to use it.
• 14:11And it's the story of
• 14:12a society that did not
• 14:14understand the mechanism of effect
• 14:15and therefore did not understand
• 14:17how to to use the
• 14:18intervention. So
• 14:19if you're if you're thinking
• 14:20about mechanisms,
• 14:22you won't be making these
• 14:23mistakes, and that's,
• 14:25hopefully what,
• 14:26will inspire more and more
• 14:28study of mechanisms in the
• 14:29implementation science space as we
• 14:32move forward in the field.
• 14:34Other things that inspire us
• 14:36to focus on mechanisms are
• 14:37these quotes that I hear
• 14:39and read about all the
• 14:40time. This strategy works here,
• 14:42but not here.
• 14:44There's no implementation
• 14:45without adaptation.
• 14:47And I think it's hard
• 14:48to,
• 14:49face these realities
• 14:51without centering mechanisms in the
• 14:53conversation.
• 14:54So our our group was
• 14:56so,
• 14:57motivated
• 14:58to,
• 15:00understand mechanisms of implementation
• 15:02strategies. We undertook the systematic
• 15:04review
• 15:05of empirical studies examining,
• 15:07just that.
• 15:08Now before I I share
• 15:10the methods and results of
• 15:11that review, I wanna ground
• 15:13us in in at least
• 15:14how our team was defining
• 15:16mechanisms for the systematic review
• 15:19and the other
• 15:20aspects of the talk today.
• 15:22So the reasons why change
• 15:23occurred or how change came
• 15:25about, so it being a
• 15:26process,
• 15:28they can be,
• 15:30mechanisms can be
• 15:32found through examining mediators.
• 15:34Mediators are not necessarily inherently
• 15:37mechanisms,
• 15:38and, of course, it all
• 15:39depends on your definition and
• 15:41design and things of that
• 15:42nature.
• 15:43But you can read more
• 15:44about,
• 15:45the
• 15:46definitions of of mechanisms in
• 15:48the paper I've put on
• 15:49the screen.
• 15:50I thought another way,
• 15:52based on what Donna was
• 15:54sharing with me about her
• 15:55group of just,
• 15:58depicting
• 15:59mechanisms in the causal chain
• 16:00was to put up this
• 16:02schematic of causal pathway diagramming,
• 16:04and it features all of
• 16:05the terms that I've mentioned
• 16:07to you already. So on
• 16:09the far right hand side,
• 16:11we're striving for some distal
• 16:13implementation outcome,
• 16:15when we bring a strategy
• 16:16into the field.
• 16:17And if we are to
• 16:19center the mechanism of action,
• 16:22we might be able to
• 16:23more,
• 16:24expertly, strongly, swiftly overcome and
• 16:27determinant to get to some
• 16:29proximal
• 16:31signal of success and ultimately
• 16:32to that implementation outcome. Of
• 16:34course, we need to be
• 16:35thinking about moderators, what is
• 16:37amplifying or dampening
• 16:38the impact of a strategy,
• 16:40and to make sure that
• 16:41there are the necessary preconditions
• 16:43in place in order to
• 16:45activate a mechanism. So just
• 16:47another way to think about
• 16:48and,
• 16:49stand up an evaluation of
• 16:51implementation strategy mechanisms.
• 16:54So for the systematic review
• 16:56that was published in twenty
• 16:57twenty, we identified thirty nine
• 16:59studies, and that was as
• 17:00of twenty eighteen. That's when
• 17:01our systematic review analysis ended.
• 17:04Fifty four of those were
• 17:05conducted in the US, so
• 17:07a lot of global work
• 17:08in this space to,
• 17:10study implementation
• 17:11mechanisms.
• 17:12Interestingly, behavioral health settings were
• 17:14the most common,
• 17:15featured in this in this
• 17:17review,
• 17:18followed by hospitals and health
• 17:19systems.
• 17:20Seventy four percent of the
• 17:22empirical studies we included were
• 17:23quantitative in nature. There were
• 17:25a handful of mixed methods
• 17:26and qualitative only, of of
• 17:28course.
• 17:30It's interesting that not all
• 17:31of these studies actually used
• 17:33a theory model or framework
• 17:34to,
• 17:35articulate,
• 17:37and con contextualize
• 17:38their, mechanism evaluation.
• 17:41But of those who did,
• 17:42about seventy four percent,
• 17:44the diffusion of innovations theory
• 17:46featured most with twenty percent
• 17:47of researchers.
• 17:49So,
• 17:51this is a legend that
• 17:52you see on your screen
• 17:53now.
• 17:54You've seen all these terms
• 17:55before implementation
• 17:56strategy, determinant
• 17:58implementation outcome, which, of course,
• 18:00is different than the clinical
• 18:01outcome. And so in some
• 18:03ways, we have two levels
• 18:04of intervention and two levels
• 18:05of outcomes.
• 18:07And then,
• 18:08we're characterizing the mechanism as
• 18:10a mediator in these models
• 18:11you'll see. So what we
• 18:12tried to do is actually,
• 18:16in a in a figure
• 18:17using these shapes and icons
• 18:19is characterize the range of
• 18:22types of mechanism evaluations that
• 18:24the field had contributed,
• 18:26to this point.
• 18:27So in the upper left
• 18:29corner,
• 18:30model one, we saw this
• 18:31in one study where,
• 18:33folks were
• 18:35actually
• 18:36looking at,
• 18:37determinants
• 18:38and then, the change in
• 18:40strategies, which was a little
• 18:41bit interesting and unexpected for
• 18:42me at least,
• 18:45to then change in in
• 18:46determinants of practice.
• 18:48This model two appeared in
• 18:50two different studies, strategy
• 18:52to,
• 18:54outcome to another outcome.
• 18:56And this was the primary
• 18:57model we were interested in
• 18:59seeing where strategies
• 19:00are changing determinants in practice
• 19:03and then yielding the implementation
• 19:04outcomes of interest. We saw
• 19:06this I'm sorry. This is
• 19:08Donna. Just because even I
• 19:09am a I understand all
• 19:11the terms, but I'm not
• 19:12quite sure what determinant of
• 19:14practice means.
• 19:15Sure. Yes. Thanks for clarifying.
• 19:17So determinant,
• 19:19I'm using that as the
• 19:20jargony term for barriers or
• 19:22facilitators. So these are Okay.
• 19:23Those contextual factors
• 19:25that,
• 19:26would otherwise undermine a successful
• 19:28implementation
• 19:29with an example being,
• 19:32something like turnover
• 19:33or implementation
• 19:34climate. Right? Mhmm. Mhmm. Yes.
• 19:37So the idea being that
• 19:39we're often choosing a strategy
• 19:42to address a determinant
• 19:44to then yield that outcome
• 19:46of interest. And it was
• 19:47interesting. We only saw of
• 19:49the thirty nine studies, this
• 19:51particular model showing up in
• 19:52seven of the studies.
• 19:54Otherwise, we're seeing a whole
• 19:56different range of constellations, and
• 19:57I'll just let the screen
• 19:59continue to populate here so
• 20:00you can see the different
• 20:01range of
• 20:03mechanism evaluations
• 20:04among those thirty nine studies
• 20:07here.
• 20:09So then also the last
• 20:11thing I'll say about the
• 20:12systematic review findings is that,
• 20:15if you if you look
• 20:17at the, you know, Seminole
• 20:19Bradford Hill
• 20:21criteria for establishing
• 20:23mechanisms then elaborated on in
• 20:25in later years by Al
• 20:27Kasdan,
• 20:28there are these criteria
• 20:30that's not just single studies,
• 20:32but,
• 20:33subsequent studies of similar mechanisms
• 20:36really need to address in
• 20:37order to have confidence that
• 20:39you've established a mechanism. So
• 20:41things like plausibility.
• 20:42There's a timeline issue. Right?
• 20:44And that's why we think
• 20:45about mediators when we talk
• 20:46about mechanisms.
• 20:48There needs to be association.
• 20:49Ideally, there's manipulation
• 20:51happening,
• 20:52consistency
• 20:53across studies,
• 20:55specificity,
• 20:56that you're able to that
• 20:58you're able to shed light
• 20:59on evidence for a specific
• 21:01mechanism and not others,
• 21:04and then,
• 21:05evidence of a gradient of
• 21:07change. And so you can
• 21:08see that,
• 21:10very few studies were able
• 21:12to address all of these
• 21:13or any of these criteria.
• 21:15So there's work to be
• 21:16done.
• 21:17This is all inspiration for
• 21:18what led
• 21:19us to wanna unpack this
• 21:21line from a to b,
• 21:22strategy to outcome,
• 21:24and, borrowed this comic from
• 21:26Byron Powell and others that
• 21:27I think you should be
• 21:28more explicit here in step
• 21:30two. And so
• 21:31here we go. A little
• 21:32bit more on the motivation.
• 21:34What are the problems we're
• 21:35trying to solve when we
• 21:36think about mechanisms?
• 21:38You know, unfortunately,
• 21:40despite the progress, the twenty
• 21:41years or more of science
• 21:43in this area,
• 21:44we see that, there is
• 21:47still some suboptimal,
• 21:49outcomes in terms of the
• 21:51implementation
• 21:52clinical trials that demand a
• 21:54huge investment,
• 21:56both domestically and abroad.
• 21:58And there's one,
• 22:00example that I like to
• 22:01bring up, the tailored implementation
• 22:03in chronic diseases.
• 22:05This is a multinational
• 22:07study in Europe that took
• 22:09seven years,
• 22:11to come to fruition.
• 22:13And
• 22:14in the introductory
• 22:15article to the special collection
• 22:17that gave us so many
• 22:18learnings in the implementation science
• 22:20space, unfold or, unfortunately, the
• 22:22effects were null in terms
• 22:24of the tailoring,
• 22:26influencing the outcomes they cared
• 22:28about. And Michelle Vinseng, the
• 22:30former editor of implementation
• 22:32science, states that the chosen
• 22:34strategies were not sufficiently matched
• 22:36with targeted determinants or not
• 22:38effective
• 22:39in the targeted groups and
• 22:40settings. So insight into the
• 22:42linkages of interventions and determinants
• 22:44is very limited.
• 22:46So that that's,
• 22:48a more recent,
• 22:49observation
• 22:50that motivates this, and I'll
• 22:52stay back on that slide
• 22:53just for another moment to
• 22:54acknowledge also that,
• 22:57the higher cost and complexity
• 22:59of implementation
• 23:00strategies is sometimes perturbing to
• 23:02me. You know, we've got,
• 23:04we've got such inequities in
• 23:07terms of access to evidence
• 23:08based interventions.
• 23:10And if here we are,
• 23:12designing and testing implementation
• 23:14strategies that cannot be accessed
• 23:16by low resource settings that
• 23:18cannot be scaled because of
• 23:19their cost, I I think
• 23:21we're doing ourselves a disservice
• 23:23and might unfortunately
• 23:24land ourselves with this ironic
• 23:26situation where we've created our
• 23:28own gap between implementation
• 23:30strategy evidence and then implementation
• 23:32practice and what is possible.
• 23:34And then finally, and this
• 23:36is actually more along the
• 23:37lines of what I was
• 23:38just saying,
• 23:39that, inequities
• 23:41could could really be exacerbated
• 23:43when we're really trying to
• 23:44address them and achieve health
• 23:46equity.
• 23:48So practically speaking, we and
• 23:50I always feel like I
• 23:51should say,
• 23:52at least a few things
• 23:53along the lines of why
• 23:55are why is studying implementation
• 23:57mechanisms useful practically,
• 23:59because it does seem like
• 24:01such a scientific sort of,
• 24:04specific,
• 24:05type of evaluation.
• 24:06Well, you've heard me talk
• 24:07about selecting strategies.
• 24:10This is not an easy
• 24:11task. When we do,
• 24:14try to characterize the literature
• 24:16on what are the most
• 24:17common
• 24:18implementation
• 24:18strategies being tested,
• 24:20we see so much of
• 24:21it being perhaps driven by
• 24:23familiarity,
• 24:24you know, groups being or
• 24:25hospital systems being familiar with
• 24:27CMEs. And so what do
• 24:29they do when they need
• 24:29to bring in a new
• 24:30clinical intervention?
• 24:32They offer a CME.
• 24:34But there should be other
• 24:36ways to inform selection of
• 24:38strategies.
• 24:39Unfortunately,
• 24:40and there are other examples,
• 24:42not just familiarity,
• 24:45which is sort of the
• 24:46first that the train and
• 24:47pray,
• 24:48But there's the kitchen sink
• 24:50approach,
• 24:51that relates to the increasing
• 24:52cost and complexity. Let's throw
• 24:54all the strategies we can
• 24:56at the,
• 24:57implementation
• 24:58problems we're facing.
• 25:00We I talked about tailoring.
• 25:02There's and the the opposite
• 25:04of that is a one
• 25:05size fits all approach. And
• 25:06so if we're selecting a
• 25:08bundle of strategies,
• 25:10and using them the the
• 25:11same across context, then,
• 25:14that might not be the
• 25:15best way to to make
• 25:16selections. And then Martin Echols
• 25:18coined the is logiat, which
• 25:21was it seemed like a
• 25:22good idea at the time.
• 25:23And so we can do
• 25:24better, and I think studying
• 25:26mechanisms and centering them in
• 25:28selection is one way to
• 25:29do
• 25:31better. Practically speaking then,
• 25:34part of selecting is matching
• 25:36to the strategies to barriers.
• 25:38So you've heard me talk
• 25:39about determinants multiple times. This
• 25:42is one of the really
• 25:43interesting,
• 25:44outputs from that tailored implementation
• 25:46of chronic disease,
• 25:47multinational study where Kraus and
• 25:49colleagues and forgive me for
• 25:50not having the citation here
• 25:52for you.
• 25:53They discovered six hundred and
• 25:55one,
• 25:57discrete plausible determinants of practice,
• 25:59and they did shake loose
• 26:01across a variety of levels,
• 26:02patient, provider, organization,
• 26:04community, and policy levels.
• 26:07So when you're faced with,
• 26:08you know, I think we
• 26:09see more often fifty or
• 26:11so determinants in a given
• 26:12project. This was across many.
• 26:14But when you're faced with
• 26:15so many determinants, how do
• 26:17you select and match strategies?
• 26:19Especially when then you're faced
• 26:20with seventy or more. I
• 26:22think more recently, we see
• 26:23at least one hundred discrete
• 26:25strategies are available to us
• 26:26for use. And I'm giving
• 26:28you some some examples across
• 26:30plan, educate, finance, restructure,
• 26:33quality management, and the policy
• 26:34context.
• 26:37So then, tailoring is another,
• 26:40approach to implementation,
• 26:42that I've been talking about,
• 26:43and,
• 26:45I'm excited about the work
• 26:46forthcoming from Sheena McHugh and
• 26:49her team. They're characterizing processes
• 26:51and outcomes of tailoring implementation
• 26:53strategies,
• 26:54because, really, there's so much
• 26:56to be,
• 26:57learned and specified in this
• 26:59space. Even the definition of
• 27:01tailoring,
• 27:02has some tension associated with
• 27:04it,
• 27:05and the methods are unclear
• 27:06about how best,
• 27:08to tailor to optimize outcomes.
• 27:10And speaking of optimization,
• 27:12there are a couple of
• 27:13groups, including yours, Donna, who
• 27:15are helping us think about
• 27:16how do we optimize
• 27:18implementation
• 27:18strategies
• 27:19when we're centering criteria like
• 27:22equity or
• 27:23cost or efficiency.
• 27:25How do we streamline? How
• 27:27do we scale implementation,
• 27:30accordingly?
• 27:31So that all of that
• 27:33led us to apply to
• 27:34the Agency for Healthcare Research
• 27:36in the United States, for
• 27:38an r thirteen award. This
• 27:39is a conference grant mechanism,
• 27:42that we fortunately received after
• 27:44our third application.
• 27:45I like to be a
• 27:46humble human and let folks
• 27:47know it takes a lot
• 27:49of tries, right, to get
• 27:50your,
• 27:51grant ideas funded.
• 27:54This was not my record.
• 27:55I think I submitted an
• 27:56r o one four times
• 27:57once before I was successful.
• 28:00But the goals of this
• 28:01r thirteen were to establish
• 28:02the priorities to guide a
• 28:04research agenda on implementation
• 28:06mechanisms and then to share
• 28:07it broadly.
• 28:09The methods we use were
• 28:10multifold.
• 28:11They featured partnership,
• 28:13with SERC,
• 28:14that society that has been
• 28:16mentioned a couple of times,
• 28:18network recruitment,
• 28:19to folks who had mechanism
• 28:21like expertise,
• 28:23and we formed this mechanisms
• 28:24network of expertise in SERC.
• 28:27We then identified challenges,
• 28:30to doing this type of
• 28:31research,
• 28:32did some synthesis work, articulated
• 28:34priorities and possibilities,
• 28:37and are in the dissemination,
• 28:39stage of things.
• 28:40So just wanted to give
• 28:41a nod to the many
• 28:43humans who participated in this
• 28:45effort. I have to acknowledge
• 28:46that they're predominantly in the
• 28:48United States. We had funding
• 28:50limitations to travel people broadly.
• 28:53And, also, as you might
• 28:54have noted with the timeline,
• 28:55this overlapped with COVID, which
• 28:58prevented some of the conference
• 28:59like activities that we had
• 29:01planned.
• 29:02But we kicked things off
• 29:03with a, plenary session at
• 29:05SERC in twenty
• 29:08seven nineteen, it looks like
• 29:10it was,
• 29:11with David Chambers and doctor
• 29:12Nate Williams where we were
• 29:13setting the stage for understanding,
• 29:15sort of trying to get
• 29:16on the same page about
• 29:17implementation
• 29:18mechanisms
• 29:19and what we knew about
• 29:20them and what we might
• 29:21not.
• 29:22And then we had several
• 29:24inputs for this research agenda.
• 29:26So the systematic reviews that
• 29:28existed in this space were
• 29:30among the inputs. We did
• 29:32an exercise we we called
• 29:33matrix mapping. So all of
• 29:35those, folks on the mechanism
• 29:37of expertise in the early
• 29:38days
• 29:39helped share,
• 29:41where they were studying mechanisms,
• 29:43what they knew about them
• 29:44in a really systematic approach.
• 29:47We also listened in on,
• 29:49studies being proposed that were
• 29:51being discussed in these,
• 29:53workshops, and we qualitatively
• 29:55extracted challenges to,
• 29:57doing mechanism evaluations.
• 29:59We hosted breakout sessions at
• 30:01SERC. So some of you
• 30:02might even been in those
• 30:03rooms where we had facilitated
• 30:06conversations, and I know some
• 30:07of our facilitators are on
• 30:08the call today. So grateful
• 30:10for all you did, to
• 30:12solicit information as one of
• 30:13these inputs. And then we
• 30:15actually coded abstracts at that
• 30:17particular conference.
• 30:20And this this was the
• 30:22theme of the CERC conference
• 30:24back in twenty seventeen, actually.
• 30:27Excuse me.
• 30:28And,
• 30:30I should make
• 30:31come off mute to share
• 30:32the methods behind concept mapping.
• 30:35It's a great way to
• 30:36bring together,
• 30:38experts to consider,
• 30:41discrete,
• 30:42pieces of information. In this
• 30:44case, they were considering one
• 30:45hundred and five unique challenges
• 30:48to studying implementation
• 30:50mechanisms,
• 30:51and they were asked to,
• 30:53group them to conceptually distinct
• 30:56piles and help us characterize
• 30:58those,
• 30:59discrete challenges. And this is
• 31:01where we landed in terms
• 31:02of a concept map.
• 31:04And so you can see
• 31:06the challenges to doing this
• 31:07work range from theoretical and
• 31:09causal in nature
• 31:11to the types of methods
• 31:13and designs that are needed,
• 31:15and and and more.
• 31:17So these one hundred and
• 31:19five,
• 31:20challenges were,
• 31:22the springboard
• 31:23for then,
• 31:24our
• 31:26mechanisms research agenda.
• 31:28Just acknowledging that during COVID,
• 31:30we got creative and we
• 31:31met on rocks and stumps
• 31:32in the woods
• 31:34to articulate
• 31:35our research agenda
• 31:36and used a lot of
• 31:37sticky notes to, help
• 31:40clarify priorities and possibilities.
• 31:43So this paper was published,
• 31:45just in September of this
• 31:47year.
• 31:47Another artifact of COVID is
• 31:49that some papers took a
• 31:50little bit longer to get
• 31:52out the door,
• 31:53but I'm delighted to have
• 31:55that out there in an
• 31:56open access journal for folks
• 31:58to consider,
• 32:00as you're thinking about implementation
• 32:02evaluations
• 32:03going forward.
• 32:04I'll spend just a few
• 32:05minutes then on what we
• 32:07featured in that paper.
• 32:09And so how we organize
• 32:10things was,
• 32:12grouping them into what we
• 32:14felt like were,
• 32:15conceptually sort of distinct areas
• 32:17of priority.
• 32:18And then within each of
• 32:20those areas of priority, we
• 32:21had at least two goals,
• 32:23and then
• 32:24we tried to make things
• 32:26concrete and actionable. So we
• 32:28called these possibilities,
• 32:29sort of action possibilities.
• 32:32And so there's a hundred
• 32:33and fifty one action possibilities,
• 32:35for studying implementation
• 32:38mechanisms moving forward, and I'll
• 32:40give you some examples of
• 32:41those.
• 32:42So one of the priority
• 32:44areas
• 32:45is accumulating knowledge within and
• 32:46across disciplines. So,
• 32:49there you know, this included
• 32:51nineteen actions actions that featured
• 32:53specific
• 32:54systematic reviews and meta analyses.
• 32:56When we think about mechanisms,
• 32:58the questions we hope scholars
• 32:59will answer are things like,
• 33:01do mechanisms
• 33:02operate universally,
• 33:04or is there variation across
• 33:05context,
• 33:07that we need to be
• 33:08aware of?
• 33:10Another priority area is,
• 33:12prioritizing mechanism research incorporating other
• 33:14knowledge. So this had eleven
• 33:16possible actions to bring transdisciplinary
• 33:30epidemiology,
• 33:30psychology, and various other disciplines.
• 33:33And so there are named
• 33:35possible actions for how to
• 33:36do that type of work
• 33:38in the table featured in
• 33:39that paper.
• 33:40We another priority area was
• 33:42overcome design challenges and innovate
• 33:44methods. There were eighteen actions
• 33:46here where there's really new
• 33:47methods needed or we need
• 33:49to borrow methods from other
• 33:50disciplines. So
• 33:51modeling time and quantitative assessment
• 33:54for isolating specific mechanisms,
• 33:57identifying underutilized
• 33:58methods that offer specific value,
• 34:00like like comparative case studies
• 34:02that could generate hypotheses about
• 34:04complex mechanistic pathways.
• 34:09Priority area around improving measurements.
• 34:11So measurement of mechanisms,
• 34:13particularly in the context where
• 34:14implementation scientists work is pretty
• 34:17challenging.
• 34:19This included thirteen different actions
• 34:21such as
• 34:22pragmatic approaches for objective data
• 34:24collection,
• 34:25those capturing lived experiences of
• 34:27individuals.
• 34:29Then we have another priority
• 34:31area for providing guidance for
• 34:33specifying mechanisms. There were fifteen
• 34:36actions here reflecting mostly tools
• 34:38and
• 34:39aids to improve researchers' approach
• 34:41to examining mechanisms.
• 34:43I, didn't share this when
• 34:44I was reporting out on
• 34:45the systematic review of mechanisms,
• 34:47but it was really challenging
• 34:50for us as a research
• 34:51team to,
• 34:53synthesize
• 34:54across those thirty nine studies
• 34:56because,
• 34:58most of the
• 35:00studies and their
• 35:02authors conceptualize mechanisms in different
• 35:04ways and conceptualize the causal
• 35:05pathway in different ways.
• 35:07And so,
• 35:09that's just one piece of
• 35:10evidence or example that,
• 35:12having some alignment or harmony
• 35:14around how we're thinking about
• 35:15this could be really helpful
• 35:17moving forward.
• 35:19Another priority area on increasing
• 35:21the focus on theorizing. So
• 35:23there were twelve possible actions
• 35:24here for developing, incorporating, refining
• 35:27theory and mechanistic research to
• 35:29better characterize mechanisms, so making
• 35:32theory explicit even in the
• 35:33strategy design phase.
• 35:35And this is the last
• 35:37group. These were emergent actions,
• 35:39so they,
• 35:40were not
• 35:41similar to the challenges
• 35:43that,
• 35:44fed into the concept mapping.
• 35:46But the group,
• 35:47who articulated the research agenda
• 35:49said just how important it
• 35:51was to engage the policy
• 35:52and practice community,
• 35:54twelve possible actions there,
• 35:57for understanding,
• 35:58the perspectives of these partners,
• 36:01using things like,
• 36:03implementation
• 36:03mapping, cognitive walk throughs to
• 36:06unpack or surface,
• 36:08mechanisms, really needing to plain
• 36:10language this space
• 36:11in order for it to,
• 36:13be engaging and informed by
• 36:15the people who really have
• 36:16the expertise to do so.
• 36:20Another priority area around engaging
• 36:22funders and the need for
• 36:23funding, and so this comes
• 36:25from the mechanisms of expertise
• 36:26saying that,
• 36:28there are seventeen actions that
• 36:29could be happening in this
• 36:31space,
• 36:32to leverage or stand up
• 36:34new opportunities
• 36:35and then building capacity.
• 36:37So, seventeen actions offering
• 36:40clarification or guidance,
• 36:42and avenues to build the
• 36:43field's capacity.
• 36:45And then finally, just this
• 36:46huge emphasis on dissemination.
• 36:48So,
• 36:49several goals listed for you,
• 36:51seventeen actions in the table
• 36:53and the paper. There are
• 36:54specific manuscript ideas that,
• 36:57if you happen to be
• 36:58looking for a paper topic,
• 36:59they're in there for you
• 37:00to consider,
• 37:02but also suggestions around forums
• 37:04to host dialogue,
• 37:07you know, ways to engage
• 37:08journal
• 37:09editors, and
• 37:10failing faster with wisdom was
• 37:11a paper type,
• 37:13that the mechanisms network of
• 37:15expertise came up with. We
• 37:17so often missed the opportunity
• 37:19to learn from null studies,
• 37:20and couldn't we just make
• 37:22that an explicit priority, for
• 37:24example?
• 37:25So,
• 37:26true to form, our mechanisms
• 37:28network of expertise couldn't stop
• 37:30with just articulating the research
• 37:31agenda. Many of them have
• 37:33written papers that address some
• 37:34of the items on the
• 37:35research agenda. So we stood
• 37:37up a special collection across
• 37:39the two,
• 37:40sibling journals, implementation science and
• 37:42implementation science communications.
• 37:44It's an open special collection.
• 37:46So if you're doing work
• 37:47in this space, we hope
• 37:48you'll consider
• 37:49selecting that special collection from
• 37:51the drop down menu and
• 37:52submitting to it directly.
• 37:55And I wanted to feature
• 37:57an article that came out
• 37:58last month by the editors
• 38:01and associate editors, several of
• 38:02them from those two
• 38:03journals where they update the
• 38:05scope,
• 38:06of those two journals. And
• 38:07I I had to highlight
• 38:09that among the,
• 38:10items included in the abstract
• 38:12was the import of the
• 38:13study
• 38:14of of implementation
• 38:16mechanisms.
• 38:17I'll wrap up with a
• 38:18few NHLBI
• 38:20specific,
• 38:21items for you. So,
• 38:23as you can see from
• 38:24my background, I sit,
• 38:26within NHLBI,
• 38:27specifically the Center for Translation
• 38:29Research and Implementation Science.
• 38:31And over the last year
• 38:33or so, we've been wrapping
• 38:34our arms around a ten
• 38:35year investment,
• 38:37in implementation
• 38:38science and making sense of
• 38:40our institute's
• 38:41contributions and that of other
• 38:43ICs,
• 38:44in the National Institutes of
• 38:45Health. And what you can
• 38:47see on the screen on
• 38:48the left here is that
• 38:50the proportion of NIH awards
• 38:52has grown substantially
• 38:53over the last
• 38:55ten years or so,
• 38:57which is very exciting. And
• 38:59NHLBI
• 39:00holds thirteen percent,
• 39:02share of all NIH
• 39:04implementation science awards with fourth,
• 39:07overall for total costs.
• 39:08It's an exciting
• 39:09trajectory to see. And then
• 39:11when we look within NHLBI,
• 39:13that trajectory of increase,
• 39:15is also evident. However,
• 39:18I think you'll find it
• 39:19staggering as do I that,
• 39:22the percent
• 39:23of our total cost for
• 39:24competing awards is about two
• 39:26percent
• 39:27focused on implementation
• 39:28science. So it's not just
• 39:30the study of implementation
• 39:32mechanisms that we need. It's
• 39:33implementation
• 39:34science more broadly, and we
• 39:35hope you'll consider targeting,
• 39:38NHLBI
• 39:39with some of your good
• 39:40grant ideas,
• 39:41in the coming months and
• 39:42years.
• 39:43So as I mentioned, just
• 39:45this week, the new program
• 39:46announcement has been released. I'm
• 39:48just showing you the r
• 39:49l one for for investigator
• 39:51initiated awards here, PAR twenty
• 39:53five one hundred and forty
• 39:54four.
• 39:55Again, US, US with foreign
• 39:57components, and foreign investigators
• 39:59can submit to this.
• 40:01And we're here for you.
• 40:02Right? So the our job
• 40:04is to be available for
• 40:06technical assistance is what we
• 40:07call it, to help you
• 40:09think through the ideas, their
• 40:10alignment with the various institutes,
• 40:13and responsiveness to the program
• 40:15announcements such as this.
• 40:17Donna shared that we're celebrating
• 40:18our tenth anniversary.
• 40:20There's a lot of red
• 40:21around me today.
• 40:22It's a hybrid event. And
• 40:23if you're interested, send me
• 40:25an email.
• 40:26There is going to be
• 40:27a cap, unfortunately,
• 40:28just given size of rooms
• 40:30and ability to host virtually,
• 40:32but you'll email me, and
• 40:33I'll I'll help you find
• 40:34your way.
• 40:36And lastly,
• 40:37we're hiring, and there are
• 40:39several opportunities if you're an
• 40:40implementation
• 40:41scientist looking to have a
• 40:43seat at the table and
• 40:45contribute to,
• 40:46the,
• 40:47strategic vision implementation
• 40:49plan for this work at
• 40:50NHLBI.
• 40:52And if you wanna stay
• 40:53abreast of when those opportunities
• 40:55emerge, follow us on LinkedIn.
• 40:58NIH largely has moved away
• 41:00from Twitter and accent is
• 41:02finding our new social media
• 41:04home in LinkedIn,
• 41:05and this is the, Citrus
• 41:07official page here. So on
• 41:09that note, thank you so
• 41:10much for hanging in there
• 41:12with me. It's a super
• 41:13nerdy topic. Hopefully, I made
• 41:15it accessible, but now is
• 41:17our chance to dialogue.
• 41:20Yeah. Thanks very much, Kara.
• 41:22That was a really excellent
• 41:23comprehensive
• 41:24talk that covered a lot
• 41:25of the key points, so
• 41:27we really appreciate it.