WEBVTT
1
00:00:00.090 --> 00:00:03.060
For joining our CMIPS seminar.
2
00:00:03.060 --> 00:00:04.740
It's really a pleasure to have you all
3
00:00:04.740 --> 00:00:06.000
and most importantly,
4
00:00:06.000 --> 00:00:09.450
a pleasure to have Dr. Brian Mittman,
5
00:00:09.450 --> 00:00:11.820
who we've been talking about bringing over here
6
00:00:11.820 --> 00:00:15.600
to CMIPS and Yale for quite some time.
7
00:00:15.600 --> 00:00:18.660
Dr. Mittman is a distinguished
8
00:00:18.660 --> 00:00:22.170
longstanding implementation scientist,
9
00:00:22.170 --> 00:00:24.540
I might even say one of the founders
10
00:00:24.540 --> 00:00:28.083
of implementation science as a formal discipline.
11
00:00:29.700 --> 00:00:33.000
He is a research scientist
12
00:00:33.000 --> 00:00:35.550
in the Department of Research and Evaluation
13
00:00:35.550 --> 00:00:37.350
with additional affiliations
14
00:00:37.350 --> 00:00:40.410
at the US Department of Veteran Affairs,
15
00:00:40.410 --> 00:00:43.800
which is another place where a huge amount
16
00:00:43.800 --> 00:00:47.130
of some of the best implementation science,
17
00:00:47.130 --> 00:00:50.310
thinking and research has emanated over the years,
18
00:00:50.310 --> 00:00:52.650
the University of Southern California
19
00:00:52.650 --> 00:00:55.620
and the University of California Los Angeles,
20
00:00:55.620 --> 00:00:56.490
where he co-leads
21
00:00:56.490 --> 00:01:01.200
the UCLA Clinical and Translational Science Institute
22
00:01:01.200 --> 00:01:04.710
Implementation and Improvement Science Initiative.
23
00:01:04.710 --> 00:01:06.840
And I find that very interesting
24
00:01:06.840 --> 00:01:09.870
in that the implementation and improvement science
25
00:01:09.870 --> 00:01:11.490
are linked in the same name,
26
00:01:11.490 --> 00:01:15.270
which is also something we at CMIPS are very interested
27
00:01:15.270 --> 00:01:19.917
in the kind of continuum between implementation science
28
00:01:19.917 --> 00:01:22.500
and improvement science, quality improvement,
29
00:01:22.500 --> 00:01:24.900
and what are the commonalities and differences
30
00:01:24.900 --> 00:01:28.380
and where does one end and the other begin.
31
00:01:28.380 --> 00:01:29.370
So I don't know if that's something
32
00:01:29.370 --> 00:01:32.163
Dr. Mittman is gonna touch upon in his talk today.
33
00:01:33.060 --> 00:01:34.830
He chaired the planning committee
34
00:01:34.830 --> 00:01:37.500
that launched the journal, "Implementation Science,"
35
00:01:37.500 --> 00:01:40.080
which now has a sort of spinoff journal,
36
00:01:40.080 --> 00:01:43.140
I forget its name, but now there's two of them,
37
00:01:43.140 --> 00:01:46.050
and served as co-editor in chief of that journal
38
00:01:46.050 --> 00:01:49.590
from 2005 to 2012.
39
00:01:49.590 --> 00:01:50.820
He was a founding member
40
00:01:50.820 --> 00:01:53.190
of the US Institute of Medicine Forum
41
00:01:53.190 --> 00:01:57.690
on the science of quality improvement and implementation
42
00:01:57.690 --> 00:02:00.390
and chair at the National Institutes of Health
43
00:02:00.390 --> 00:02:02.670
Special Emphasis Panel
44
00:02:02.670 --> 00:02:05.430
on Dissemination and Implementation Research in Health
45
00:02:05.430 --> 00:02:07.940
in 2007 and 2010.
46
00:02:07.940 --> 00:02:10.830
And for those of us in the audience
47
00:02:10.830 --> 00:02:14.160
who are thinking about NIH grants,
48
00:02:14.160 --> 00:02:17.491
what I've very recently learned is now that
49
00:02:17.491 --> 00:02:19.650
the DNI panel, as we call it,
50
00:02:19.650 --> 00:02:21.360
has been renamed.
51
00:02:21.360 --> 00:02:23.190
Maybe Dr. Mittman knows the name,
52
00:02:23.190 --> 00:02:25.290
I don't remember the name,
53
00:02:25.290 --> 00:02:28.443
and maybe even there's multiple ones of it now.
54
00:02:29.280 --> 00:02:30.330
But if you're interested,
55
00:02:30.330 --> 00:02:32.490
maybe write to me later and we can figure that out
56
00:02:32.490 --> 00:02:33.750
because it's very important
57
00:02:33.750 --> 00:02:37.740
for our implementation dissemination science applications
58
00:02:37.740 --> 00:02:39.720
to NIH here at Yale.
59
00:02:39.720 --> 00:02:41.040
Dr. Mittman directed
60
00:02:41.040 --> 00:02:44.340
the VA's Quality Enhancement Research Initiative
61
00:02:44.340 --> 00:02:47.100
from 2002 to 2004.
62
00:02:47.100 --> 00:02:50.130
His research examines innovative approaches
63
00:02:50.130 --> 00:02:52.740
to healthcare delivery and improvement
64
00:02:52.740 --> 00:02:55.830
and efforts to strengthen learning healthcare systems,
65
00:02:55.830 --> 00:02:59.490
another area in which we're very interested in CMIPS
66
00:02:59.490 --> 00:03:01.890
and many others at Yale are as well.
67
00:03:01.890 --> 00:03:04.920
So today, Dr. Mittman is gonna talk to us
68
00:03:04.920 --> 00:03:08.040
about Addressing Heterogeneity and Adaptability
69
00:03:08.040 --> 00:03:10.380
and Multi-Component Implementation
70
00:03:10.380 --> 00:03:12.360
and HIV Interventions:
71
00:03:12.360 --> 00:03:14.640
Emerging Frameworks for Research
72
00:03:14.640 --> 00:03:16.380
on Complex Health Interventions.
73
00:03:16.380 --> 00:03:18.510
And actually, I wanted to say one thing
74
00:03:18.510 --> 00:03:20.100
before I turn it over to him.
75
00:03:20.100 --> 00:03:21.940
He also serves as a consultant
76
00:03:23.531 --> 00:03:25.110
for our R3EDI Hub,
77
00:03:25.110 --> 00:03:27.670
which is a technical support hub
78
00:03:28.770 --> 00:03:30.780
that supports seven projects
79
00:03:30.780 --> 00:03:33.870
devoted to ending the AIDS epidemic
80
00:03:33.870 --> 00:03:38.280
under a general coordinating center based in Illinois.
81
00:03:38.280 --> 00:03:40.140
And it's been a pleasure to have Brian
82
00:03:40.140 --> 00:03:42.720
as a part of our R3EDI Hub team as well.
83
00:03:42.720 --> 00:03:44.970
So without any further ado now,
84
00:03:44.970 --> 00:03:47.700
I will turn things over to Dr. Mittman.
85
00:03:47.700 --> 00:03:49.500
Great, thank you, Donna,
86
00:03:49.500 --> 00:03:50.970
both for the kind introduction
87
00:03:50.970 --> 00:03:55.170
as well as more importantly the opportunity to present
88
00:03:55.170 --> 00:03:58.811
and to meet with some of your colleagues today and tomorrow.
89
00:03:58.811 --> 00:04:01.173
As we were saying before we started,
90
00:04:02.070 --> 00:04:03.540
my hope is to have the opportunity
91
00:04:03.540 --> 00:04:06.720
to join you in person at some point down the line,
92
00:04:06.720 --> 00:04:08.010
but I know we all share that hope
93
00:04:08.010 --> 00:04:10.560
for lots of in-person gatherings.
94
00:04:10.560 --> 00:04:13.350
You touched on several of my favorite topics,
95
00:04:13.350 --> 00:04:16.590
including implementation science, improvement science.
96
00:04:16.590 --> 00:04:19.050
I'll mention that I think very briefly
97
00:04:19.050 --> 00:04:21.450
as well as other topics.
98
00:04:21.450 --> 00:04:24.270
And I'm glad to schedule follow-up talks
99
00:04:24.270 --> 00:04:25.710
to speak about them.
100
00:04:25.710 --> 00:04:28.713
One quick comment on some of your kind remarks.
101
00:04:30.030 --> 00:04:32.340
I always counsel junior colleagues
102
00:04:32.340 --> 00:04:35.610
to pick a very small field that's likely to grow
103
00:04:35.610 --> 00:04:37.290
and get in on the ground floor
104
00:04:37.290 --> 00:04:39.210
because it makes you look important.
105
00:04:39.210 --> 00:04:42.660
That's sort of the big fish, small pond kind of idea.
106
00:04:42.660 --> 00:04:46.200
But also, the fact that I spend much more of my time
107
00:04:46.200 --> 00:04:48.670
advocating and helping to develop and expand
108
00:04:49.730 --> 00:04:53.760
fields that I'm interested in sometimes
109
00:04:53.760 --> 00:04:55.290
rather than actually doing the research,
110
00:04:55.290 --> 00:04:58.020
although I do have a research portfolio.
111
00:04:58.020 --> 00:05:00.210
So implementation science is a field
112
00:05:00.210 --> 00:05:04.710
that I was able to, again, get in on the ground floor
113
00:05:04.710 --> 00:05:06.810
and help to wave the flag,
114
00:05:06.810 --> 00:05:09.990
promote interest and advocate at NIH,
115
00:05:09.990 --> 00:05:12.540
at PCORI and many other places.
116
00:05:12.540 --> 00:05:14.905
And used to spend a lot of time
117
00:05:14.905 --> 00:05:16.920
on the freeways in Los Angeles
118
00:05:16.920 --> 00:05:19.890
traveling between different facilities and institutions
119
00:05:19.890 --> 00:05:21.690
as well as in planes trying to,
120
00:05:21.690 --> 00:05:24.060
again, advocate and promote interest
121
00:05:24.060 --> 00:05:25.890
in implementation science.
122
00:05:25.890 --> 00:05:27.510
But the implementation science field,
123
00:05:27.510 --> 00:05:30.270
in my view, is well established.
124
00:05:30.270 --> 00:05:33.390
There are many of us who are interested
125
00:05:33.390 --> 00:05:35.430
and have active research portfolios.
126
00:05:35.430 --> 00:05:37.470
I don't know that I have that much to offer at this point
127
00:05:37.470 --> 00:05:39.480
as far as new ideas,
128
00:05:39.480 --> 00:05:41.370
but I have a different view about the field
129
00:05:41.370 --> 00:05:43.140
of complex health interventions
130
00:05:43.140 --> 00:05:47.490
where I think there is a need to continue to think hard
131
00:05:47.490 --> 00:05:50.430
and promote some of the newer emerging frameworks
132
00:05:50.430 --> 00:05:53.940
and point out that as researchers are tasked
133
00:05:53.940 --> 00:05:56.390
in studying complex health interventions
134
00:05:56.390 --> 00:05:57.870
is a bit different from our tasks
135
00:05:57.870 --> 00:05:59.970
and studying other kinds of interventions.
136
00:06:02.670 --> 00:06:05.880
I think my most important focus lately in my research
137
00:06:05.880 --> 00:06:08.700
is trying to help, again, advocate
138
00:06:08.700 --> 00:06:12.360
and share information on some of these emerging frameworks
139
00:06:12.360 --> 00:06:15.120
and encourage more development.
140
00:06:15.120 --> 00:06:17.790
One more comment in terms of the truth in advertising.
141
00:06:17.790 --> 00:06:19.900
I actually won't spend much time at all
142
00:06:21.030 --> 00:06:23.460
talking about specific implementation science
143
00:06:23.460 --> 00:06:26.400
or HIV/AIDS intervention examples,
144
00:06:26.400 --> 00:06:28.050
but I think it'll be very clear
145
00:06:28.050 --> 00:06:31.230
as to how and why the comments that I will make
146
00:06:31.230 --> 00:06:35.340
are directly relevant to both of those bodies of activity.
147
00:06:35.340 --> 00:06:36.990
So let me move on
148
00:06:36.990 --> 00:06:41.130
and I try to remember which button allows me to advance.
149
00:06:41.130 --> 00:06:44.217
So let me start with a very high-level question,
150
00:06:44.217 --> 00:06:46.890
and that is ask us all to think a little bit
151
00:06:46.890 --> 00:06:49.650
about what we as researchers do
152
00:06:49.650 --> 00:06:53.970
in addition to producing scientific generalizable knowledge,
153
00:06:53.970 --> 00:06:56.940
what we do to support policy decision makers
154
00:06:56.940 --> 00:06:59.190
and practice decision makers questions.
155
00:06:59.190 --> 00:07:02.520
And much of the research that's conducted in medical schools
156
00:07:02.520 --> 00:07:05.190
and on other health-related institutions
157
00:07:05.190 --> 00:07:06.960
pursues these questions.
158
00:07:06.960 --> 00:07:09.090
Does it work or is it effective?
159
00:07:09.090 --> 00:07:10.710
The FDA of course would like to know
160
00:07:10.710 --> 00:07:12.570
if a new drug should be approved.
161
00:07:12.570 --> 00:07:14.310
CMS and others would like to know
162
00:07:14.310 --> 00:07:16.800
if it should be funded and promoted.
163
00:07:16.800 --> 00:07:18.810
Should it even be mandated?
164
00:07:18.810 --> 00:07:19.800
Within health systems,
165
00:07:19.800 --> 00:07:21.900
P&T committees have decisions to make
166
00:07:21.900 --> 00:07:25.680
about inclusion of new drugs in a formulary.
167
00:07:25.680 --> 00:07:28.980
And frontline practicing clinicians need to know
168
00:07:28.980 --> 00:07:31.200
whether they should use a new drug
169
00:07:31.200 --> 00:07:32.640
or another intervention.
170
00:07:32.640 --> 00:07:36.360
So much of the questions here
171
00:07:36.360 --> 00:07:39.780
in the guidance that we endeavor to provide
172
00:07:39.780 --> 00:07:43.440
to our policy and practice decision maker colleagues
173
00:07:43.440 --> 00:07:45.900
is a set of answers to these questions.
174
00:07:45.900 --> 00:07:46.740
Does it work?
175
00:07:46.740 --> 00:07:48.090
Is it effective?
176
00:07:48.090 --> 00:07:51.360
Or in the case of comparative effectiveness research,
177
00:07:51.360 --> 00:07:53.820
is intervention A better than B?
178
00:07:53.820 --> 00:07:57.330
And of course, we focus on outcomes and impacts
179
00:07:57.330 --> 00:08:00.660
when we try to answer this yes/no question.
180
00:08:00.660 --> 00:08:02.940
We often have the sample size and the funding
181
00:08:02.940 --> 00:08:05.880
and the ability to examine heterogeneity
182
00:08:05.880 --> 00:08:08.520
and subgroup effects and so on,
183
00:08:08.520 --> 00:08:10.050
and whether contextual factors
184
00:08:10.050 --> 00:08:12.660
influence the effects and outcomes.
185
00:08:12.660 --> 00:08:17.550
And of course, our gold standard research method of RCTs
186
00:08:17.550 --> 00:08:19.500
and similar experimental methods
187
00:08:19.500 --> 00:08:22.290
where we randomize and measure outcome differences,
188
00:08:22.290 --> 00:08:25.650
that's how we go about conducting this research.
189
00:08:25.650 --> 00:08:28.890
But again, the focus is on impact and outcomes.
190
00:08:28.890 --> 00:08:32.670
Are the outcomes better for those in the intervention group
191
00:08:32.670 --> 00:08:33.960
versus the control group?
192
00:08:33.960 --> 00:08:35.700
And if the answer is yes,
193
00:08:35.700 --> 00:08:37.950
then the intervention is effective,
194
00:08:37.950 --> 00:08:39.450
it's approved by the FDA,
195
00:08:39.450 --> 00:08:42.903
it's promoted, it's reimbursed and it's used.
196
00:08:43.860 --> 00:08:48.210
And there are in fact many examples of magic bullets
197
00:08:48.210 --> 00:08:51.420
or very strong robustly effective drugs
198
00:08:51.420 --> 00:08:54.480
for which we can produce a clear answer to that question.
199
00:08:54.480 --> 00:08:56.700
Yes, this drug is very effective.
200
00:08:56.700 --> 00:08:58.230
Precision medicine, of course,
201
00:08:58.230 --> 00:09:01.980
is leading us down the path for drugs and interventions
202
00:09:01.980 --> 00:09:04.110
for which there isn't a clear answer
203
00:09:04.110 --> 00:09:07.470
where there are high levels of heterogeneity.
204
00:09:07.470 --> 00:09:10.950
And we do need to tailor the interventions
205
00:09:10.950 --> 00:09:13.563
and that's really the theme of this talk.
206
00:09:14.460 --> 00:09:16.010
When we think about complex interventions
207
00:09:16.010 --> 00:09:17.910
or complex health interventions,
208
00:09:17.910 --> 00:09:20.190
and I'll define them more formally in a minute,
209
00:09:20.190 --> 00:09:24.900
but health promotion programs, HIV/AIDS prevention,
210
00:09:24.900 --> 00:09:28.920
treatment programs, implementation strategies,
211
00:09:28.920 --> 00:09:32.100
there are some examples of highly robust,
212
00:09:32.100 --> 00:09:34.290
highly effective complex health interventions
213
00:09:34.290 --> 00:09:38.010
for which we can produce a strong answer.
214
00:09:38.010 --> 00:09:40.770
Yes, this intervention tends to be effective
215
00:09:40.770 --> 00:09:43.059
across multiple settings
216
00:09:43.059 --> 00:09:46.140
and in multiple sets of circumstances.
217
00:09:46.140 --> 00:09:48.780
But by and large, for most complex health interventions,
218
00:09:48.780 --> 00:09:51.390
when we ask the question, is it effective?
219
00:09:51.390 --> 00:09:53.280
The answer that comes out of our research
220
00:09:53.280 --> 00:09:55.470
is sometimes or it depends.
221
00:09:55.470 --> 00:09:56.520
The heterogeneity,
222
00:09:56.520 --> 00:09:59.370
I'll sometimes use the term extreme heterogeneity,
223
00:09:59.370 --> 00:10:04.370
is just so great that the impacts vary considerably
224
00:10:04.530 --> 00:10:07.290
and it's impossible to produce a simple answer,
225
00:10:07.290 --> 00:10:10.830
yes or no, it is effective or it's not effective.
226
00:10:10.830 --> 00:10:14.430
So there is no formal established definition
227
00:10:14.430 --> 00:10:16.650
of complex health interventions at this point,
228
00:10:16.650 --> 00:10:18.960
but here are some of the key features
229
00:10:18.960 --> 00:10:22.380
that tend to be mentioned in most discussions
230
00:10:22.380 --> 00:10:24.090
of complex health interventions.
231
00:10:24.090 --> 00:10:26.070
The fact that there are multiple components
232
00:10:26.070 --> 00:10:28.620
and those components interact.
233
00:10:28.620 --> 00:10:31.590
The intervention, the multi-component intervention
234
00:10:31.590 --> 00:10:34.560
tends to target multiple levels,
235
00:10:34.560 --> 00:10:37.200
not always, but certainly multiple entities.
236
00:10:37.200 --> 00:10:39.450
So we have interventions that target patients
237
00:10:39.450 --> 00:10:42.810
and family caregivers and other peers
238
00:10:42.810 --> 00:10:45.783
as well as clinicians and other health system staff,
239
00:10:46.620 --> 00:10:48.240
as well as in many cases,
240
00:10:48.240 --> 00:10:51.303
communities and even regulatory levels.
241
00:10:52.380 --> 00:10:55.650
These interventions tend to be highly adaptable.
242
00:10:55.650 --> 00:10:56.730
They're not fixed.
243
00:10:56.730 --> 00:10:59.880
So unlike a drug that comes from the factory
244
00:10:59.880 --> 00:11:03.046
in a very consistent chemical formulation
245
00:11:03.046 --> 00:11:06.510
with a high degree of consistency and homogeneity,
246
00:11:06.510 --> 00:11:09.000
these interventions adapt.
247
00:11:09.000 --> 00:11:11.940
And that's the case even when we try to achieve fidelity
248
00:11:11.940 --> 00:11:13.560
to the manualized intervention
249
00:11:13.560 --> 00:11:17.700
and prevent adaptations and modifications,
250
00:11:17.700 --> 00:11:20.610
and that's another theme I'll come back to.
251
00:11:20.610 --> 00:11:22.110
Because of all of these features,
252
00:11:22.110 --> 00:11:25.110
the interventions tend to achieve their effects
253
00:11:25.110 --> 00:11:26.820
through multiple pathways
254
00:11:26.820 --> 00:11:28.620
and they tend to be mediated.
255
00:11:28.620 --> 00:11:31.200
So it's not a drug that has a direct impact
256
00:11:31.200 --> 00:11:32.970
on a physiologic process,
257
00:11:32.970 --> 00:11:34.770
but instead an intervention
258
00:11:34.770 --> 00:11:38.040
that changes attitudes or beliefs,
259
00:11:38.040 --> 00:11:40.170
those changes in attitudes and beliefs
260
00:11:40.170 --> 00:11:42.780
lead to changes in knowledge and intentions.
261
00:11:42.780 --> 00:11:44.610
Those changes in knowledge and intentions
262
00:11:44.610 --> 00:11:46.950
eventually lead to changes in behavior.
263
00:11:46.950 --> 00:11:50.130
But those behaviors are influenced by multiple factors.
264
00:11:50.130 --> 00:11:52.560
So it's not only the patient's own beliefs
265
00:11:52.560 --> 00:11:54.270
and knowledge and attitudes,
266
00:11:54.270 --> 00:11:57.123
but peer influence, clinician influence,
267
00:11:57.990 --> 00:12:00.330
social influence from key opinion leaders
268
00:12:00.330 --> 00:12:01.380
and a number of others.
269
00:12:01.380 --> 00:12:04.530
So the causal pathways tend to be very complex
270
00:12:04.530 --> 00:12:07.080
and I'll illustrate that in a few minutes.
271
00:12:07.080 --> 00:12:09.390
So when we think about a comparison
272
00:12:09.390 --> 00:12:12.330
between simple interventions like drugs
273
00:12:12.330 --> 00:12:14.070
versus complex interventions,
274
00:12:14.070 --> 00:12:16.387
these are some of the key dimensions
275
00:12:16.387 --> 00:12:18.480
where there are differences.
276
00:12:18.480 --> 00:12:20.820
The difference between a single fixed
277
00:12:20.820 --> 00:12:23.790
and highly stable and homogeneous drug
278
00:12:23.790 --> 00:12:27.750
that targets a single stable physiologic process
279
00:12:27.750 --> 00:12:31.680
to achieve a simple goal such as reducing blood pressure
280
00:12:31.680 --> 00:12:35.670
in patients that are not always homogeneous.
281
00:12:35.670 --> 00:12:37.140
There are differences,
282
00:12:37.140 --> 00:12:40.230
but the argument is that patients,
283
00:12:40.230 --> 00:12:42.360
despite genetic profile differences
284
00:12:42.360 --> 00:12:43.740
and other physiologic,
285
00:12:43.740 --> 00:12:46.980
as well as clearly socioeconomic status
286
00:12:46.980 --> 00:12:51.570
in neighborhood and contextual differences,
287
00:12:51.570 --> 00:12:53.940
those differences tend to be somewhat smaller
288
00:12:53.940 --> 00:12:55.260
than the differences we see
289
00:12:55.260 --> 00:12:57.753
across communities and organizations.
290
00:12:58.620 --> 00:13:00.360
And again, we can argue that point,
291
00:13:00.360 --> 00:13:02.730
but these are the key distinctions
292
00:13:02.730 --> 00:13:05.610
between these two categories of interventions.
293
00:13:05.610 --> 00:13:07.230
And the consequences, of course,
294
00:13:07.230 --> 00:13:09.723
or the implications for research are that,
295
00:13:10.620 --> 00:13:12.930
when we study drugs, oftentimes,
296
00:13:12.930 --> 00:13:15.330
not always, but oftentimes we do see
297
00:13:15.330 --> 00:13:18.180
a relatively high level of homogeneity
298
00:13:18.180 --> 00:13:20.820
with very consistent and often strong,
299
00:13:20.820 --> 00:13:23.040
easily detected main effects.
300
00:13:23.040 --> 00:13:25.170
Whereas again, with complex interventions,
301
00:13:25.170 --> 00:13:26.850
we get the answer along the lines
302
00:13:26.850 --> 00:13:28.950
of it depends or sometimes.
303
00:13:28.950 --> 00:13:32.580
We see lots of complexity, instability and heterogeneity.
304
00:13:32.580 --> 00:13:33.990
And the average effects,
305
00:13:33.990 --> 00:13:36.693
because of the heterogeneity, tend to be very weak.
306
00:13:37.680 --> 00:13:40.230
We have many subjects or targets in the intervention
307
00:13:40.230 --> 00:13:43.110
that do very well, others that do very poorly,
308
00:13:43.110 --> 00:13:45.630
but on average, an average effect size estimates
309
00:13:45.630 --> 00:13:47.550
that are close to zero.
310
00:13:47.550 --> 00:13:49.680
One key point, and that is,
311
00:13:49.680 --> 00:13:53.010
this is not a dichotomy, but instead of a continuum.
312
00:13:53.010 --> 00:13:56.370
There are elements of complexity in all interventions.
313
00:13:56.370 --> 00:13:58.110
The key question is,
314
00:13:58.110 --> 00:14:00.900
when is an intervention sufficiently complex
315
00:14:00.900 --> 00:14:03.690
that we can't study it through an RCT
316
00:14:03.690 --> 00:14:06.690
with a focus on average effect sizes,
317
00:14:06.690 --> 00:14:09.870
but instead need to use the more complex kinds of approaches
318
00:14:09.870 --> 00:14:12.420
that I'll talk about over the next several minutes.
319
00:14:13.560 --> 00:14:15.510
So getting back to this question,
320
00:14:15.510 --> 00:14:16.767
does it work, is it effective?
321
00:14:16.767 --> 00:14:19.980
And the answer being sometimes or it depends,
322
00:14:19.980 --> 00:14:21.000
that answer, of course,
323
00:14:21.000 --> 00:14:23.160
is not at all useful for decision makers.
324
00:14:23.160 --> 00:14:25.050
So we need to think about a different way
325
00:14:25.050 --> 00:14:27.630
of designing, conducting our studies
326
00:14:27.630 --> 00:14:29.520
and a different type of evidence
327
00:14:29.520 --> 00:14:32.790
or a set of insights and findings
328
00:14:32.790 --> 00:14:35.550
that we need to produce for science,
329
00:14:35.550 --> 00:14:38.250
but also for policy and practice.
330
00:14:38.250 --> 00:14:40.080
So let me back up and illustrate
331
00:14:40.080 --> 00:14:41.700
some of the challenges that we face
332
00:14:41.700 --> 00:14:43.590
when we deal with complex health interventions.
333
00:14:43.590 --> 00:14:47.190
So this is a pattern of results from a hypothetical study
334
00:14:47.190 --> 00:14:50.412
that could be a guideline implementation study.
335
00:14:50.412 --> 00:14:53.310
We are attempting to improve adherence
336
00:14:53.310 --> 00:14:56.250
to evidence-based clinical practice guidelines.
337
00:14:56.250 --> 00:15:00.450
In the blue sample,
338
00:15:00.450 --> 00:15:02.340
the blue bars in this histogram
339
00:15:02.340 --> 00:15:04.860
shows that all of the sites
340
00:15:04.860 --> 00:15:07.500
in the intervention group did very well.
341
00:15:07.500 --> 00:15:09.120
Our intervention managed
342
00:15:09.120 --> 00:15:12.150
to significantly improve rates of adherence
343
00:15:12.150 --> 00:15:14.160
among all the intervention physicians
344
00:15:14.160 --> 00:15:16.380
or clinics or hospitals,
345
00:15:16.380 --> 00:15:19.440
whereas the sites in the yellow or light green
346
00:15:19.440 --> 00:15:22.500
are all scattered around zero.
347
00:15:22.500 --> 00:15:25.920
So on average, we saw no change in adherence levels
348
00:15:25.920 --> 00:15:29.640
among the usual care comparison sites,
349
00:15:29.640 --> 00:15:31.560
although some of course did better and some did worse.
350
00:15:31.560 --> 00:15:34.860
It's just because of random variation.
351
00:15:34.860 --> 00:15:37.440
I don't know that we've ever seen findings
352
00:15:37.440 --> 00:15:39.120
from any implementation study
353
00:15:39.120 --> 00:15:42.330
that resembled this kind of pattern
354
00:15:42.330 --> 00:15:43.830
or anything close to it.
355
00:15:43.830 --> 00:15:46.320
This clearly would be "New England Journal"
356
00:15:46.320 --> 00:15:49.170
or "Lancet" caliber work
357
00:15:49.170 --> 00:15:51.450
if we had a strong finding of this sort,
358
00:15:51.450 --> 00:15:53.910
but that's what we would hope to see with our interventions,
359
00:15:53.910 --> 00:15:56.880
that we would find or design an intervention
360
00:15:56.880 --> 00:16:01.440
and see very robust, very significant effects.
361
00:16:01.440 --> 00:16:03.180
This is what we tend to see more often
362
00:16:03.180 --> 00:16:05.160
when we study complex health interventions.
363
00:16:05.160 --> 00:16:06.840
There's almost complete overlap
364
00:16:06.840 --> 00:16:10.950
between the blue and the light green yellow sites.
365
00:16:10.950 --> 00:16:13.050
If you are an intervention site,
366
00:16:13.050 --> 00:16:16.290
you are almost as likely to show
367
00:16:16.290 --> 00:16:17.640
reduced rates of adherence
368
00:16:17.640 --> 00:16:19.140
as you are increases.
369
00:16:19.140 --> 00:16:21.960
And similarly, the usual care sites,
370
00:16:21.960 --> 00:16:24.450
many of them did show rates of improvement
371
00:16:24.450 --> 00:16:27.750
that are comparable to those in the intervention site.
372
00:16:27.750 --> 00:16:30.303
So when you have a pattern of results like this,
373
00:16:31.620 --> 00:16:34.110
you can't say to decision makers,
374
00:16:34.110 --> 00:16:35.760
my complex health intervention,
375
00:16:35.760 --> 00:16:38.340
my HIV/AIDS prevention program
376
00:16:38.340 --> 00:16:40.350
or my implementation strategy
377
00:16:40.350 --> 00:16:43.320
or quality improvement program is highly effective,
378
00:16:43.320 --> 00:16:44.793
I would advise you to use it.
379
00:16:45.634 --> 00:16:46.890
As a decision maker,
380
00:16:46.890 --> 00:16:48.510
if I know I'm almost likely
381
00:16:48.510 --> 00:16:50.370
to end up spending a lot of money
382
00:16:50.370 --> 00:16:53.520
and staff time and disruption
383
00:16:53.520 --> 00:16:56.490
and end up with decreased performance,
384
00:16:56.490 --> 00:16:58.920
obviously, I'm not going to be interested in this program.
385
00:16:58.920 --> 00:17:01.740
So what is our goal then as a researcher?
386
00:17:01.740 --> 00:17:03.180
Our goal, of course, is to understand
387
00:17:03.180 --> 00:17:06.120
who ended up on the right hand side of this distribution,
388
00:17:06.120 --> 00:17:08.520
what the factors were that led to those improvements
389
00:17:08.520 --> 00:17:11.247
for both intervention as well as control sites,
390
00:17:11.247 --> 00:17:14.460
and what can we do to counsel decision makers
391
00:17:14.460 --> 00:17:17.550
to allow them to end up on the right hand side
392
00:17:17.550 --> 00:17:20.433
rather than the left hand side of the distribution.
393
00:17:21.570 --> 00:17:24.870
So when we think about finding
394
00:17:24.870 --> 00:17:27.600
or designing developing complex health interventions
395
00:17:27.600 --> 00:17:28.623
that are effective,
396
00:17:29.580 --> 00:17:31.410
one position that we could take
397
00:17:31.410 --> 00:17:34.680
is our goal as researchers is to develop
398
00:17:34.680 --> 00:17:37.350
and generate the evidence showing
399
00:17:37.350 --> 00:17:41.220
that our interventions are highly effective,
400
00:17:41.220 --> 00:17:44.490
but that assumes that those interventions exist.
401
00:17:44.490 --> 00:17:46.623
Hang on one second, I will be right back.
402
00:17:53.100 --> 00:17:54.150
While he's out,
403
00:17:54.150 --> 00:17:57.420
I can say if people have questions or comments,
404
00:17:57.420 --> 00:17:59.880
why don't you put them in the chat as we go along
405
00:17:59.880 --> 00:18:02.760
and then at the end of Brian's talk,
406
00:18:02.760 --> 00:18:06.180
I'll pose some of the questions and comments to him.
407
00:18:06.180 --> 00:18:07.860
Go ahead, Brian.
Thank you.
408
00:18:07.860 --> 00:18:09.420
Yeah, so my apologies.
409
00:18:09.420 --> 00:18:10.290
For those who joined earlier,
410
00:18:10.290 --> 00:18:12.640
we were talking about the renovations underway.
411
00:18:13.800 --> 00:18:15.330
My wife was stuck outside
412
00:18:15.330 --> 00:18:17.370
because I forgot to open the door
413
00:18:17.370 --> 00:18:19.980
for the second pathway into the kitchen
414
00:18:19.980 --> 00:18:24.980
because the main path is covered with paint paraphernalia.
415
00:18:25.140 --> 00:18:28.860
So again, because complex health interventions
416
00:18:28.860 --> 00:18:30.240
tend not to be robust
417
00:18:30.240 --> 00:18:31.860
and we tend not to have the ability
418
00:18:31.860 --> 00:18:35.680
to find or develop the needle in the haystack
419
00:18:37.230 --> 00:18:38.700
or they don't exist at all,
420
00:18:38.700 --> 00:18:40.700
that a robust complex health intervention
421
00:18:40.700 --> 00:18:42.273
is a mythical beast,
422
00:18:43.650 --> 00:18:46.470
we need to take a different strategy
423
00:18:46.470 --> 00:18:47.490
and a different approach
424
00:18:47.490 --> 00:18:49.920
in designing and conducting research
425
00:18:49.920 --> 00:18:53.580
and supporting health decision makers.
426
00:18:53.580 --> 00:18:55.770
So rather than pursuing questions
427
00:18:55.770 --> 00:18:58.020
such as is it effective or does it work
428
00:18:58.020 --> 00:19:00.060
or which is more effective,
429
00:19:00.060 --> 00:19:02.490
we need to be thinking about deriving
430
00:19:02.490 --> 00:19:05.760
and developing insights and guidance for practice,
431
00:19:05.760 --> 00:19:08.730
such as how does it work, why does it work?
432
00:19:08.730 --> 00:19:09.840
Where, when and for whom,
433
00:19:09.840 --> 00:19:12.960
the realistic evaluation key questions,
434
00:19:12.960 --> 00:19:15.360
but also how can we enhance its effectiveness?
435
00:19:15.360 --> 00:19:18.393
Which again gets back to this issue of adaptability.
436
00:19:19.350 --> 00:19:21.210
We'd have very few degrees of freedom
437
00:19:21.210 --> 00:19:24.060
to enhance the effectiveness of a drug.
438
00:19:24.060 --> 00:19:25.920
We can obviously titrate the dose
439
00:19:25.920 --> 00:19:29.550
and we can prescribe supportive interventions,
440
00:19:29.550 --> 00:19:32.493
but we can't modify the chemical formulation of the drug.
441
00:19:33.390 --> 00:19:36.600
We can modify the so-called chemical formulation
442
00:19:36.600 --> 00:19:37.950
of a complex health intervention.
443
00:19:37.950 --> 00:19:40.920
So our goal and our task as researchers
444
00:19:40.920 --> 00:19:43.983
is to guide that tailoring and that adaptation.
445
00:19:45.030 --> 00:19:49.830
So we should strive to support decision makers
446
00:19:49.830 --> 00:19:52.110
as they try to answer these questions.
447
00:19:52.110 --> 00:19:55.920
How do I choose an appropriate complex health intervention?
448
00:19:55.920 --> 00:19:58.380
How do I implement or deploy that program
449
00:19:58.380 --> 00:20:01.890
and tailor it to increase its effectiveness?
450
00:20:01.890 --> 00:20:04.980
But also how do I modify or manage the organization?
451
00:20:04.980 --> 00:20:08.610
Oftentimes just as we can improve health outcomes
452
00:20:08.610 --> 00:20:11.670
by changing diet and exercise
453
00:20:11.670 --> 00:20:15.570
and changing the social surroundings of our patients,
454
00:20:15.570 --> 00:20:18.330
we can certainly improve outcomes
455
00:20:18.330 --> 00:20:19.950
for complex health interventions
456
00:20:19.950 --> 00:20:22.110
by modifying the organization.
457
00:20:22.110 --> 00:20:24.930
So again, another task for researchers.
458
00:20:24.930 --> 00:20:26.790
But back to the key questions.
459
00:20:26.790 --> 00:20:29.880
We need to understand and develop insights
460
00:20:29.880 --> 00:20:33.090
and provide guidance regarding how, when, why
461
00:20:33.090 --> 00:20:35.070
and where do these interventions work
462
00:20:35.070 --> 00:20:38.013
and how can we modify them to make them work.
463
00:20:39.210 --> 00:20:44.210
So the focus here instead of on impact in simply asking,
464
00:20:44.370 --> 00:20:46.110
does intervention A produce
465
00:20:46.110 --> 00:20:48.990
a greater impact or outcome than intervention B,
466
00:20:48.990 --> 00:20:52.170
we need to instead focus on the black box.
467
00:20:52.170 --> 00:20:55.080
We need to understand the mediators and the moderators,
468
00:20:55.080 --> 00:20:56.910
the mechanisms of effect.
469
00:20:56.910 --> 00:20:59.700
We need to explicitly study adaptation
470
00:20:59.700 --> 00:21:02.190
and we need to study context
471
00:21:02.190 --> 00:21:03.540
and how to manage context.
472
00:21:03.540 --> 00:21:07.110
So again, another point related to the key theme
473
00:21:07.110 --> 00:21:09.870
of different types of research,
474
00:21:09.870 --> 00:21:11.670
not a focus on measuring impact,
475
00:21:11.670 --> 00:21:16.503
but instead to focus on understanding and studying process.
476
00:21:17.610 --> 00:21:22.410
So again, rather than thinking about evidence-based practice
477
00:21:22.410 --> 00:21:26.910
and generating or producing an estimate of effect sizes,
478
00:21:26.910 --> 00:21:30.030
to me, research on complex health interventions
479
00:21:30.030 --> 00:21:31.860
should focus on deriving
480
00:21:31.860 --> 00:21:34.470
or developing insights and guidance.
481
00:21:34.470 --> 00:21:37.830
So it's insights and guidance rather than evidence
482
00:21:37.830 --> 00:21:40.180
in the way that we typically think of evidence.
483
00:21:41.730 --> 00:21:44.280
So, getting back to the features
484
00:21:44.280 --> 00:21:46.650
or the characteristics of complex health interventions
485
00:21:46.650 --> 00:21:50.640
and why they tend to have such weak average effect sizes
486
00:21:50.640 --> 00:21:52.950
and such extreme heterogeneity,
487
00:21:52.950 --> 00:21:55.159
we know as I argued that,
488
00:21:55.159 --> 00:21:57.000
or we believe or I would assert
489
00:21:57.000 --> 00:21:59.280
that the intervention targets and settings
490
00:21:59.280 --> 00:22:01.290
are much more heterogeneous.
491
00:22:01.290 --> 00:22:04.440
Communities differ, individuals differ,
492
00:22:04.440 --> 00:22:06.860
and the same behavioral approach that we use
493
00:22:06.860 --> 00:22:10.560
or the same implementation strategy for one hospital
494
00:22:10.560 --> 00:22:13.320
is not likely to be effective
495
00:22:13.320 --> 00:22:16.080
or to work in the same way as in another hospital.
496
00:22:16.080 --> 00:22:19.860
Differences in hospital leadership and culture
497
00:22:19.860 --> 00:22:23.130
and staffing patterns and resources and so on
498
00:22:23.130 --> 00:22:27.210
all mediate and moderate the effects of the intervention.
499
00:22:27.210 --> 00:22:28.860
If we think about health psychology
500
00:22:28.860 --> 00:22:30.603
and patient behavior change,
501
00:22:30.603 --> 00:22:32.550
and one of the topics that we're studying
502
00:22:32.550 --> 00:22:35.970
in Kaiser Southern California, which is HPV vaccination,
503
00:22:35.970 --> 00:22:38.100
we know that clinician brief interventions
504
00:22:38.100 --> 00:22:41.610
are likely to be effective for some patients and parents
505
00:22:41.610 --> 00:22:44.340
who retain respect for their physicians
506
00:22:44.340 --> 00:22:46.710
and will follow their advice.
507
00:22:46.710 --> 00:22:48.000
But for other patients,
508
00:22:48.000 --> 00:22:49.620
that physician brief intervention
509
00:22:49.620 --> 00:22:51.750
can in fact be counterproductive
510
00:22:51.750 --> 00:22:53.430
because it reinforces a patient's
511
00:22:53.430 --> 00:22:55.440
or parent's op priori belief
512
00:22:55.440 --> 00:22:57.990
that these vaccines are poison
513
00:22:57.990 --> 00:23:01.170
and my physician is sort of an agent
514
00:23:01.170 --> 00:23:05.100
of the drug company trying to enhance profits.
515
00:23:05.100 --> 00:23:06.990
So again, lots of heterogeneity
516
00:23:06.990 --> 00:23:09.450
in the targets in the settings.
517
00:23:09.450 --> 00:23:11.790
We also know that the underlying pathologies,
518
00:23:11.790 --> 00:23:13.920
their etiology, their root causes differ.
519
00:23:13.920 --> 00:23:16.683
And again, the vaccine example is a good one.
520
00:23:16.683 --> 00:23:20.130
When we're dealing with low vaccination rates
521
00:23:20.130 --> 00:23:22.710
in a set of clinics or hospitals
522
00:23:22.710 --> 00:23:25.860
where patients tend to be respectful
523
00:23:25.860 --> 00:23:28.560
and responsive to brief interventions,
524
00:23:28.560 --> 00:23:33.090
we can suspect that the reason for low rates of adherence
525
00:23:33.090 --> 00:23:35.250
don't relate to patient resistance,
526
00:23:35.250 --> 00:23:38.370
but instead, physicians and staff or the systems
527
00:23:38.370 --> 00:23:41.730
not necessarily optimizing their activities.
528
00:23:41.730 --> 00:23:45.720
Whereas in other parts of Kaiser Southern California,
529
00:23:45.720 --> 00:23:48.060
we know that the hospitals and the clinics
530
00:23:48.060 --> 00:23:49.800
and the organizational policies
531
00:23:49.800 --> 00:23:52.560
and the clinicians are doing everything in their power
532
00:23:52.560 --> 00:23:55.020
to improve vaccination rates.
533
00:23:55.020 --> 00:23:56.940
The reason for low vaccination rates
534
00:23:56.940 --> 00:23:59.370
is patient and parent resistance
535
00:23:59.370 --> 00:24:02.100
that is tied to their own beliefs.
536
00:24:02.100 --> 00:24:03.737
So understanding differences
537
00:24:03.737 --> 00:24:08.737
in the root causes of low adherence rates
538
00:24:08.880 --> 00:24:10.585
or quality or outcomes
539
00:24:10.585 --> 00:24:12.870
or poor patient behavior
540
00:24:12.870 --> 00:24:15.360
and recognizing the heterogeneity,
541
00:24:15.360 --> 00:24:16.377
again, is important.
542
00:24:16.377 --> 00:24:17.880
And that's one of the reasons
543
00:24:17.880 --> 00:24:21.870
for the highly variable effects of interventions
544
00:24:21.870 --> 00:24:23.520
because they sometimes address
545
00:24:23.520 --> 00:24:25.170
the root causes and solve the problem,
546
00:24:25.170 --> 00:24:28.260
but other times the same intervention does not.
547
00:24:28.260 --> 00:24:29.970
And then finally, as I've said,
548
00:24:29.970 --> 00:24:32.880
the interventions themselves tend to be highly variable
549
00:24:32.880 --> 00:24:34.530
and irrespective of our efforts
550
00:24:34.530 --> 00:24:37.750
to achieve adherence to a manualized intervention
551
00:24:38.640 --> 00:24:41.070
and achieve high rates of fidelity,
552
00:24:41.070 --> 00:24:44.520
we know that we won't always see that intervention
553
00:24:44.520 --> 00:24:46.683
be delivered the same way across sites.
554
00:24:47.670 --> 00:24:49.260
There's drift over time,
555
00:24:49.260 --> 00:24:50.760
there are local adaptations,
556
00:24:50.760 --> 00:24:52.110
but again, more importantly,
557
00:24:52.110 --> 00:24:55.230
we shouldn't try to achieve fidelity
558
00:24:55.230 --> 00:24:56.910
because one version of intervention
559
00:24:56.910 --> 00:25:00.150
that does match local circumstances in one setting
560
00:25:00.150 --> 00:25:01.830
is not likely to be effective
561
00:25:01.830 --> 00:25:04.260
and match local circumstances elsewhere.
562
00:25:04.260 --> 00:25:08.130
So the adaptability of interventions,
563
00:25:08.130 --> 00:25:10.530
their heterogeneity across place,
564
00:25:10.530 --> 00:25:13.470
but also across time is a challenge.
565
00:25:13.470 --> 00:25:14.970
But we should view it as a strength
566
00:25:14.970 --> 00:25:18.930
that we need to embrace and use to our advantage.
567
00:25:18.930 --> 00:25:22.290
So some of you who are in my generation or have kids
568
00:25:22.290 --> 00:25:24.060
because I believe this game is still sold,
569
00:25:24.060 --> 00:25:26.860
will recognize the image in the upper right hand corner.
570
00:25:27.720 --> 00:25:29.040
And this is the way that I often think
571
00:25:29.040 --> 00:25:31.320
about complex health interventions,
572
00:25:31.320 --> 00:25:34.260
that if we were to watch the very beginning
573
00:25:34.260 --> 00:25:36.270
of the mouse trap contraption
574
00:25:36.270 --> 00:25:37.980
where we drop the marble
575
00:25:37.980 --> 00:25:41.100
and then focus only on the very end
576
00:25:41.100 --> 00:25:43.203
and whether the trap falls or not,
577
00:25:44.040 --> 00:25:45.840
sometimes it will, sometimes it won't.
578
00:25:45.840 --> 00:25:48.570
But that set of empirical observations
579
00:25:48.570 --> 00:25:49.980
doesn't help us at all
580
00:25:49.980 --> 00:25:53.790
in improving the performance of this mouse trap.
581
00:25:53.790 --> 00:25:57.120
We need to follow every step in the causal chain
582
00:25:57.120 --> 00:25:59.310
and understand which part of the contraption
583
00:25:59.310 --> 00:26:00.720
was not built correctly
584
00:26:00.720 --> 00:26:02.970
or where things are going wrong.
585
00:26:02.970 --> 00:26:05.700
So again, the question is not,
586
00:26:05.700 --> 00:26:07.920
is it effective, but how does it work?
587
00:26:07.920 --> 00:26:10.770
And we need to shine our spotlight,
588
00:26:10.770 --> 00:26:13.290
our flashlight and our research attention
589
00:26:13.290 --> 00:26:15.510
in terms of data collection analysis
590
00:26:15.510 --> 00:26:18.241
on the mechanisms of effect.
591
00:26:18.241 --> 00:26:20.940
As I've said, we need to,
592
00:26:20.940 --> 00:26:24.630
rather than try to ignore adaptations or suppress them,
593
00:26:24.630 --> 00:26:26.010
we need to embrace them.
594
00:26:26.010 --> 00:26:29.580
We need to study and guide those adaptations.
595
00:26:29.580 --> 00:26:31.740
The concept of a manualized intervention,
596
00:26:31.740 --> 00:26:33.660
I think for a complex health intervention
597
00:26:33.660 --> 00:26:35.490
requires rethinking.
598
00:26:35.490 --> 00:26:37.440
My favorite example here is a story
599
00:26:37.440 --> 00:26:39.330
that I believe is accurate
600
00:26:39.330 --> 00:26:41.970
of one of the sites
601
00:26:41.970 --> 00:26:46.740
in one of the patient self-management studies
602
00:26:46.740 --> 00:26:51.000
where the patient self-management program
603
00:26:51.000 --> 00:26:54.540
had a highly detailed manualized intervention,
604
00:26:54.540 --> 00:26:57.540
including a very clear script
605
00:26:57.540 --> 00:27:02.160
for the leader of a patient self-management education group
606
00:27:02.160 --> 00:27:06.480
to use in educating members of the group.
607
00:27:06.480 --> 00:27:09.000
And the story is that members of the study team
608
00:27:09.000 --> 00:27:10.800
were observing a leader
609
00:27:10.800 --> 00:27:13.560
deliver the patient self-management program
610
00:27:13.560 --> 00:27:16.122
in an African American church in Baltimore.
611
00:27:16.122 --> 00:27:21.090
And the leader of that program was not following the script.
612
00:27:21.090 --> 00:27:24.540
She was making up the comments
613
00:27:24.540 --> 00:27:27.180
and the educational content as she went along
614
00:27:27.180 --> 00:27:29.580
and the research assistants who were observing
615
00:27:29.580 --> 00:27:30.930
came up to her afterwards
616
00:27:30.930 --> 00:27:33.900
and congratulated her on a successful session,
617
00:27:33.900 --> 00:27:34.733
but said,
618
00:27:34.733 --> 00:27:37.590
"I noticed that you were deviating from the script.
619
00:27:37.590 --> 00:27:38.580
Why is that?
620
00:27:38.580 --> 00:27:41.340
Don't you know that this is an evidence-based intervention?
621
00:27:41.340 --> 00:27:42.870
And if you follow the manual
622
00:27:42.870 --> 00:27:44.730
and follow the script to the letter,
623
00:27:44.730 --> 00:27:47.070
you're guaranteed to see positive outcomes,
624
00:27:47.070 --> 00:27:48.990
but if you deviate from it,
625
00:27:48.990 --> 00:27:51.720
we don't know what sort of outcomes you will observe."
626
00:27:51.720 --> 00:27:53.797
And the leader of the church group said,
627
00:27:53.797 --> 00:27:57.150
"Well, as you know, your manual and your script
628
00:27:57.150 --> 00:27:59.400
was written in Stanford English.
629
00:27:59.400 --> 00:28:00.990
We don't speak Stanford English here.
630
00:28:00.990 --> 00:28:03.690
So I was using language and concepts
631
00:28:03.690 --> 00:28:04.950
and ideas and examples
632
00:28:04.950 --> 00:28:09.150
that I felt were more suitable for my local circumstances.
633
00:28:09.150 --> 00:28:11.730
So that's a somewhat extreme example, of course,
634
00:28:11.730 --> 00:28:14.700
but it does point out that a manualized intervention
635
00:28:14.700 --> 00:28:17.900
typically was developed from a study
636
00:28:17.900 --> 00:28:21.090
at a specific point in time in a specific region,
637
00:28:21.090 --> 00:28:23.280
in a specific set of settings.
638
00:28:23.280 --> 00:28:26.220
And the details of that intervention
639
00:28:26.220 --> 00:28:31.220
might in fact be highly optimal for that particular setting,
640
00:28:31.350 --> 00:28:33.600
but are not likely to be feasible
641
00:28:33.600 --> 00:28:36.810
and certainly not optimal for other settings.
642
00:28:36.810 --> 00:28:38.730
So again, we have to rethink the concept
643
00:28:38.730 --> 00:28:40.440
of manualized interventions.
644
00:28:40.440 --> 00:28:44.100
Similarly, we have to rethink the concept of core components
645
00:28:44.100 --> 00:28:45.360
and the term, core components,
646
00:28:45.360 --> 00:28:47.550
that concept is used relatively broadly,
647
00:28:47.550 --> 00:28:52.140
but oftentimes it talks about the intervention activities,
648
00:28:52.140 --> 00:28:55.680
the scripts, the tools, the protocols, the procedures.
649
00:28:55.680 --> 00:28:59.310
And again, those tend to be highly idiosyncratic
650
00:28:59.310 --> 00:29:01.590
and often optimized and developed
651
00:29:01.590 --> 00:29:06.590
by and for a specific set of settings
652
00:29:06.690 --> 00:29:08.370
in target audiences.
653
00:29:08.370 --> 00:29:12.690
So the alternative to the concept of core components
654
00:29:12.690 --> 00:29:15.720
and that way of thinking about complex health interventions
655
00:29:15.720 --> 00:29:19.920
is to specify a set of core functions in a menu of forms.
656
00:29:19.920 --> 00:29:22.500
And I'll talk through that in a few minutes.
657
00:29:22.500 --> 00:29:25.080
But let me just briefly point out
658
00:29:25.080 --> 00:29:27.690
that in the implementation field,
659
00:29:27.690 --> 00:29:30.870
and again using guideline adherence as an example,
660
00:29:30.870 --> 00:29:33.330
as I said, we often have very complex,
661
00:29:33.330 --> 00:29:35.730
multi-path, mediated
662
00:29:35.730 --> 00:29:40.290
and highly moderated sorts of causal pathways.
663
00:29:40.290 --> 00:29:44.580
And a typical multi-component guideline adherence program
664
00:29:44.580 --> 00:29:47.070
targeting physicians has to worry
665
00:29:47.070 --> 00:29:48.930
about the physician's attitudes and norms
666
00:29:48.930 --> 00:29:50.340
and try to address them
667
00:29:50.340 --> 00:29:51.870
as well as their knowledge and skill,
668
00:29:51.870 --> 00:29:54.270
as well as their motivation of their activation.
669
00:29:55.200 --> 00:29:58.830
And many of these are influenced
670
00:29:58.830 --> 00:30:02.670
by, again, multiple mediated pathways,
671
00:30:02.670 --> 00:30:06.030
but also some of those causal effects are highly moderated.
672
00:30:06.030 --> 00:30:10.800
We know that a financial incentive to follow the guideline
673
00:30:10.800 --> 00:30:13.500
that consists of a $20,000 bonus
674
00:30:13.500 --> 00:30:16.200
is likely to be highly effective
675
00:30:16.200 --> 00:30:19.050
for a junior family physician
676
00:30:19.050 --> 00:30:22.440
with an income in the $150,000 range.
677
00:30:22.440 --> 00:30:24.600
But for the senior surgeon
678
00:30:24.600 --> 00:30:28.410
with a multimillion dollar income
679
00:30:28.410 --> 00:30:32.760
who knows how to practice and doesn't need the guidelines,
680
00:30:32.760 --> 00:30:35.250
that bonus is not likely to have much effect.
681
00:30:35.250 --> 00:30:39.090
So again, highly heterogeneous effects
682
00:30:39.090 --> 00:30:43.053
in complex causal pathways that we need to understand.
683
00:30:44.070 --> 00:30:46.290
So let me again, as an aside,
684
00:30:46.290 --> 00:30:49.140
briefly present the PCORI method standards
685
00:30:49.140 --> 00:30:51.420
for complex health interventions.
686
00:30:51.420 --> 00:30:52.950
I actually won't talk about these,
687
00:30:52.950 --> 00:30:57.480
but there is both a PCORI methodology report
688
00:30:57.480 --> 00:30:59.010
that provides some supportive detail
689
00:30:59.010 --> 00:31:02.010
as well as an article that came out in JGIM
690
00:31:02.010 --> 00:31:03.000
several months ago
691
00:31:03.000 --> 00:31:06.150
that discusses each of these in more detail.
692
00:31:06.150 --> 00:31:07.860
But it's the issue of core functions
693
00:31:07.860 --> 00:31:11.670
that I wanted to talk about for a bit.
694
00:31:11.670 --> 00:31:13.230
And again, the underlying motivation
695
00:31:13.230 --> 00:31:16.860
is the fact that complex interventions can be adapted.
696
00:31:16.860 --> 00:31:20.100
They will be adaptive irrespective of our efforts
697
00:31:20.100 --> 00:31:21.510
to achieve fidelity,
698
00:31:21.510 --> 00:31:24.540
but more importantly they should be adapted.
699
00:31:24.540 --> 00:31:27.060
Now I'll often say adaptation happens.
700
00:31:27.060 --> 00:31:30.090
We should embrace it and study it and ultimately guide it.
701
00:31:30.090 --> 00:31:33.300
We should not be trying to ignore or suppress it.
702
00:31:33.300 --> 00:31:36.660
So the concepts of core functions and forms
703
00:31:36.660 --> 00:31:38.250
were introduced by Penelope Hall
704
00:31:38.250 --> 00:31:40.830
a good 15 years ago
705
00:31:40.830 --> 00:31:43.440
without a whole lot of attention
706
00:31:43.440 --> 00:31:46.350
and follow-up activity in the intervening years
707
00:31:46.350 --> 00:31:47.970
until relatively recently
708
00:31:47.970 --> 00:31:51.180
where researchers who study complex health interventions,
709
00:31:51.180 --> 00:31:52.860
implementation strategies,
710
00:31:52.860 --> 00:31:55.200
health promotion programs began to realize
711
00:31:55.200 --> 00:31:57.513
that they have a lot of relevance and value.
712
00:31:58.440 --> 00:32:03.030
And this is a short list of publications.
713
00:32:03.030 --> 00:32:04.470
There actually are many more
714
00:32:04.470 --> 00:32:06.630
just within the last year or two
715
00:32:06.630 --> 00:32:09.693
that have applied concepts of core functions and forms.
716
00:32:10.650 --> 00:32:15.650
So forms are the specific detailed activities.
717
00:32:15.780 --> 00:32:18.510
So if we think about physical activity as a broad category,
718
00:32:18.510 --> 00:32:19.680
walking, running, swimming
719
00:32:19.680 --> 00:32:23.040
are all examples of physical activity.
720
00:32:23.040 --> 00:32:27.360
And the argument is that our manualized intervention
721
00:32:27.360 --> 00:32:30.660
should not specify 20 minutes of walking,
722
00:32:30.660 --> 00:32:33.830
but instead should specify physical activity.
723
00:32:33.830 --> 00:32:35.700
In the case of patient education,
724
00:32:35.700 --> 00:32:37.710
the underlying core function again
725
00:32:37.710 --> 00:32:40.230
is to educate patients and their parents.
726
00:32:40.230 --> 00:32:43.830
The different forms we can use are listed here.
727
00:32:43.830 --> 00:32:45.690
And again, selecting a form
728
00:32:45.690 --> 00:32:50.130
that matches the particular features of the target audience
729
00:32:50.130 --> 00:32:52.560
is important in increasing fidelity.
730
00:32:52.560 --> 00:32:55.200
So we should not be providing a script,
731
00:32:55.200 --> 00:32:57.270
a strict script,
732
00:32:57.270 --> 00:33:00.780
but instead laying out the goals of the education
733
00:33:00.780 --> 00:33:03.240
and providing a menu of different strategies
734
00:33:03.240 --> 00:33:07.683
for achieving those goals through different kinds of forms.
735
00:33:09.360 --> 00:33:12.030
So I won't spend a lot of time on this,
736
00:33:12.030 --> 00:33:13.530
but encourage those of you interested
737
00:33:13.530 --> 00:33:15.180
to both look at the articles
738
00:33:15.180 --> 00:33:17.100
as well as these slides in more detail.
739
00:33:17.100 --> 00:33:20.910
But the general approach that we advocate
740
00:33:20.910 --> 00:33:25.590
is to think about, again, a set of core functions
741
00:33:25.590 --> 00:33:29.160
and think through all the different kinds of forms
742
00:33:29.160 --> 00:33:30.660
that might be available
743
00:33:30.660 --> 00:33:33.630
to operationalize those core functions.
744
00:33:33.630 --> 00:33:36.330
And then, also think about how to decide
745
00:33:36.330 --> 00:33:38.370
which form to select,
746
00:33:38.370 --> 00:33:40.080
and that's the purpose of research.
747
00:33:40.080 --> 00:33:41.910
In addition to identifying
748
00:33:41.910 --> 00:33:43.890
and describing the core functions,
749
00:33:43.890 --> 00:33:47.700
also to provide guidance for the local tailoring,
750
00:33:47.700 --> 00:33:51.723
which item from the menu is optimal for particular setting.
751
00:33:52.950 --> 00:33:55.050
Now we know that if we think only
752
00:33:55.050 --> 00:33:57.870
about core components or activities,
753
00:33:57.870 --> 00:34:01.560
we often can go down the route
754
00:34:01.560 --> 00:34:04.020
of modifying those forms or components
755
00:34:04.020 --> 00:34:06.450
in what appears to be a very minor way,
756
00:34:06.450 --> 00:34:09.300
but in fact completely eliminate
757
00:34:09.300 --> 00:34:11.070
achievement in one of the core functions.
758
00:34:11.070 --> 00:34:15.780
So if you think about drug detailing or academic detailing,
759
00:34:15.780 --> 00:34:18.450
that academic detailing interaction
760
00:34:18.450 --> 00:34:21.930
conveys information and education and knowledge,
761
00:34:21.930 --> 00:34:23.730
but also conveys professional norms.
762
00:34:23.730 --> 00:34:26.040
So that's an activity or a form
763
00:34:26.040 --> 00:34:30.210
that actually operationalizes two core functions.
764
00:34:30.210 --> 00:34:32.190
Audit and feedback is another example.
765
00:34:32.190 --> 00:34:34.950
Audit and feedback conveys information,
766
00:34:34.950 --> 00:34:37.350
but it also conveys professional norms
767
00:34:37.350 --> 00:34:39.330
and leadership expectations.
768
00:34:39.330 --> 00:34:43.170
And if you're focused only on the information function,
769
00:34:43.170 --> 00:34:44.610
you could easily decide,
770
00:34:44.610 --> 00:34:47.580
rather than convey the audit and feedback information
771
00:34:47.580 --> 00:34:50.070
in a departmental meeting,
772
00:34:50.070 --> 00:34:52.830
to convey that information via memo.
773
00:34:52.830 --> 00:34:56.970
If you do that, you weaken the professional norm function.
774
00:34:56.970 --> 00:34:59.460
One of the advantages of an audit and feedback session
775
00:34:59.460 --> 00:35:01.890
that's conducted in the departmental meeting
776
00:35:01.890 --> 00:35:03.570
is the physicians have an opportunity
777
00:35:03.570 --> 00:35:05.970
to talk about the guideline,
778
00:35:05.970 --> 00:35:08.040
the performance metrics or benchmarks
779
00:35:08.040 --> 00:35:09.963
and the variation of performance,
780
00:35:10.980 --> 00:35:13.140
and ideally help convince each other
781
00:35:13.140 --> 00:35:15.420
that maybe there is room for improvement.
782
00:35:15.420 --> 00:35:19.380
Whereas if you receive your own performance via a memo,
783
00:35:19.380 --> 00:35:23.010
you're not likely to be influenced in quite the same way.
784
00:35:23.010 --> 00:35:24.240
One final example,
785
00:35:24.240 --> 00:35:26.850
and that is quality improvement collaboratives
786
00:35:26.850 --> 00:35:30.510
where we know that having a multidisciplinary team
787
00:35:30.510 --> 00:35:32.790
and ensuring that the collaborative that's focusing,
788
00:35:32.790 --> 00:35:37.497
for example, on high contamination rates
789
00:35:40.350 --> 00:35:42.060
in the OR,
790
00:35:42.060 --> 00:35:45.420
that the QI team needs not only surgeons and nurses,
791
00:35:45.420 --> 00:35:47.530
but also members of the housekeeping staff
792
00:35:49.410 --> 00:35:52.200
because they collectively will develop
793
00:35:52.200 --> 00:35:55.080
a better understanding of the root causes
794
00:35:55.080 --> 00:35:57.930
of those infection rates of contamination
795
00:35:57.930 --> 00:35:59.880
than the physicians alone.
796
00:35:59.880 --> 00:36:04.200
But the other core function of that multidisciplinary team
797
00:36:04.200 --> 00:36:06.330
is the focus on acceptability
798
00:36:06.330 --> 00:36:08.520
of the findings and the recommendations.
799
00:36:08.520 --> 00:36:11.880
If a QI team consisting only of surgeons comes out
800
00:36:11.880 --> 00:36:13.560
and says that the high infection rates
801
00:36:13.560 --> 00:36:16.050
are due to the fact that the housekeeping staff
802
00:36:16.050 --> 00:36:18.360
are not wiping down the walls properly,
803
00:36:18.360 --> 00:36:20.280
you can be sure that the housekeeping staff
804
00:36:20.280 --> 00:36:23.310
are going to discount that
805
00:36:23.310 --> 00:36:25.350
because they know that they do their job properly
806
00:36:25.350 --> 00:36:27.450
and in their minds the problem
807
00:36:27.450 --> 00:36:29.430
is that the hand washing practices
808
00:36:29.430 --> 00:36:32.190
of the surgeons are deficient.
809
00:36:32.190 --> 00:36:35.160
So again, it's a single component
810
00:36:35.160 --> 00:36:37.320
or feature of intervention
811
00:36:37.320 --> 00:36:40.080
that operationalizes two different core functions.
812
00:36:40.080 --> 00:36:42.540
And understanding those core functions
813
00:36:42.540 --> 00:36:44.580
allows us to avoid making mistakes
814
00:36:44.580 --> 00:36:47.670
when we modify the intervention activity
815
00:36:47.670 --> 00:36:49.740
in a way that we think may be minor,
816
00:36:49.740 --> 00:36:54.180
but again can completely eliminate its ability
817
00:36:54.180 --> 00:36:58.500
to successfully carry out one of the core functions.
818
00:36:58.500 --> 00:37:00.540
So I think I've covered each of these already,
819
00:37:00.540 --> 00:37:02.370
but let me walk through them quickly.
820
00:37:02.370 --> 00:37:04.470
Again, the way that we typically think
821
00:37:04.470 --> 00:37:06.990
of a manualized intervention is highly detailed,
822
00:37:06.990 --> 00:37:11.990
is in fact more likely to do harm in many cases than value.
823
00:37:13.710 --> 00:37:16.650
Core components should be replaced by core functions.
824
00:37:16.650 --> 00:37:18.900
There are many implications of that rethinking,
825
00:37:18.900 --> 00:37:22.470
one of which of course is that a measurement of fidelity
826
00:37:22.470 --> 00:37:24.990
is not a measurement of whether you followed the script,
827
00:37:24.990 --> 00:37:28.020
but instead whether you successfully operationalized
828
00:37:28.020 --> 00:37:30.060
or carried out the core function.
829
00:37:30.060 --> 00:37:31.860
I've already talked about the fact
830
00:37:31.860 --> 00:37:34.590
that main effect estimates are not very helpful
831
00:37:34.590 --> 00:37:37.050
in evidence as we typically think of it.
832
00:37:37.050 --> 00:37:38.970
And again, it gets back to the point
833
00:37:38.970 --> 00:37:40.140
I've made a couple of times
834
00:37:40.140 --> 00:37:44.460
on the need to rethink the purpose of our research.
835
00:37:44.460 --> 00:37:46.900
So let me wrap up with just a few more slides
836
00:37:47.880 --> 00:37:51.150
that list some of the kinds of analytic approaches
837
00:37:51.150 --> 00:37:55.140
and research approaches that we need to be leveraging
838
00:37:55.140 --> 00:37:58.470
in order to, again, shine our flashlight
839
00:37:58.470 --> 00:38:01.440
on the processes and the mechanisms of effect
840
00:38:01.440 --> 00:38:03.101
rather than outcomes.
841
00:38:03.101 --> 00:38:06.300
These are some of the quantitative methods,
842
00:38:06.300 --> 00:38:10.080
qualitative comparative analysis is becoming more popular.
843
00:38:10.080 --> 00:38:11.010
There are also, of course,
844
00:38:11.010 --> 00:38:14.160
a number of qualitative methods as well.
845
00:38:14.160 --> 00:38:17.370
Process evaluation, theory-based evaluation
846
00:38:17.370 --> 00:38:19.830
and the continued emergence
847
00:38:19.830 --> 00:38:24.480
and illustrations of approaches to adaptation.
848
00:38:24.480 --> 00:38:26.280
Here are some examples of publications
849
00:38:26.280 --> 00:38:28.720
that are now quite dated that illustrate
850
00:38:29.670 --> 00:38:32.130
and talk about some of these approaches
851
00:38:32.130 --> 00:38:35.850
for measuring and taking into account context
852
00:38:35.850 --> 00:38:38.640
for examining moderator effects and mediator effects
853
00:38:38.640 --> 00:38:40.503
and mechanisms of effect.
854
00:38:41.970 --> 00:38:45.000
Here's some examples of implementation studies
855
00:38:45.000 --> 00:38:48.690
that have embraced and studied adaptation
856
00:38:48.690 --> 00:38:51.063
rather than suppressing it or ignoring it.
857
00:38:52.020 --> 00:38:55.110
Theory-based evaluation, realistic evaluation,
858
00:38:55.110 --> 00:38:57.450
again, are relatively new
859
00:38:57.450 --> 00:39:00.540
or underutilized approaches in the qualitative realm
860
00:39:00.540 --> 00:39:02.913
to study mechanisms of effect.
861
00:39:03.810 --> 00:39:05.610
There's still a lot of development work
862
00:39:05.610 --> 00:39:07.650
to be done, in my view, in these methods
863
00:39:07.650 --> 00:39:10.320
to get to the level of transparency
864
00:39:10.320 --> 00:39:13.443
and reproducibility that we need,
865
00:39:14.400 --> 00:39:17.250
but valuable approaches.
866
00:39:17.250 --> 00:39:18.600
And then, if we look outside
867
00:39:18.600 --> 00:39:21.960
the typical conventional toolkit
868
00:39:21.960 --> 00:39:23.250
to some of the other approaches
869
00:39:23.250 --> 00:39:26.250
such as statistical process control that are implemented,
870
00:39:26.250 --> 00:39:28.980
I'm sorry, our improvement science colleagues use
871
00:39:28.980 --> 00:39:30.510
as well as others,
872
00:39:30.510 --> 00:39:33.000
these represent other approaches.
873
00:39:33.000 --> 00:39:35.370
So just to wrap up,
874
00:39:35.370 --> 00:39:37.620
when we study complex health interventions,
875
00:39:37.620 --> 00:39:40.830
again, we need to begin by identifying the core functions
876
00:39:40.830 --> 00:39:42.870
and developing the menu of forms.
877
00:39:42.870 --> 00:39:46.890
Ideally, our research would validate
878
00:39:46.890 --> 00:39:48.180
our list of core functions
879
00:39:48.180 --> 00:39:49.710
or allow us to revise it
880
00:39:49.710 --> 00:39:51.420
so that we understand all the functions
881
00:39:51.420 --> 00:39:53.100
that need to be included
882
00:39:53.100 --> 00:39:56.673
and also provide evidence that guides the local tailoring.
883
00:39:58.500 --> 00:40:00.690
It would be documented
884
00:40:00.690 --> 00:40:03.900
in a set of adaptation or tailoring algorithms.
885
00:40:03.900 --> 00:40:07.110
And the bottom line, again, is a goal of understanding
886
00:40:07.110 --> 00:40:09.720
how complex interventions achieve their effects
887
00:40:09.720 --> 00:40:11.490
and how to modify them
888
00:40:11.490 --> 00:40:13.500
rather than pursuing the simpler question
889
00:40:13.500 --> 00:40:15.990
of whether they are effective.
890
00:40:15.990 --> 00:40:18.540
So I will stop there and open up
891
00:40:18.540 --> 00:40:22.590
for what I hope will be some robust discussion
892
00:40:22.590 --> 00:40:24.573
and comments and questions.
893
00:40:29.580 --> 00:40:30.753
Thanks, Brian.
894
00:40:31.770 --> 00:40:35.580
That was a really kind of interesting and important talk
895
00:40:35.580 --> 00:40:37.440
at sort of the cutting edge
896
00:40:37.440 --> 00:40:40.650
of where implementation science is today,
897
00:40:40.650 --> 00:40:43.470
and with a lot of information packed in
898
00:40:43.470 --> 00:40:46.170
and concepts and things like that
899
00:40:46.170 --> 00:40:47.310
for us to think about
900
00:40:47.310 --> 00:40:50.820
and potentially absorb into our own work.
901
00:40:50.820 --> 00:40:55.170
So maybe I might start with this first question
902
00:40:55.170 --> 00:40:56.120
which is...
903
00:40:57.030 --> 00:40:59.550
Actually, I'll even integrate something else
904
00:40:59.550 --> 00:41:01.380
that probably is worth both of us mentioning,
905
00:41:01.380 --> 00:41:04.710
which is that Brian is actually the co-founder and director
906
00:41:04.710 --> 00:41:06.630
of the Multilevel Training Institute
907
00:41:06.630 --> 00:41:10.800
that's offered every year in collaboration with NCI.
908
00:41:10.800 --> 00:41:12.990
And Raul Hernandez-Ramirez,
909
00:41:12.990 --> 00:41:15.510
who's one of our CMIPS primary faculty
910
00:41:15.510 --> 00:41:17.310
is a graduate of that institute.
911
00:41:17.310 --> 00:41:19.800
And I'm actually one of the instructors
912
00:41:19.800 --> 00:41:24.800
teaching about the analysis of multilevel interventions.
913
00:41:24.930 --> 00:41:27.150
But the question is,
914
00:41:27.150 --> 00:41:29.850
my thought and I think a lot of us out here think
915
00:41:29.850 --> 00:41:31.020
that the reason,
916
00:41:31.020 --> 00:41:33.540
sort of the opposite of what you said,
917
00:41:33.540 --> 00:41:36.300
that the reason we like multilevel interventions
918
00:41:36.300 --> 00:41:39.240
is because it seemed like the medical model
919
00:41:39.240 --> 00:41:41.190
of isolating one,
920
00:41:41.190 --> 00:41:43.950
like what we would've said before in the past component
921
00:41:43.950 --> 00:41:47.070
and maybe right now you might say function
922
00:41:47.070 --> 00:41:50.910
and studying it and holding everything else constant,
923
00:41:50.910 --> 00:41:53.430
these sorts of implementation studies
924
00:41:53.430 --> 00:41:55.020
have been disappointing.
925
00:41:55.020 --> 00:41:58.590
And so, the thought was that actually,
926
00:41:58.590 --> 00:42:01.020
first of all, it's totally unrealistic in real life.
927
00:42:01.020 --> 00:42:03.000
You don't just have one thing.
928
00:42:03.000 --> 00:42:05.970
All of these public health interventions are complex,
929
00:42:05.970 --> 00:42:08.220
whether we choose to study them or not.
930
00:42:08.220 --> 00:42:11.070
And so, the idea then evolved
931
00:42:11.070 --> 00:42:13.920
to that it might make sense
932
00:42:13.920 --> 00:42:16.320
to intervene on an entire,
933
00:42:16.320 --> 00:42:18.360
sometimes we might say package of components,
934
00:42:18.360 --> 00:42:20.520
which now you're kind of redefining
935
00:42:20.520 --> 00:42:22.563
as package of forms maybe.
936
00:42:24.480 --> 00:42:26.100
Or maybe it's package of functions
937
00:42:26.100 --> 00:42:29.250
and then the forms are the specific ways
938
00:42:29.250 --> 00:42:30.810
that the functions are ways
939
00:42:30.810 --> 00:42:32.760
that functions can be implemented.
940
00:42:32.760 --> 00:42:35.070
If I caught all of that very quickly, I think I did,
941
00:42:35.070 --> 00:42:36.300
and I think we're somewhat familiar
942
00:42:36.300 --> 00:42:38.910
with this idea in our center as well.
943
00:42:38.910 --> 00:42:41.460
So that would strengthen
944
00:42:41.460 --> 00:42:44.850
the ability to see an impactful intervention
945
00:42:44.850 --> 00:42:47.910
and allow us to translate
946
00:42:47.910 --> 00:42:52.910
into practice evidence-based interventions as a whole.
947
00:42:53.160 --> 00:42:54.300
But now you're saying
948
00:42:54.300 --> 00:42:57.060
that because of the adaptations
949
00:42:57.060 --> 00:42:59.250
and the variability and the heterogeneity,
950
00:42:59.250 --> 00:43:02.310
actually these kinds of approaches also
951
00:43:02.310 --> 00:43:03.690
are giving weak results.
952
00:43:03.690 --> 00:43:07.230
So I'm just wondering if you can comment on that.
953
00:43:07.230 --> 00:43:08.250
Sure.
954
00:43:08.250 --> 00:43:11.442
No, I would concur with everything that you've said
955
00:43:11.442 --> 00:43:13.833
and I think the main point is that,
956
00:43:14.670 --> 00:43:17.730
it's really that the argument that one size doesn't fit all.
957
00:43:17.730 --> 00:43:20.220
But to begin with, I do agree that,
958
00:43:20.220 --> 00:43:22.410
for most of these kinds of problems,
959
00:43:22.410 --> 00:43:24.840
the barriers are multi-component and multi-level,
960
00:43:24.840 --> 00:43:26.580
and we definitely need multi-level,
961
00:43:26.580 --> 00:43:29.970
multi-component complex health interventions.
962
00:43:29.970 --> 00:43:32.027
The simple example goes back to the studies
963
00:43:32.027 --> 00:43:36.050
in the 1970s and 1980s of CME
964
00:43:36.050 --> 00:43:38.610
as a method for improving physician practices.
965
00:43:38.610 --> 00:43:41.250
And the dominant finding from that body of research
966
00:43:41.250 --> 00:43:44.400
was physician knowledge and education changed
967
00:43:44.400 --> 00:43:46.590
and sometimes physician attitudes changed,
968
00:43:46.590 --> 00:43:48.630
but practices didn't change at all
969
00:43:48.630 --> 00:43:50.610
because the practices are held in place
970
00:43:50.610 --> 00:43:52.440
by multiple barriers.
971
00:43:52.440 --> 00:43:54.210
And if you don't provide the equipment
972
00:43:54.210 --> 00:43:56.100
or the staff support or the time
973
00:43:56.100 --> 00:43:59.491
or as I said, work on the patient resistance,
974
00:43:59.491 --> 00:44:02.040
no amount of educating physicians is likely
975
00:44:02.040 --> 00:44:05.040
to lead to the outcomes that we want.
976
00:44:05.040 --> 00:44:07.170
And in the causal diagram,
977
00:44:07.170 --> 00:44:09.480
the diag that I showed again is an example of that.
978
00:44:09.480 --> 00:44:12.480
If we focus on only one of those causal pathways,
979
00:44:12.480 --> 00:44:14.280
believe the other's untouched.
980
00:44:14.280 --> 00:44:19.020
So the point though is that we absolutely do need
981
00:44:19.020 --> 00:44:23.130
multi-component, often multi-level interventions.
982
00:44:23.130 --> 00:44:25.977
The issue though is the need to adapt and tailor them
983
00:44:25.977 --> 00:44:28.110
and the same mix of components
984
00:44:28.110 --> 00:44:30.750
or the same mix of forms and activities
985
00:44:30.750 --> 00:44:33.150
that is highly effective in one setting
986
00:44:33.150 --> 00:44:35.370
is not likely to be effective elsewhere.
987
00:44:35.370 --> 00:44:38.307
And when we take a complex health intervention
988
00:44:38.307 --> 00:44:41.100
and we try to scale it and spread it,
989
00:44:41.100 --> 00:44:45.150
or we move from efficacy research to effectiveness research,
990
00:44:45.150 --> 00:44:47.790
we are often disappointed in the findings.
991
00:44:47.790 --> 00:44:51.570
And that is because of the erroneous belief
992
00:44:51.570 --> 00:44:53.190
that one size fits all
993
00:44:53.190 --> 00:44:55.890
and that an so-called evidence-based practice
994
00:44:55.890 --> 00:44:57.990
is going to be evidence-based and robust
995
00:44:57.990 --> 00:45:00.450
and effective across multiple settings.
996
00:45:00.450 --> 00:45:01.890
It has to be tailored
997
00:45:01.890 --> 00:45:04.440
and we as researchers have to guide that tailoring.
998
00:45:06.690 --> 00:45:08.220
Thanks.
999
00:45:08.220 --> 00:45:09.390
I have lots of questions,
1000
00:45:09.390 --> 00:45:11.160
but I don't wanna hog the time.
1001
00:45:11.160 --> 00:45:13.503
So we have lots of people on here.
1002
00:45:14.340 --> 00:45:16.560
Do others have any questions they'd like to ask?
1003
00:45:16.560 --> 00:45:19.260
I think you can simply unmute yourself
1004
00:45:19.260 --> 00:45:20.673
and ask your question.
1005
00:45:26.250 --> 00:45:28.920
Sure, thank you for the wonderful talk.
1006
00:45:28.920 --> 00:45:30.450
I am an investigator working a lot
1007
00:45:30.450 --> 00:45:31.800
in low and middle income countries
1008
00:45:31.800 --> 00:45:33.810
and I was interested at the beginning of your talk
1009
00:45:33.810 --> 00:45:36.150
when you were using the term impact.
1010
00:45:36.150 --> 00:45:39.870
I think it's often used as a synonym for effectiveness.
1011
00:45:39.870 --> 00:45:41.760
But I think sometimes with public health
1012
00:45:41.760 --> 00:45:44.250
or even population health interventions,
1013
00:45:44.250 --> 00:45:46.680
we're thinking about numbers of people that can be served.
1014
00:45:46.680 --> 00:45:47.970
I think this is particularly relevant
1015
00:45:47.970 --> 00:45:49.620
if you think about communicable diseases
1016
00:45:49.620 --> 00:45:52.470
because there may be indirect benefits for addressing that.
1017
00:45:52.470 --> 00:45:55.890
And so, I guess that's sort of a very general question,
1018
00:45:55.890 --> 00:45:57.270
just curious if you've encountered that,
1019
00:45:57.270 --> 00:45:59.040
but maybe the more specific question
1020
00:45:59.040 --> 00:46:00.740
related to your research would be,
1021
00:46:02.978 --> 00:46:05.220
if volume then is kind of really important
1022
00:46:05.220 --> 00:46:06.630
about how we deliver interventions,
1023
00:46:06.630 --> 00:46:08.970
how do you think about that
1024
00:46:08.970 --> 00:46:11.426
with regard to understanding the fidelity?
1025
00:46:11.426 --> 00:46:14.100
Are we looking at sort of the contextual factors
1026
00:46:14.100 --> 00:46:17.197
related to how many people are served
1027
00:46:17.197 --> 00:46:19.080
maybe when we pilot an intervention,
1028
00:46:19.080 --> 00:46:21.930
but when we think about taking it to scale,
1029
00:46:21.930 --> 00:46:24.150
what are some of the considerations
1030
00:46:24.150 --> 00:46:26.190
about kind of understanding the impact of volume
1031
00:46:26.190 --> 00:46:28.680
on fidelity and adaptation?
1032
00:46:28.680 --> 00:46:29.871
Thank you so much.
1033
00:46:29.871 --> 00:46:32.687
Yeah, so first of all,
1034
00:46:32.687 --> 00:46:37.200
I'm a strong fan of the REAM framework
1035
00:46:37.200 --> 00:46:38.100
and I think to think
1036
00:46:38.100 --> 00:46:40.710
about the different dimensions of impact
1037
00:46:40.710 --> 00:46:43.800
and how they relate to one another is critically important,
1038
00:46:43.800 --> 00:46:45.570
that we focus only on effectiveness
1039
00:46:45.570 --> 00:46:47.550
and ignore the other issues.
1040
00:46:47.550 --> 00:46:49.560
The ultimate societal impact
1041
00:46:49.560 --> 00:46:52.860
that we are seeking will not be seen.
1042
00:46:52.860 --> 00:46:55.950
And I think the heterogeneity
1043
00:46:55.950 --> 00:46:58.988
may apply differently across different outcomes
1044
00:46:58.988 --> 00:47:00.930
in the kind of approach that we need
1045
00:47:00.930 --> 00:47:04.240
in order to engage a high volume
1046
00:47:04.240 --> 00:47:07.230
and a high proportion of the target audience
1047
00:47:07.230 --> 00:47:09.270
versus the approach that we need to use
1048
00:47:09.270 --> 00:47:12.030
to ensure that the intervention is effective
1049
00:47:12.030 --> 00:47:15.720
across a large proportion of the target audience
1050
00:47:15.720 --> 00:47:20.720
which does have its own heterogeneity in subgroups.
1051
00:47:20.760 --> 00:47:22.053
Those may be different.
1052
00:47:24.810 --> 00:47:26.760
And this may or may not be an answer,
1053
00:47:26.760 --> 00:47:29.910
but at least this is the way that I would think about it,
1054
00:47:29.910 --> 00:47:32.220
the vast majority if not all of these studies,
1055
00:47:32.220 --> 00:47:34.020
we need to begin with REAM
1056
00:47:34.020 --> 00:47:37.560
and explicitly think about all the different dimensions
1057
00:47:37.560 --> 00:47:42.210
that contribute to the overall impact and outcomes.
1058
00:47:42.210 --> 00:47:45.360
And then, we need to recognize and anticipate
1059
00:47:45.360 --> 00:47:48.870
and explicitly address the heterogeneity
1060
00:47:48.870 --> 00:47:51.540
across all of those different dimensions
1061
00:47:51.540 --> 00:47:55.260
and know that as we again scale up and spread
1062
00:47:55.260 --> 00:47:57.240
and adapt and tailor interventions
1063
00:47:57.240 --> 00:48:00.780
from one setting to another,
1064
00:48:00.780 --> 00:48:04.830
that tailoring and adaptations are likely to be needed
1065
00:48:04.830 --> 00:48:08.040
in different ways and different facets of the intervention
1066
00:48:08.040 --> 00:48:11.620
in order to ensure that we maximize
1067
00:48:12.960 --> 00:48:14.250
outcomes and success
1068
00:48:14.250 --> 00:48:16.950
across all the REAM dimensions.
1069
00:48:16.950 --> 00:48:19.530
So again, it's just a very different way
1070
00:48:19.530 --> 00:48:23.310
of thinking about research and interventions
1071
00:48:23.310 --> 00:48:26.040
compared to the typical evidence-based practice
1072
00:48:26.040 --> 00:48:27.480
that we develop an intervention,
1073
00:48:27.480 --> 00:48:29.490
we can describe it very simply
1074
00:48:29.490 --> 00:48:31.980
and we can deploy it anywhere
1075
00:48:31.980 --> 00:48:35.340
and we will see the same kinds of results.
1076
00:48:35.340 --> 00:48:36.300
And that's not the case,
1077
00:48:36.300 --> 00:48:39.180
both because we can't deploy the interventions
1078
00:48:39.180 --> 00:48:40.680
as we were designed elsewhere,
1079
00:48:40.680 --> 00:48:42.117
and that's especially true of course.
1080
00:48:42.117 --> 00:48:45.510
And we take US-designed interventions,
1081
00:48:45.510 --> 00:48:48.150
try to deploy them in low resource settings
1082
00:48:48.150 --> 00:48:50.100
within the US and elsewhere,
1083
00:48:50.100 --> 00:48:51.890
but even if we could deploy them
1084
00:48:51.890 --> 00:48:54.540
in the same way and implement them,
1085
00:48:54.540 --> 00:48:56.990
the effectiveness is likely to vary considerably.
1086
00:49:01.200 --> 00:49:03.600
Thanks so much, that's really interesting.
1087
00:49:03.600 --> 00:49:05.010
Okay, thank you.
1088
00:49:05.010 --> 00:49:05.843
That was a good answer.
1089
00:49:05.843 --> 00:49:08.040
Luke, do you have a follow-up question?
1090
00:49:08.040 --> 00:49:09.450
Yeah, I mean I would be curious
1091
00:49:09.450 --> 00:49:11.767
maybe just to think a little bit
1092
00:49:11.767 --> 00:49:14.160
about operationalizing some of these things.
1093
00:49:14.160 --> 00:49:17.730
Obviously, you work with one of the premier organizations
1094
00:49:17.730 --> 00:49:20.383
about thinking about how to answer these questions.
1095
00:49:20.383 --> 00:49:22.770
In terms of Kaiser,
1096
00:49:22.770 --> 00:49:26.070
very large health system, many different units,
1097
00:49:26.070 --> 00:49:27.390
however you'd wanna define those,
1098
00:49:27.390 --> 00:49:31.263
whether those are sites or providers and so on so forth.
1099
00:49:32.310 --> 00:49:34.800
I know I'm just curious
1100
00:49:34.800 --> 00:49:36.210
about how you think about integrating
1101
00:49:36.210 --> 00:49:38.010
quantitative and qualitative data
1102
00:49:38.010 --> 00:49:41.010
with respect to certain types of problems of this nature.
1103
00:49:41.010 --> 00:49:43.140
Maybe if there's any examples
1104
00:49:43.140 --> 00:49:46.290
that you might be able to share from your work in Kaiser.
1105
00:49:46.290 --> 00:49:49.350
So I think that integration is critical.
1106
00:49:49.350 --> 00:49:52.170
And this actually relates to the point that Donna raised
1107
00:49:52.170 --> 00:49:54.090
about implementation science and improvement science.
1108
00:49:54.090 --> 00:49:57.270
So the improvement science folks do accept
1109
00:49:57.270 --> 00:50:01.137
and anticipate and address the heterogeneity
1110
00:50:01.137 --> 00:50:05.100
and the whole issue of rapid cycle implementation
1111
00:50:05.100 --> 00:50:06.630
and improvement
1112
00:50:06.630 --> 00:50:09.000
where you do something and you sort of see
1113
00:50:09.000 --> 00:50:10.980
what the impacts are and then you refine it.
1114
00:50:10.980 --> 00:50:12.780
That's a form of tailoring.
1115
00:50:12.780 --> 00:50:15.870
So I think they do recognize the heterogeneity
1116
00:50:15.870 --> 00:50:17.730
and that's one way of dealing with it.
1117
00:50:17.730 --> 00:50:19.830
But I think that rapid cycle evaluation,
1118
00:50:19.830 --> 00:50:21.150
any kind of evaluation,
1119
00:50:21.150 --> 00:50:24.810
and understanding the mechanisms of effect.
1120
00:50:24.810 --> 00:50:28.560
We can only learn so much through mediation analysis
1121
00:50:28.560 --> 00:50:29.910
and other quantitative methods.
1122
00:50:29.910 --> 00:50:31.650
And if our goal is ultimately
1123
00:50:31.650 --> 00:50:34.080
to understand how the world works
1124
00:50:34.080 --> 00:50:36.480
and to understand causal pathways
1125
00:50:36.480 --> 00:50:37.983
and causal relationships,
1126
00:50:39.390 --> 00:50:42.450
we do have to mix the quantitative and qualitative.
1127
00:50:42.450 --> 00:50:44.940
And I think all of our projects at Kaiser
1128
00:50:44.940 --> 00:50:47.010
that are embedded research projects
1129
00:50:47.010 --> 00:50:48.570
that are a synthesis
1130
00:50:48.570 --> 00:50:51.660
of quality improvement activity and approaches
1131
00:50:51.660 --> 00:50:54.420
where we're trying to improve things in the near term
1132
00:50:54.420 --> 00:50:57.930
and implementation science and scientific approaches
1133
00:50:57.930 --> 00:51:00.569
where we're trying to generate scientific knowledge,
1134
00:51:00.569 --> 00:51:03.300
I think we almost invariably combine
1135
00:51:03.300 --> 00:51:05.310
quantitative and qualitative
1136
00:51:05.310 --> 00:51:07.290
as a way of, again, trying to understand
1137
00:51:07.290 --> 00:51:08.640
how the world works,
1138
00:51:08.640 --> 00:51:12.240
try to design the intervention, deploy it,
1139
00:51:12.240 --> 00:51:14.010
evaluate it early and often
1140
00:51:14.010 --> 00:51:16.680
in order to refine it and tailor it
1141
00:51:16.680 --> 00:51:17.850
and ultimately generate
1142
00:51:17.850 --> 00:51:20.040
the summative evaluation findings as well.
1143
00:51:20.040 --> 00:51:23.460
And I think we all need more guidance
1144
00:51:23.460 --> 00:51:26.130
and more examples of how this is done
1145
00:51:26.130 --> 00:51:27.750
because there are a lot of moving parts
1146
00:51:27.750 --> 00:51:30.030
and a lot of different factors to think of,
1147
00:51:30.030 --> 00:51:31.980
not only the REAM multiple dimensions,
1148
00:51:31.980 --> 00:51:34.470
but the different kinds of data
1149
00:51:34.470 --> 00:51:36.540
and the different ways of understanding
1150
00:51:36.540 --> 00:51:38.670
and tracking the mechanisms of effect
1151
00:51:38.670 --> 00:51:41.340
and the intermediate or proximal outcomes
1152
00:51:41.340 --> 00:51:43.500
in addition to the distal outcomes.
1153
00:51:43.500 --> 00:51:44.970
So lots of challenges,
1154
00:51:44.970 --> 00:51:47.913
but lots of opportunity for innovation and creativity.
1155
00:51:51.750 --> 00:51:54.453
Anyone else that wants to ask you questions?
1156
00:51:58.300 --> 00:52:01.140
While we're waiting to see, I have another question.
1157
00:52:01.140 --> 00:52:03.690
So you're probably familiar, Brian,
1158
00:52:03.690 --> 00:52:06.990
with Linda Collins' MOST approach
1159
00:52:06.990 --> 00:52:11.970
to developing and assessing or testing interventions
1160
00:52:11.970 --> 00:52:16.970
and her focus is also with complex multilevel interventions
1161
00:52:17.940 --> 00:52:19.530
where there are the three phases.
1162
00:52:19.530 --> 00:52:21.930
But in the third phase,
1163
00:52:21.930 --> 00:52:25.113
the third phase is kind of a traditional,
1164
00:52:26.820 --> 00:52:28.140
fixed.
1165
00:52:28.140 --> 00:52:29.310
You call it manualized,
1166
00:52:29.310 --> 00:52:31.350
it doesn't necessarily have to just be a manual,
1167
00:52:31.350 --> 00:52:32.520
it could be other things,
1168
00:52:32.520 --> 00:52:37.520
but fixed set of components at certain levels
1169
00:52:37.620 --> 00:52:41.700
and that's kind of tested in a standard RCT-type approach.
1170
00:52:41.700 --> 00:52:43.800
And I'm wondering if you think
1171
00:52:43.800 --> 00:52:48.690
that the MOST design is useful in certain settings
1172
00:52:48.690 --> 00:52:52.680
or do you think that maybe that kind of approach
1173
00:52:52.680 --> 00:52:54.630
has kind of seen its better days
1174
00:52:54.630 --> 00:52:58.860
because of the fact that it doesn't take into account
1175
00:52:58.860 --> 00:53:02.520
sort of the contextual aspects and the need for adaptation.
1176
00:53:02.520 --> 00:53:04.080
But I know among other circles,
1177
00:53:04.080 --> 00:53:06.690
the MOST design is very popular
1178
00:53:06.690 --> 00:53:10.290
and even we've had training in our center in MOST
1179
00:53:10.290 --> 00:53:12.600
and I'm currently discussing
1180
00:53:12.600 --> 00:53:15.810
a possible grant application with some investigators here
1181
00:53:15.810 --> 00:53:16.920
who'd like to use MOST.
1182
00:53:16.920 --> 00:53:20.340
So I'm just wondering what your thinking is about that.
1183
00:53:20.340 --> 00:53:23.830
Yeah, so I think that approach I think is highly valuable.
1184
00:53:23.830 --> 00:53:26.430
As our standard RCTs,
1185
00:53:26.430 --> 00:53:29.940
as long as we recognize that they need to be augmented
1186
00:53:29.940 --> 00:53:33.510
with additional kinds of data collection analysis activities
1187
00:53:33.510 --> 00:53:36.660
that try to understand the mechanisms of effect.
1188
00:53:36.660 --> 00:53:38.860
But certainly and when I say
1189
00:53:40.070 --> 00:53:43.770
that RCTs and a focus on impact and outcomes
1190
00:53:43.770 --> 00:53:47.850
are not what we should be doing,
1191
00:53:47.850 --> 00:53:49.410
that's probably too extreme.
1192
00:53:49.410 --> 00:53:52.020
I think we obviously need to measure outcomes
1193
00:53:52.020 --> 00:53:55.020
and we need to use traditional quantitative methods,
1194
00:53:55.020 --> 00:53:56.640
but we need to augment them
1195
00:53:56.640 --> 00:53:59.160
and have an equal, in some cases greater focus,
1196
00:53:59.160 --> 00:54:01.260
on the mechanisms of effect.
1197
00:54:01.260 --> 00:54:02.610
And certainly, to try to get a handle
1198
00:54:02.610 --> 00:54:04.440
on some of the heterogeneity
1199
00:54:04.440 --> 00:54:06.330
and some of the factors using MOST
1200
00:54:06.330 --> 00:54:08.790
and other approaches that Linda and others
1201
00:54:08.790 --> 00:54:10.830
have developed and advocated
1202
00:54:10.830 --> 00:54:13.650
I think is quite important and valuable.
1203
00:54:13.650 --> 00:54:15.090
I just think that they're incomplete
1204
00:54:15.090 --> 00:54:16.050
and we need to make sure
1205
00:54:16.050 --> 00:54:19.680
we, again, have the accompanying process evaluation
1206
00:54:19.680 --> 00:54:21.753
and mediation analysis and others.
1207
00:54:23.010 --> 00:54:23.973
That makes sense.
1208
00:54:26.220 --> 00:54:27.183
Anyone else?
1209
00:54:28.338 --> 00:54:30.663
Okay, so we have a question in the chat.
1210
00:54:33.180 --> 00:54:34.710
What are some other frameworks
1211
00:54:34.710 --> 00:54:36.930
you would consider reviewing
1212
00:54:36.930 --> 00:54:39.720
when considering a multi-component and multi-level study?
1213
00:54:39.720 --> 00:54:44.580
So presumably this question is other than REAM.
1214
00:54:44.580 --> 00:54:45.570
Sure.
I'm guessing
1215
00:54:45.570 --> 00:54:48.180
and if the person who asked it would like to elaborate
1216
00:54:48.180 --> 00:54:50.250
or if this is totally clear to you,
1217
00:54:50.250 --> 00:54:51.153
Brian.
Yeah,
1218
00:54:51.153 --> 00:54:52.530
let me give it a quick shot
1219
00:54:52.530 --> 00:54:54.990
and then you can elaborate if I'm missing a point.
1220
00:54:54.990 --> 00:54:57.630
But I think that CFIR of course,
1221
00:54:57.630 --> 00:55:00.520
which continues to be the go-to high level framework
1222
00:55:01.650 --> 00:55:03.090
is a way of identifying
1223
00:55:03.090 --> 00:55:05.370
all the different categories of factors.
1224
00:55:05.370 --> 00:55:08.400
So multi-component, multi-level interventions
1225
00:55:08.400 --> 00:55:13.050
address multiple sources of barriers and factors
1226
00:55:13.050 --> 00:55:15.797
that influence the outcomes that we're interested in.
1227
00:55:15.797 --> 00:55:18.990
And CFIR is, in my mind, the best organizing framework
1228
00:55:18.990 --> 00:55:22.080
that gives us that sort of 50,000 foot level.
1229
00:55:22.080 --> 00:55:23.490
Then when we've identified
1230
00:55:23.490 --> 00:55:26.790
the different categories of factors,
1231
00:55:26.790 --> 00:55:29.730
we may need to bring in accompanying frameworks.
1232
00:55:29.730 --> 00:55:31.410
So, behavior change wheel,
1233
00:55:31.410 --> 00:55:36.410
theoretical domains framework is quite useful in identifying
1234
00:55:36.720 --> 00:55:40.290
some of the physician-level behavioral factors.
1235
00:55:40.290 --> 00:55:43.710
So I think it does depend on what CFIR tells us.
1236
00:55:43.710 --> 00:55:45.630
If many of the barriers and influences
1237
00:55:45.630 --> 00:55:48.120
are regulatory or community,
1238
00:55:48.120 --> 00:55:50.730
then we may need to bring in political science frameworks
1239
00:55:50.730 --> 00:55:52.770
or other bodies of theory.
1240
00:55:52.770 --> 00:55:56.220
But we start with CFIR to get sort of the lay of land
1241
00:55:56.220 --> 00:56:00.090
and then we identify frameworks for subsets of factors.
1242
00:56:00.090 --> 00:56:02.250
And to me, REAM and I think to most of us
1243
00:56:02.250 --> 00:56:04.200
is more of an evaluation framework.
1244
00:56:04.200 --> 00:56:07.560
It doesn't really give us the theory,
1245
00:56:07.560 --> 00:56:08.880
but it directs our attention
1246
00:56:08.880 --> 00:56:11.310
to the different categories of outcomes
1247
00:56:11.310 --> 00:56:13.560
that we need to take into account
1248
00:56:13.560 --> 00:56:16.263
and measure and attempt to improve.
1249
00:56:18.090 --> 00:56:20.700
Great, that was a very clear answer.
1250
00:56:20.700 --> 00:56:21.533
Thank you so much.
1251
00:56:21.533 --> 00:56:24.960
And we're at time, so I think we'll just thank our speaker.
1252
00:56:24.960 --> 00:56:26.760
Hopefully we'll see him again in person
1253
00:56:26.760 --> 00:56:28.920
sometime in the near future
1254
00:56:28.920 --> 00:56:32.760
and we look forward to some of our one-on-one meetings today
1255
00:56:32.760 --> 00:56:34.710
and in a few of the subsequent days.
1256
00:56:34.710 --> 00:56:37.620
So thank you so much, Dr. Mittman.
1257
00:56:37.620 --> 00:56:39.490
Okay, thank you all.
Bye everybody.
1258
00:56:39.490 --> 00:56:40.533
Okay, bye-bye.