Use PEDALs Model to Pedal for Implementation Research

February 12, 2024

Information

The presentation introduced PEDALs – a simple model to guide implementation research. According to PEDALs, implementation research starts by identifying a “Problem” in your clinical or public health work, which then leads to the search for an “Evidence-based practice” (EBP) to address that “Problem.”

Speaker: Dong (Roman) Xu, PhD, MPP

Monday, February 14, 2022

ID11301

To CiteDCA Citation Guide

00:03<v Maur>Okay, great.</v>
00:05So good morning and good evening to our friends
00:08joining from China.
00:08Welcome all, my name is Maur Desai.
00:11I'm a faculty member in the Department
00:14of Chronic Disease Epidemiology at the Yale School
00:15of Public Health and also the school's Associate Dean
00:18for Diversity, Equity and Inclusion.
00:20It is my pleasure to step in for Donna Spigelman
00:23to introduce our speaker, professor Dong Roman Xu.
00:27But first I'd like to acknowledge that this seminar
00:30is co-sponsored by the Yale Center for Methods
00:32and Implementation and Prevention Science,
00:35also known as CMIPS, which Professor Spigelman directs.
00:37The seminar is co-sponsored by the Department
00:40of Chronic Disease Epidemiology and the global health
00:42concentration at YSPH, as well as the newly funded
00:46NIHT 32 training grant Implementation Science Research
00:51and Methods.
00:53Based at the Yale School of Public Health, CMIPS develops
00:56and disseminates innovative methodological approaches
00:59to address implementation gaps and improve public health
01:02worldwide strategically selecting the issues
01:05that carry the greatest burden and hold the greatest promise
01:08for amelioration right now.
01:10If you would like to be informed about future CMIP seminars,
01:14please let William Tutel know in the chat and he will add
01:17you to the CMIPS listserv.
01:20Professor Roman Xu is one of the foremost implementation
01:25science researchers and methodologists in China.
01:26His research focuses on health system innovations
01:29and implementation science, particularly those
01:32involving chronic diseases quality of primary health care
01:35and eHealth in the context of global health.
01:38He's leading several large studies including primary
01:42healthcare quality cohort in China, the Silk Road Labs
01:46for health system strengthening in Nepal and Mozambique,
01:50an implementation trial for stroke guidelines in China
01:54and the shared medical appointment trial for diabetes
01:57or smart trial.
01:59Professor Xu received his PhD in global health
02:01implementation Science from the University of Washington
02:04and his master's in Public Policy from Harvard University.
02:09The title of his talk today is Use PEDALs Model to PEDAL
02:12for Implementation Research.
02:15Roman, over to you.
02:18<v Roman>Thank you Maur.</v>
02:19Thank you very much for the very nice introduction,
02:21let me share my screen first.
02:32Okay, I suppose you can now see my screen.
02:35Today I'm going to talk about the PEDALs model
02:39and in that process I will use one of our ongoing,
02:42<v Maur>Oh, sorry, Roman.</v>
02:44Roman, do you want to put it in presentation mode?
02:47<v Roman>It is not in presentation mode now?</v>
02:49On my screen it is in the presentation mode,
02:54it's a little strange.
02:56<v Maur>Ahh.</v>
02:57<v Roman>Let me share again.</v>
02:58<v Maur>Okay and while you're doing that,</v>
03:00I'll just say very quickly, if you have questions,
03:03please hold them to the end, but you're welcome to put them
03:05in the chat and then when we get to the Q&A,
03:09you can use the raise hand feature, just unmute yourself,
03:13jump right in and we'll make sure that we get to
03:16as many questions as possible.
03:22<v Roman>Can see now.</v>
03:27<v Maur>I don't know about others, but I'm still seeing it</v>
03:28in sort of the regular mode, not the presentation mode.
03:32<v Roman>Okay, just one second, sorry for that,</v>
03:35it's a little I tried but just one second.
03:53Sorry for that everybody.
03:57<v Maur>No problem.</v>
03:59<v Roman>Let me share my screen to see whether</v>
04:02this will be better.
04:13Now, can you see my screen now?
04:15<v Maur>Yes and you may want to, we're seeing</v>
04:18also the preview slide, so if you swap display
04:20maybe that would help, that would help at the top.
04:26<v Roman>Does it work now?</v>
04:32<v Maur>And we can start, if it's gonna be a,</v>
04:34you don't wanna create too much delay,
04:36but if at the top, I think it's the second tab,
04:38if you say switch display, we should be able
04:41to then see it in full mode.
04:44So it was fine, but we were just seeing
04:49your previous slide as well.
05:20There we go, perfect.
05:33<v Donna>Oh, Dr Xu, you on mute?</v>
05:35<v Maur>Oh, you're on mute.</v>
05:43<v Roman>Okay, now it's good.</v>
05:44<v Maur>Perfect.</v>
05:46<v Roman>Ah, technology, I am supposed to know this well.</v>
05:49So today I'm going to talk about the PEDALs model,
05:53which is a model we have developed essentially at the
05:56beginning for our students so they can understand quickly
05:59with a nice acronym about the general procedures
06:02of conducting implementation research.
06:06And we'll use one of our ongoing trial for shared
06:10medical appointment for the management of diabetes
06:13as an illustration.
06:15And in that process I will talk about some
06:18of the common designs for implementation research
06:21and the choices and the rationale we choose some
06:26of the designs versus others in this presentation.
06:31But before that, let me spend a few minutes on some
06:34of the advertisement.
06:36I'm currently the principal investigator for a lab
06:39we call Acacia lab, which is sort of the child of
06:45a parent study called Acacia Study.
06:49In that study we have set up a consortium
06:52of researchers from 10 universities in China.
06:56And gradually because of using that study
06:59we have formed strong collaborative team in China
07:02for primary healthcare and implementation science.
07:05So looking to the future, we really want to use
07:08this platform to collaborate more with all of you.
07:12I'm also leading the Southern Medical University Institute
07:16for Global Health site.
07:18Southern Medical University is one of the first
07:22and largest medical center receiving international patients,
07:26especially from the low and middle income countries.
07:29And in terms of research, our institute holds largest
07:33total competitive grant size in China.
07:36And in terms of education, we are one of the four
07:38and one of the earliest program for international MPH,
07:43every year we gave 25 scholarship to people from low
07:47and middle income countries to study MPH in China.
07:52I'm also the co-editor in chief along with the professor
07:56Ann Sales for a new journal called implementation science
08:00communications, which is a facial companion journal
08:04to implementation science.
08:07We are a brand new journal two years, but so far
08:11we have received last year we have received almost
08:16a 400 submission, which is quite good for new journal
08:20and the downloads are also increased dramatically from
08:25two years ago to last year.
08:27So yeah, welcome to submit your work to our journal
08:32and thank you so much.
08:34So now before I talk about the PEDALs model, I'd like
08:37to go over a few key concepts in implementation science.
08:41That is very much related to my model as well.
08:45So what is implementation science?
08:49This is a the question that I normally get in China
08:54and sometimes I find it's not so easy as it appears
08:58to discuss.
09:00According to a review in last HIV journal,
09:04they have identified 73 unique definitions
09:09for implementation science.
09:12So this definition I like it quite a bit
09:16but I still feel it is a little long, I prefer a much
09:20shorter definition modified on the definition given
09:24by the authors of this review.
09:26In my view, implementation science is a multidisciplinary
09:29specialty to seek generalized ology.
09:32So implementation science, one of the questions
09:35I got in China is it a field?
09:38Is it a discipline?
09:39Is it just a method?
09:41I would say it's a multidisciplinary specialty
09:45to seek generalizable knowledge, because it is a science
09:49so it has to generate generalizable knowledge,
09:52it is about the scale of reasons for the strategies
09:56to close the evidence to practice gap.
09:59So what implementation science is about is really to
10:03put evidence-based practice into routine practice
10:08and our research is trying to understand how large
10:12that gap is and what are the determinants barriers
10:14and the facilitators for implementing that evidence-based
10:19practice and making it to a routine practice
10:21and what are the implementation strategies which are also
10:24intervention but we distinguish that from
10:27the health intervention, so we call it
10:29implementation strategies.
10:30What are the implementation strategies which can help
10:33close this gap?
10:34So that is about implementation science,
10:37so some of the other key concepts.
10:40In order to do implementation research,
10:42we have to start with health interventions
10:45and those healthy, not all of the health
10:48interventions can go into the process
10:49of implementation research.
10:51We have to first have evidence this health intervention
10:57can be regarded as evidence-based practice EBP.
11:00Once we have that, we need to understand the determinants
11:03of implementation, then based on that we will develop
11:06implementation strategies.
11:08We which we will tackle the barriers and the facilitators
11:12for implementing the EBP, some of them may come
11:16in from the health intervention itself,
11:18some are contextual factors.
11:20Then we need to understand the implementation
11:22outcomes which are different from the health outcomes
11:25and the clinical outcomes, then after that
11:28we make it into routine practice.
11:31Now let's talk the PEDALs models we have developed.
11:36First of all why we developed this model.
11:39Implementation science has already been inundated
11:42by the theories, models and the framework.
11:45A 2012 review identified more than 100 frameworks
11:50and it has ever since been increasing, why we are
11:54getting another framework.
11:55The motivation is from our students because when we are
11:59teaching implementation science to a master
12:02and an undergraduate students, very often we are challenged
12:05to give them a quick way to understand the essence
12:08of conducting implementation research.
12:11PEDALs has been developed as a teaching tool
12:14to wrap essential steps of conducting implementation
12:17research in an easy to remember acronym
12:19and also this acronym supposedly need to have an appropriate
12:23metaphor which can encompass
12:26the implementation science essence.
12:30This model has not yet been used, has not yet been published
12:34or peer reviewed, but we have been using it
12:37for our teaching already, so what is the PEDALs?
12:39PEDALs has some of the letters and but first off
12:42for the PEDALs it's when your PEDAL, your bike,
12:46which has a image of a cycling forward with PEDALs
12:49which has a metaphor of moving forward with the
12:53short cycles, which indicates implementation science
12:56is very often indicating continued improvement.
12:58This is not a graphical representation of a model.
13:04So for the PEDALs for implementation science,
13:07we have to start with your problems in your work,
13:11which can be a clinical problem, which can be
13:14a public health problem.
13:15Once you have identified this problem, you need to search
13:18for whether or not there are existing evidence-based
13:22practice EBP, which can address the problem you have
13:25encountered after you have identified an EBP.
13:28To address your problem we have to think about how to,
13:32what are the barriers and the determinants to implementing
13:35this EBP in your work setting.
13:38So after the good understanding of the determinants
13:42to the implementation of the EVP, we need to develop
13:47implementation strategies which can facilitate
13:50the adoption and uptake of this EVP, then finally
13:54we hope it can get into sustained use.
13:57All across this process there is a S for the PEDALs,
14:01a small s some has two meanings here
14:05for one way it is plural form so means this might be
14:09a cycle, a continuous improvement.
14:13Another meaning of small s is it is a scale
14:17so it is indicating we need to have monitoring
14:22and evaluation designs and the methods for
14:25particularly in the last two process of EBP
14:30developing of the determined determinants of EBP
14:35and also developing implementation strategy
14:38and also to test the effectiveness
14:41of the implementation strategy.
14:42All of that need to have a strong monitoring
14:45and an evaluation design.
14:47So that's use our shared medical appointment study
14:51to illustrate those process first work challenges
14:56and problems starting with P.
14:59In China, China is facing with a severe diabetic epidemic.
15:04In 2000 we only have 22 meaning people with diabetes.
15:08But in 2019 it is already 116 meaning the prevalence
15:14has increased dramatically from 2.7% to almost 10%.
15:21And the way to deal with the diabetic management in China
15:25is primary is through primary healthcare.
15:27However, we have a challenging here in China
15:30with a very much overburdened public health
15:32and clinical workforce and there is also reported
15:35very poor quality of care and insufficient communication
15:39between care providers and the patients,
15:41there is also a very much lack of patient centered care.
15:45So the service model is insufficient to really
15:50integrate public health work and also the curative services
15:54in diabetic management here in China.
15:58So we performed this gap analysis and we trying to
16:01identify whether there is other service model which has
16:06evidence which can meet and solve the problem we have
16:09encountered in this work setting.
16:13So that's come to the second step
16:15evidence-based practice EBP.
16:20So in order to do that, first we need to have
16:22some understanding to what extent health intervention
16:28can be considered EBP, I don't think we have some consensus
16:32on that, but most of the people I believe
16:35are familiar with this evidence pyramids evidence can
16:39change from expert opinions to cross-sectional studies
16:43all the way up to individual studies to synthesis of know
16:47to systematic reviews on the top.
16:50So normally for the journal implementation science
16:54and the implementation science communications,
16:56we will regard some health intervention as a EBP
17:02if they at least have several randomized control
17:06studies to support them in a health setting.
17:08But I have to say different settings, different studies,
17:12like some of the policy study, it's very difficult
17:15to have randomized control studies.
17:17But normally in the healthcare vicinities in this setting we
17:22consider several randomized control studies or even better,
17:25more since that systematic reviews is good to prove it
17:29is an EBP.
17:31So with that standard, fortunately we have identified
17:34systematic review for shared medical appointment
17:37which has approved SMA is a good way to tackle
17:41diabetic management in countries outside of china,
17:46so what is the shared medical appointment?
17:49Shared medical appointment is a new method
17:51of managing your patients.
17:53In traditionally in China, patients are managed
17:55under one-on-one consultation.
17:59So one patient go to see the doctor one-on-one,
18:02but for shared medical appointment patients
18:05with similar conditions are grouped together
18:08and they go and see doctor together and sometimes also
18:12it's not only one doctor, it's a group of
18:15a multidisciplinary team come together.
18:16So it becomes a group and group consultation between
18:20and the education and the management
18:23between the clinicians and the patient.
18:25And there are systematic reviews suggest shared
18:28and medical appointment has substantially improved clinical
18:31outcomes in terms of glucose control
18:36and blood pressure control and also it improves,
18:39it has a lot of benefits to improve patient behavior,
18:43self-management and also to improve (indistinct) adherence
18:47to best practice and adherence to clinical guidelines.
18:52And the very good is the study also suggests shared medical
18:56appointment for diabetic does not really increase
19:01the economics.
19:03So you don't really spend extra money,
19:06have extra spenditures to manage your patients
19:10with a shared medical appointment model.
19:12So it looks like it is effective and it does not
19:16increase your expenditure.
19:18It is no brainer we should use SMA but unfortunately
19:21in China, China we don't really use it a lot
19:24actually it is not used at all in all across China.
19:29So that's why we need to go to the third step.
19:32We need to understand what are the determinants various
19:36and facilitators which are determine the implementation
19:39of SMA shared medical appointment in China in our setting,
19:44in our primary care.
19:46So there are two types of barriers in the facilitators
19:48in my view.
19:49One is embedded with the shared medical appointment itself.
19:53So the health intervention itself can be a strong barrier
19:57then it moves to contextual factors.
20:00So first let's talk about shared medical appointment itself
20:04and what we need to do is we need to optimize SMA
20:07for the context of China.
20:11In that we're going to use the method proposed by
20:14Professor Linda Conius.
20:16Linda Conius is proposing a method she has developed a code
20:21optimizing of bio behavior and biomedical interventions,
20:26the multi-phase optimization strategy coll it MOST.
20:31So according to MOST, we have to first come up with a
20:35conceptual model for SMA for the setting
20:40of China because almost the all of the health interventions
20:44are sort of the complex intervention for shared
20:48medical appointment it's the same thing,
20:50it has many components.
20:52We need to decide what individual components for SMA
20:56can be combined together and it can be configured in a way
21:00which is best suited for the setting of China.
21:03So we have developed this conceptual model,
21:06at least it has four components which are important for SMA,
21:10one is you put patients with similar conditions together
21:13so they go to see the doctor together.
21:16So this component can be versus individual
21:19one-on-one session.
21:20The second component is the patient can go accompanied
21:24by their family members.
21:26The third component it is a multidisciplinary team
21:30from the clinicians come to see the patient.
21:34The fourth is the education for diabetes can
21:37be either online or offline, so we have at least
21:40four components.
21:42So in this conceptual model, those campaign,
21:44those components works through one of those nine mechanisms,
21:48through those two types of mediators finally it can
21:52improve self-management of the patients in theory
21:56and finally to improve primary outcome of glucose control.
22:01So for the conceptual model it is good in a view,
22:05you have your components layout clearly and also have
22:09your mediators layout clearly.
22:11But mostly important, you have all of the mechanisms
22:15which are supposed to work to connecting each
22:17of the individual component to the mediators
22:20and in the end to the outcome health outcome
22:24and the clinical outcome you are trying to achieve.
22:26So with the development of this SMA conception model,
22:30we can develop with the information from literature
22:33and also with consultation with clinicians
22:36and the stakeholders, we need to really have optimization
22:39trial to really understand whether or not
22:43those individual components can work and whether
22:46you combine those individual components together,
22:49they can be working together rather than canceling
22:52each other out.
22:53So in order to do that we are going to do a factorial design
22:58while we are going to do a factorial design,
23:00because we have four components.
23:02Let's say if we only have three components
23:05with the three components you can have eight different
23:08configuration of those components and making them into
23:12an complex intervention.
23:14If we are going to do in the individual two arm trial,
23:18we need to do three separate randomized control study
23:22that is very time consuming and a very resource consuming.
23:27But with the factorial design as proposed by
23:30Professor Linda Conius, we can use one trial the same
23:34sample size, but we can deal with all three or four
23:38components, so with that study we can understand whether
23:41or not each individual component in your complex study
23:45is effective or not.
23:47And even better with the design of a factorial design,
23:52they can also test interaction between those
23:55individual components.
23:56So that is important for implementation science
23:59because sometimes individual component can work,
24:03but if you put different individual components together,
24:07they may cancel out the effects from each other
24:08or they can virtually reinforce, so one plus one may be
24:13larger than two or less than two, so factorial design
24:16can deal with all those issues, so that is very good.
24:19And also good to remember is most it's not only
24:24concerning factorial design because it is optimization,
24:28optimization means not the best if it is not optimization.
24:33We are only looking for health interventions which works
24:37best in terms of improving health outcomes,
24:39but with optimization we are looking for under the resource
24:43constraints which we have agreed upon with the stakeholders
24:47what is working best.
24:49So we have to set clear, something we call optimization
24:54criteria, which can be money, which can be time
24:58to implementing SMA.
25:00So once you we have those criteria set, we can do the
25:04factorial design we and the way we are going to pick up
25:06the configuration which best suited the resource
25:10concentration constraints.
25:12But under that umbrella, whatever configuration works best
25:16can be picked up for our final traditional randomized
25:21control study.
25:22So that is what we are going to do to select the components
25:27for our SMA in that way we can reconfigure SMA
25:31to the context of China.
25:33So suppose after we have optimized SMA itself
25:38for the context of China, we still need to understand
25:41this reconfigured and optimized SMA and what are the
25:47other contextual factors which can determine
25:50the implementation of this optimized SMA.
25:55In order to do that we will use a lot of the frameworks.
26:00You know, implementation science is inundated by frameworks,
26:04but use of frameworks is really the essence
26:07of implementation research.
26:08I very much use this when I'm talking about
26:11implementation science theories, models and a framework
26:15without a theory.
26:17Think about if you have many pieces of clothing, shoes,
26:20juries and you do not have really a very nichey
26:24and very neat, very nice walking closet,
26:27then you are buried in your clothing and if you want to
26:31walk up and do a interview, it's very difficult for you
26:34to organize your clothing and dress up very nicely.
26:37But if you have a very good framework, which is almost like
26:41your walkin closet, you can organize the things
26:44systematically and you can also standing on the shooters
26:47of many giants because other people have done
26:49the work of for you what are the items that you need
26:52to looking at.
26:54So frameworks provide us with more systematic
26:57and comprehensive way of looking at it, the things you want
27:00to look at.
27:03And we have very good taxonomy of series models
27:08and the frameworks and for the PEDALs we are going
27:11to use a determinants frameworks for the understanding
27:15of barriers and the facilitators and we will use
27:18process models and implementation series to understand
27:22how to develop implementation strategy.
27:24Then we are going to use evaluation frameworks
27:27for the evaluation of your implementation strategy
27:31for health outcomes.
27:35So for specifically for SMA, we are going to use
27:37a concern related framework for implementation research.
27:40Why we use this very commonly used framework,
27:43CFIR is probably most widely used implementation framework.
27:48Why we choose it, one of the biggest reason
27:51is it is very comprehensive and the second is
27:54it has a really nice website which has layout all of the
27:58tools and options and the literature concerning
28:01this framework and also the tutorials available,
28:05so it's very easy to find resources.
28:07So once we have a implementation team and the research team
28:11if we use safer is much easier to teach the entire team
28:15how to use CFIR.
28:19But CFIR is a framework, it is not a model.
28:22So by that it does not really suggest causal linkage
28:26between the components and the outcome.
28:29So we are also considering use normalization process theory
28:33NPT as a complementary framework to CFIR.
28:40However, I have to say, even though normalization process
28:42theory is very nice in terms of illustrating
28:45the causal linkage of implementing process and the outcome,
28:51sometimes it is not so easy to use.
28:54For instance, in this picture is one of our
28:59reconfiguration of the domains and the constructs
29:03from normalization process theory.
29:06We have spent quite a bit of time in studying
29:07and understanding NPT and organize it in a way our students
29:12and the research team can understand better.
29:14But even with that we still found some of the constructs
29:18of this model is a little difficult to distinguish
29:21like inter action workability relational integration
29:25and skillset workability, those constructs can be very
29:30easily like confusing for our researchers.
29:34However, it is still one of the rare implementation
29:37science theories specifically for implementation,
29:40so we are considering using it for SMA as well.
29:43So how we are going to use it, this is a picture
29:46from the journey to the west, one of the very famous
29:49Chinese classic.
29:50It has many of the stakeholders for the journey to the West
29:54from China to India to fit the classic scripts for Buddhism,
29:59so you can get the Buddhism back to China
30:02and implementing that.
30:04So essentially we want to do stakeholder analysis
30:07to determine and engage the community.
30:11We want to have community engagement involvement
30:15and engage in your stakeholders so we can determine
30:17what are possible the facilitators and the barriers
30:22for your implementation of SMA, so in this process
30:28we will use CFIR and NPT and use them to design
30:32or survey form.
30:34So those can be used as a quantitative survey form,
30:37but we will also use them to design interview guides.
30:41So we can use them to do in-depth interview or even use it
30:45for focus group and we'll also use those frameworks
30:49to analyze data.
30:50So implementation science framework, CFIR and NPT.
30:54We actually guide us throughout the entire process
30:58of our study.
31:00This is the barriers and the facilitators we have identified
31:04from the literature, not from a actual study as our study
31:07is currently ongoing.
31:11But now think if we have already determined the barriers
31:14and the facilitators to normalize EBP of SMA
31:20in our clinical setting in primary healthcare,
31:22now we need to develop implementation strategies to deal
31:26with each of the barriers.
31:29In order to identify implementation techniques,
31:32then you can package those individual techniques into the
31:36package we call implementation strategy,
31:39which can deal with the implementation barriers effectively
31:43so we can improve uptake.
31:45We already have good studies in developing taxonomy
31:50of implementation strategies which are expert recommendation
31:55for implementation change.
31:56The ERIC, so ERIC is one of the popular framework
32:00which have categorized all the available implementation
32:04technique they can identify.
32:06But the key is really to, I identify those
32:09available implementation technique which very often
32:13have already some evidence-based and match them
32:16to your implementation barrier.
32:18So this is a step critical in developing your action,
32:22which means developing your implementation strategy.
32:25But the key, the big challenge is what are the methods
32:29you can use to match entertainer implementation strategies
32:36to your barriers in your setting.
32:39This is a really a under-researched area in
32:41implementation science.
32:43However, one of the researcher has suggest four methods
32:48we can consider, one is called concept mapping,
32:51which is a visual mapping using mixed and methods.
32:56The map here is one of the visualization of the barriers
33:01in implementing some EBP.
33:07Then the second method is group model building,
33:10which is sort of the a causal loop diagram of complex
33:14problems.
33:15The third is a conjoint analysis, conjoint analysis
33:20has different forms.
33:21One of the most popular form is called a discrete choice
33:24experiment, which we are going to talk a little later
33:28because we have opted for DCE Discrete Choice Experiment
33:32for our study.
33:34The last one is intervention mapping,
33:36which is a systematic and multi-step development
33:39of interventions.
33:41All of those four methods have been extensively used in
33:45other fields but not as much in implementation research.
33:48So I'm really highly encouraging all of us
33:51in doing implementation research to use some of those
33:54methods in systematically match and retainer implementation
33:58strategies to the barriers you have identified
34:01in your study.
34:02So for us, we are going to use a difficult choice experiment
34:05to tailor implementation strategies for SMA,
34:09DCE is widely used in health economics,
34:12but not as much in implementation science.
34:15DCE belongs to the method in conjoin analysis,
34:18DCE in our team we have used the DCE before
34:23in understanding healthcare professionals preference
34:27for working in the primary care setting job preferences.
34:31According to review, they have identified
34:3422 DCE studies comparing different implementation
34:38strategies.
34:40So it is not so much used as as much in other field of work.
34:47So use of DCE in our SMA study is like this
34:51in basically in DC you it's a combination of a quantitative
34:56and a quantitative work.
34:58You first it's most likely we'll use a quantitative work
35:03and also literature review to identify what are the
35:07possible implementation techniques to be developed.
35:11So we develop those implementation strategies,
35:14the techniques through initial review of literature
35:19and expert consensus.
35:21So for instance, if we have identified through this process
35:26audit and feedback is one of the major implementation
35:30strategy to deal with this barriers,
35:33we have identified the DCE then can do the work of
35:37painter audit and feedback to the specific setting
35:41to implementing SMA, because why we are going to do this,
35:45because even it is called audit and feedback,
35:49it actually has many components.
35:52This is very much like EBP of SMA can be complex
35:56as many components.
35:57Our implementation strategy can also have many components.
36:01So we can develop these different components of audit
36:06and feedback including format of feedback.
36:09Is it a verbal or written recipients of feedback?
36:13Do we feedback to individual clinician or feedback
36:16to the entire group the source of feedback,
36:18is the feedback coming from that influential source
36:23like their peers or supervisor or is it coming from
36:26the researchers?
36:27How we are going to deliver the feedback by emails,
36:30by letter or in person?
36:32How frequent your feedback should be monthly
36:36or every four months.
36:38Now how the instruction for feedback need to be developed.
36:41Will it be explicit, measurable, targeted but no action plan
36:47or should it be accompanied with action plan,
36:50but no explicit target or in addition to audit and feedback,
36:55do we need to copy that with another
36:57implementation strategy?
36:59Say giving people financial incentive.
37:01If we are going to do the SMA, we give them extra money
37:05to do that.
37:06Okay, so the audit and feedback and our implementation
37:10strategy have all those individual attributes
37:13and all those attributes have levels.
37:17So we can based on those, we can develop different choices
37:22have one to many, many, many, many choices for.
37:27So we present those choices side by side to our respondents,
37:32to our stakeholders to so they can choose between those
37:35two choices.
37:36Which set would you prefer is our approach to improve SMA
37:41in your organization So they can make the choice,
37:43so after the respondents the stakeholders have
37:47making all those choices from those choices then we can do a
37:53statistical analysis.
37:55With that we can determine how preference are influenced
37:59by each attributes and we can also give the
38:02relative importance of those attributes.
38:06And in particular, once we give a financial incentive here
38:09have a dollar amount, we can actually measure and transform
38:14all those attributes into something called
38:16the willingness to pay.
38:18So we can precisely quantify the value of all those
38:22individual attributes.
38:24So after we have done this exercise,
38:27we can understand the preference of our stakeholders,
38:31what kind of audit and feedback they think might work best
38:37even though this is pre-implementation,
38:41so after doing this we can develop a complete package
38:44of implementation strategy.
38:47So after we have done this, the important thing
38:50before we can move this to sustain the use
38:53is to come up with very good monitoring and evaluation plan,
38:57which will entail to develop evaluation designs
39:01implementation outcomes and measurement tools.
39:05So let's focus on this S part of the S model.
39:09First we need to understand what kind of design we want to
39:14tailor this into.
39:16There are something called a hybrid design.
39:19Hybrid design is sort of the design you are trying
39:23to balance to what extent you want to have this study
39:26as a effectiveness study of your EBP, which is SMA
39:30or to what extent you want to test the implementation
39:33outcome of your implementation strategy,
39:36which in our case can be audited the feedback.
39:40So depending on the priority of set to those two outcomes,
39:44it can be type one, type two or type three hybrid design.
39:49For type two hybrid design, you are going to test
39:53both the effectiveness of the EBP
39:56and also to test the effectiveness of your implementation
40:00strategy, because SMA has not yet been done in China before.
40:05So it's very important for us to test the factories of SMA,
40:11but it is also important for us as an
40:13implementation scientist to test the implementation strategy
40:17of audit and feedback in implementing SMA.
40:21So we opted for type two hybrid design for implementation
40:25for implementing SMA in our setting, but what type of design
40:32we are going to use for our trial, we are considering to use
40:36something called the step wedge design, step wedge design
40:40is a unique type of randomized control study,
40:43but it allow gradual implementation of SMA across
40:48the primary care institutions centers in our sample.
40:53So for the use of a step wedge design, eventually everybody
40:58in your setting in your S participants will receive SMA.
41:04So this give the gradual implementation of SMA
41:09has some advantage, because then we can facing our manpower
41:14so we can ensure we really implement SMA in our institutions
41:19step by step gradually.
41:21And also all sites eventually receive SMA means ethically it
41:25is better than some people only serving as controls
41:29without the benefits of the SMA.
41:32Also it says the step wedge design has a great statistical
41:37property so with the same sample size,
41:40normally it has a much higher statistical power
41:44than conventional two (indistinct) control study.
41:47But the complications of step wedge design is the analysis
41:52plan is much more complicated than the two randomized
41:56control study and also the length of your study
42:00is much longer than your study.
42:03So in our specific for SMA study in the study we are going
42:07to do the effectiveness trial for SMA.
42:10So five counties in China, each county have two
42:15primary healthcare centers.
42:17So we will randomize those five counties in those six steps,
42:23randomize them into receiving SMA gradually until in the end
42:27all of them are receiving SMA.
42:30But for each of the county, one of the primary care centers,
42:35one of them will receive the audit and the feedback
42:39as a implementation strategy.
42:41The other will receive another type of usual implementation
42:44strategy, so we can compare in this study
42:47both the effectiveness of SMA but also can use experimental
42:53design to compare the effectiveness
42:57for implementation outcomes for audit and feedback
43:01versus usually of implementation.
43:07<v Maur>I just wanted to let you know we have less</v>
43:10than 15 minutes, so I just wanted to make sure.
43:12<v Roman>Sure, sure, sure.</v>
43:14I only need two more minutes to wrap this up.
43:17I have a big timer on my side.
43:19<v Maur>Oh perfect.</v>
43:20<v Roman>Reminding of that.</v>
43:22So we also have implementation outcomes,
43:25which I think people are already familiar with.
43:27The one thing I want to emphasize is for implementation
43:30outcomes and also for patient and the service outcomes,
43:33they all have two dimensions, the absolute obtainment
43:37and also the equity which is the distribution
43:40of your outcomes among your stakeholders.
43:44We're going to use the RE-AIM as well, but there are many
43:48challenges actually in using a RE-AIM, RE-AIM
43:52is not as simple as it appears, because it is sometimes
43:55it's difficult to operationalize say
43:58the implementation outcome for RE-AIM framework,
44:02say how do you measure fidelity of delivering SMA?
44:06One of the things we're considering the measurement tool
44:10is to use standardized patient.
44:11We have not yet decided on this yet, but because
44:13we have already been the conducting a very massive study
44:18of using standardized patients in assess quality of care
44:23in China, which is a fake patients, but they are trained
44:26so they control the case mix and there is no
44:30Hawthorne effects compared with other direct observation
44:34of your clinical practice and using standardized patients
44:41can also enable quick audit and feedback work.
44:44So we are considering seriously because of our experiences
44:48and the expertise in using this method
44:50in assessing primary care quantity, we are considering
44:55using this as a quantity outcome collecting tool
44:58to understand the fidelity and implementation process
45:04of SMA for our settings.
45:07If we are interested in this method further,
45:09you can check out two of the papers that we have published
45:12to illustrating how this can be used in other setting.
45:16So that is what I am trying to talk about this PEADLs model.
45:21So basically this is the model is to give researchers
45:25and students to think about your thought implementation
45:28research from identifying the problem in your work.
45:32Then you need to identify EBP to address that problem,
45:36but you really need to understand what might be the barriers
45:39and the facilitators and based on that to develop
45:42your implementation strategy in order to achieve
45:45sustained use.
45:46But all across this process you have to have a very sound
45:50and a good evaluation design plans and measurement tools.
45:57So thank you so much, I hope this presentation
46:00can motivating some people to come for our program
46:05for postdoctoral fellows in implementation science
46:09in our Acacia lab and our center for
46:12Institute for Global Health.
46:14We give very nice benefits for people coming to China
46:18to do two to three year postdoctoral fellowship
46:21in implementation science, all of you are welcome to apply.
46:24Thank you so much.
46:27<v Maur>Great, thank you Roman, that was fantastic,</v>
46:31really a terrific presentation.
46:33I'll go ahead and open it up to questions.
46:36If you have any questions feel free to just unmute yourself,
46:40introduce yourself and ask or I see Donna
46:43has raised her hand.
46:43Donna, why don't please go ahead.
46:46<v Donna>Hi everybody, sorry I'm on the train actually,</v>
46:50I had to go into New York City today for an appointment,
46:53so there's some background noise I apologize for.
46:56But Roman, I just wanna say that this was just an
46:59absolutely brilliant talk where you walked us through all
47:03of the essential aspects of implementation science
47:07from the beginning to end and connected.
47:09How these various theories and frameworks where they jump in
47:13where we need 'em, what might be a recommended approach.
47:16I mean just absolutely fantastic and I'm sure
47:20the audience learned very much from this talk.
47:23I know all of us struggle with the confusions
47:26of these theories and models and frameworks and where they
47:30fit in what is implementation science and the steps of it.
47:33You just laid it out so clearly it's just,
47:36I'm just floored at how nice this was.
47:39So thank you so much and on behalf of all of us for this,
47:44I have two questions actually.
47:46One comment you mentioned, you know I'm somebody who's
47:50done research on developing statistical methods
47:54for step wedge design, you mentioned two drawbacks.
47:57One is that it takes longer, which I completely agree with
48:01and I think it's worth documenting that better,
48:04because I'm not sure there's any kind of papers
48:06or publications that actually show that trade off.
48:09And then the other is, and people ask me
48:11and I know it's longer but I can't really say
48:14how much longer exactly.
48:16So I think it probably gets the longer,
48:19the more step times you have.
48:20But anyway, I'm not gonna speculate right now.
48:23But the other thing you mentioned was that the analysis
48:26was more complicated and it's true that in
48:28a parallel cluster randomized design,
48:30which is usually the other alternative,
48:33you can just basically compare the mean outcome rates,
48:36whether they're continuous or binary at the end of the study
48:40using a two sample tests, but you do have to account
48:43for clustering even there.
48:45And then with the step wedge design, it's only one step
48:49more complicated in that that comparison has to adjust
48:53the time effect.
48:54But there's very standard statistical methods
48:58that basically every kind of software to do a
49:02generalized linear model or a regression model
49:05that accounts for clustering and allows for
49:07a binary intervention effect and then indicator variables
49:11for every time effect and then perform the test
49:14of the difference between the two groups based
49:17on the regression coefficient using either a robust wall
49:21or a robust square tests.
49:23So I'm not sure why you felt it was like an actual barrier,
49:28I just don't feel that that should be so,
49:31and then my last comment, because we've been chatting about
49:34this is the issue of quality and how that fits into,
49:37especially in low and middle income countries,
49:41I think it's sort of assumed in the United States
49:44all you have to do is get the service out to somebody
49:47and the quality is already very high,
49:49we don't have to worry about that.
49:51It's probably not true, but that's the assumption.
49:53But in low and middle income countries may be,
49:55and it may not even be true, the quality issue
49:57is even bigger and it doesn't seem to be something,
50:01it seems to be addressed in the health systems
50:03research field but not, I haven't heard any chatter about it
50:08implementation science.
50:09So anyway, my question was complexity of step wedge design
50:13analysis and then this issue of quality.
50:16And just thank you so much again for an absolutely
50:19like fantastic crystal clear talk.
50:23<v Roman>Thank you Donna.</v>
50:25I think I'm not a statistician, but I think this is
50:29precisely where statisticians like Donna,
50:33you can play a really big role in help strengthening
50:37the methods in implementation science.
50:39Step wedge design has a lot of potential
50:42for implementation science, I think that
50:44is my understanding, but statistician can correct me.
50:47I think it's generally longer than traditional two RCT.
50:52The reason is for the steps, like the step here,
50:56we allow it three months for one phase, one step
50:59because for each step you have to have long enough
51:03a duration to allow SMA effect to be fully released
51:10if the effect cannot be fully released during one step,
51:14you have to use even more complicated statistical
51:18method for analysis.
51:20So because in theory if it is a two arm RCT
51:27in three months you can wrap up this study.
51:30But for implementation for step wedge it is much longer,
51:34so that is one reason.
51:35In terms of analysis, I agree with
51:38Donna now compared with the several years ago
51:42there now have been many software R package coming out,
51:47which can enable analysis much easier.
51:50But still sometimes it's difficult for researchers
51:53which have no statistical background to understand
51:57why they need to do this and why not.
52:00Sometimes it's always good to embed some good statistician
52:05in your team even though the software can do a lot of
52:08work for you.
52:09In terms of quality of care,
52:11I agree with Donna, it's a severe issue,
52:14increasing coverage is not the only thing to do
52:18in low income countries.
52:20The coverage without a quality service can be harmful
52:24and the risk of your resources.
52:27So that's why our research team case study focus,
52:31a lot of our study we use as standardized patient
52:34to assess quality of care across seven provinces in China,
52:38which is unprecedented, because of the implementation
52:42of standardized patients is really very difficult
52:47to that scale, but we have demonstrated it's possible even
52:50using in that setting.
52:53So we now have a precise understanding of the quality
52:56in China, which is, I have to say very, very poor.
52:59We have not yet get this paper out,
53:01but once it is out we'll share, thank you so much.
53:08<v Maur>Great, thank you.</v>
53:10I don't know if there, we have a couple minutes left.
53:12I don't know if anyone has any last comment or question.
53:25Go ahead, I see your hand raised.
53:27<v Attendee 1>Yeah, thank you Dr. Xi</v>
53:30and it's very great to meet you here.
53:33Yeah, I think this is a great talk.
53:36I really learned a lot about implementation science
53:39and as I can see, I actually have a question about
53:44the third stage in PEDALs, the third stage is determinants.
53:48So I can see you list health intervention factors
53:51and also contextual factors.
53:55So I would like to know whether you have insights on
53:59how to analyze or disentangle the relationship between
54:04health intervention factors and contextual factors,
54:10how would you consider their relationship?
54:12Like whether contextual factors can be considered
54:15as mediators or whether it's hierarchical design
54:21that contextual factors need to be on a higher level
54:25and whether it involve very complex analysis
54:31and yeah, like what we want to get out of this analysis.
54:39<v Roman>Thank you Pungfe, actually in most of the</v>
54:43implementation science framework, they put the
54:47barriers associated with health intervention
54:53itself and the contextual factors together
54:57in the framework like CFIR and TDF, many other frameworks,
55:04which already wrapped both elements in one framework,
55:11but I have tear them apart because in our SMA study,
55:17we essentially have decided to do this in two steps.
55:22One is to optimize SMA so it can work better
55:29in the Chinese setting,
55:31as I have described in the past SMA, although there are many
55:35RCTs to prove its effectiveness,
55:38but most of them are in high income countries
55:41and almost none of them have compare head to head
55:45the individual components in that complex SMA study,
55:50so we don't really know what individual components
55:53can work best.
55:56So after we have done that, then we goes to understand
55:59for the optimized package complex intervention,
56:03what are the contextual factors which can contribute in
56:06to the implementation of that health intervention SMA.
56:10So in our study we sort of have clearly divided those
56:15into two steps even though the optimized SMA
56:20can still be a factor, which can create barriers
56:23in our final study I have to say.
56:26The other thing is it's very difficult to distinguish
56:29sometimes, sometimes people use contextual factors,
56:32environmental factors, settings.
56:34Sometimes it is very difficult to distinguish
56:38the difference be between them.
56:39I tend to not to distinguish them, because different people
56:43have different ideas whether one off to use
56:45other hierarchical analysis for the intervention factor
56:51and the contextual factor, I can't answer that
56:55because I need more time to think about that.
56:57I don't yet have a clear answer to that yet,
57:02but I tend to think it may not be a hierarchical
57:05analytical question here.
57:07Thank you Pungfe.
57:09<v Pungfe>Yeah, thank you very much.</v>
57:14<v Maur>Great, I see a couple of other hands raised.</v>
57:16I think Gloria was next.
57:19<v Gloria>Yes.</v> <v Maur>Go ahead.</v>
57:20<v Gloria>Thank you very much Maur, thank you very much.</v>
57:22Very nice talk, I really enjoyed.
57:26You mentioned several techniques in order to choose
57:30the implementation strategies.
57:33And this is, you know very complex issue
57:34because you have a lot of implementation strategies
57:38that how to use them in the context or with the problem
57:43that you have and how to choose them, right?
57:45Is a like a real point in implementation process.
57:49Can you please elaborate on that?
57:52Thank you.
57:55<v Roman>I have some challenges of understand the question.</v>
58:00Maur, can you paraphrase the question?
58:04<v Gloria>Yeah.</v>
58:04So basically, you know, you mentioned that you use
58:07several techniques to choose the implementation strategies.
58:12So can you please elaborate on those techniques
58:16or methods that you use to choose
58:18the implementation strategies?
58:20<v Roman>Oh, okay.</v>
58:21Okay, thank you.
58:22Thank you, sorry for that.
58:23<v Gloria>That's okay, thank you.</v>
58:25<v Roman>Yeah, there are many implementation,</v>
58:30there are many methods which can be used to map
58:34your strategies to barriers.
58:38I would say the simplest strategy is not one of those four.
58:44The simplest strategy is simply stakeholder consensus.
58:50For many time, if people do not have a higher level of
58:54methods, you can simply have a group consensus
59:00to be achieved through a Delphi process or a nominal group
59:06process.
59:08There are many simple way of achieving stakeholder
59:13consensus on what type of implementation techniques,
59:18which you can select from the ERIC framework
59:21to match each of the barriers you have identified,
59:24but the four methods here listed on this slide
59:28are more methods driven and I have never used
59:33a concept mapping group model building
59:37and intervention mapping, but we have used
59:39a conjoint analysis in a way to use difficult
59:43choice experiments.
59:45So as I've discussed earlier, which is the sort of
59:48the questionnaire you have developed (indistinct).
59:51So you present those products, each product consists
59:55of different attributes of your implementation strategy.
59:59So you'll present those products side by side
01:00:03to your stakeholders.
01:00:04They make a choice out of the two, but you have
01:00:08many of them.
01:00:09So once they have complete data, all of the choices,
01:00:12then you can perform a logistical regression
01:00:14and other statistical methods to really evaluate
01:00:19and quantify the value of those individual attributes.
01:00:24So then you can choose the attributes with the highest
01:00:27valuation and package them into the packaging
01:00:32of your implementation strategy, I hope this helps a bit.
01:00:36<v Gloria>A lot if you, thank you.</v>
01:00:38<v Roman>There are lots of literature.</v>
01:00:40I'm using DCE in health economics and health service
01:00:46literature, it is not really a very difficult method
01:00:48to understand, so I can send out some of the literature
01:00:55as well.
01:00:56<v Maur>Okay, you thank you very much.</v>
01:00:58<v Roman>Thank you. (indistinct)</v>
01:01:00<v Maur>Roman, thank you.</v>
01:01:02I know it's very late where you're now in China,
01:01:05but we have one last question if you wouldn't mind.
01:01:08Mariana, go ahead.
01:01:10<v Mariana>Yes, thank you so much for these great talk.</v>
01:01:13Mariano Kaori from the University of Miami Miller
01:01:15School of Medicine.
01:01:18I just have a quick question.
01:01:19Can you implement this very comprehensive study
01:01:23in five years?
01:01:26<v Roman>Ah, that's a good question.</v>
01:01:30That's our hope in five years, but let me share the story.
01:01:36I used to do a study using texting as a reminder
01:01:42for people with schizophrenia in the rural Chinese
01:01:46village to take medication.
01:01:49The implementation is for six months and we thought
01:01:52we are going to have that done simply within eight months,
01:01:56but it takes us three years.
01:01:58So implementation of those trials always take longer
01:02:02than we thought, but I have to say we are exactly trying
01:02:08to do this within four years time.
01:02:12But with, I think one of the element is the pandemic.
01:02:19A (indistinct) know China is fortunate in a way.
01:02:23We are not very much affected by the pandemic
01:02:27for the past two years.
01:02:28Our life here in China is essentially normal
01:02:31for majority of the people, so we are able to done
01:02:32a lot of the field work.
01:02:34But now with the omicron all the other part of the world
01:02:39are opening up, China has some challenges.
01:02:42So we don't know whether in the future years
01:02:44whether this will be playing a part,
01:02:46but even without pandemic, sometimes it's difficult
01:02:48to get the implementation done.
01:02:50However, what I have to say is we have develop it
01:02:55and really excellent consortium of collaborators in China.
01:03:00We have 12 research teams in China.
01:03:03We have always been working together
01:03:05and many of the health service and implementation research,
01:03:09even without any grant support, our teams are
01:03:12working together, so we know each other extremely well.
01:03:15So when we are doing those multi-site trial,
01:03:18it's almost much easier now to set up your team,
01:03:24because it's communication is a simple and how to divide up
01:03:27your work is already established and how to share
01:03:29your intellectual property is prior grade upon
01:03:34and how to mobilize your resources and what are
01:03:36the statistical data management platform,
01:03:41all this has already been been constructed
01:03:44in our prior studies.
01:03:45So very easy for us to conduct multi-site study in China
01:03:49because of our existing work of the Acacia Labs
01:03:53with those a dozen research teams always
01:03:55be working together.
01:03:57We also have very strong support from the clinical centers,
01:04:01because we work with some of the clinical centers
01:04:04in many other ways, so we get to know them much better
01:04:08and we can get a support from them as well.
01:04:10So hopefully we can get this done within four years.
01:04:13But I have to say things happens, it may get longer
01:04:16than we thought, thank you.
01:04:22<v Maur>Great.</v>
01:04:23Great, well thank you again.
01:04:24I don't wanna cut off the discussion and the comments,
01:04:28but I know it's getting late there.
01:04:31If there are no other questions, I'll just end by thanking
01:04:35you again, Roman, for being with us today,
01:04:37it was a fantastic presentation, really enjoyed it
01:04:41and learned a lot as I know everybody else did on the call.
01:04:45So thanks so much for being with us
01:04:47and see you all again soon, thanks everyone.
01:04:52<v Roman>Thank you, bye-Bye</v>
01:04:54<v Donna>bye.</v>
01:04:55<v Roman>Thank you, bye-Bye, bye-Bye.</v>
01:04:57<v Maur>Bye everyone.</v>