Using Qualitative Thematic Analysis to Explore Financial Donor and Recipient Collaboration in Liberia

December 15, 2023

Information

Speaker: Brigid Cakouros, DrPh, MPH

Friday October 20^th, 2023

Description: This seminar discusses how she and her team conducted a qualitative analysis of how donors and recipients of global health funding discuss what collaboration looks like within this landscape of Liberia. The analysis consisted of two phases, the first phase sought to map the simple causal loop of who holds funding power in Liberia, and the second phase explored the underlying dynamics that perpetuated unequitable collaboration practices.

ID11098

To CiteDCA Citation Guide

00:00<v Ashley>Okay, it's two after,</v>
00:02so I think we'll go ahead and get started.
00:08My name is Ashley Hagaman.
00:11I'm a faculty member
00:13in our Social and Behavioral Sciences Department
00:17and the Director of the Qualitative Methods
00:19Innovation Program within our Center for Methods
00:22and Implementation and Prevention Science here at Yale.
00:26And we are delighted to welcome Dr. Cakouros
00:34as our seminar speaker today.
00:39She is a global health scientist and mixed methodologist
00:43with expertise in health systems.
00:46I met Bridget a couple of different ways.
00:49One was through one of our
00:51Master's in Public Health students
00:53and another was an email just asking
00:57about access to qualitative methods training
01:00and methods just like more generally here at Yale.
01:04And then of course our shared interest in global health.
01:06So we had been in some common spaces together
01:10and I was just immediately impressed
01:12with her wealth of experience
01:14working on complex systems studies
01:18and in sort of implementing programs around the world.
01:22She has a really interesting and really important focus
01:25on developing and development research more broadly.
01:30Global health research and global health development
01:33are topics that I engage with every day
01:35as a part of my own career.
01:37And my teams and I often think about the larger dynamics
01:41that we have with our composition
01:43and our positionality and relationality
01:46both within our team and our project is being implemented.
01:49But I hadn't thought about how we understand that
01:55or what we do with that and kind of what we can learn
01:58about these dynamics more broadly.
02:03And so Dr. Cakouros' work is so important
02:06because it will help us not only do better
02:09global health work and health systems work
02:11but it'll help global health programs be more successful
02:15which is really all health programs.
02:16And so at Yale, she is a postdoctoral fellow
02:20with Professor Talbert-Slagle, the Associate Director
02:23of our Yale Institute for Global Health.
02:25And today she's going to talk about qualitative methods
02:29to study global health collaboration
02:32anchoring in her collaborative work in Liberia.
02:36And so thank you so much for joining us Dr. Cakouros
02:40and I'll hand it over to you.
02:42And then just a note, I can kind of monitor the chat as well
02:46in case there were any questions
02:47but let us know if you want to take questions
02:49in the middle or if you just want to wait till the end.
03:04(Brigid speaks faintly)
03:11(audio distortion)
03:35<v Brigid>I'm just going to go to my camera now.</v>
03:37Here's my computer, there you're welcome.
03:40<v Ashley>Hey Brigid, just to stop you for a second.</v>
03:43It sounds very muffled on our end.
03:45<v Brigid>How about now?</v>
03:46<v Ashley>Ooh, that's great.</v>
03:47<v Brigid>Okay, we switch it to my computer.</v>
03:49So when we start maybe in chat as a group.
03:53<v Ashley>Okay, thank you.</v>
03:55<v Brigid>Perfect.</v>
03:56All right.
03:58So to summarize what I was saying,
03:59I'm just really excited to talk about this project.
04:02I've been working on it for about a year now,
04:04a little over a year.
04:06And I'm really excited about qualitative methodology
04:10and especially in global health
04:11and what you can learn about systems
04:13and how systems operate using qualitative methods.
04:17It's kind of a fun frontier to be working on
04:20and I'm looking forward to sharing this project with you.
04:23So this is titled using qualitative thematic analysis
04:26to explore financial donor
04:28and recipient collaboration in Liberia.
04:32And a little overview, there we go,
04:37for what I'll be talking about today.
04:39I'll talk a little bit about why Liberia,
04:41why it makes sense to be thinking
04:43about collaborations in Liberia, kind of what sets the stage
04:46for the importance of studying this topic there.
04:48And then I'll talk about the study design and methodology,
04:52why we chose the methods we did,
04:53the processes we went through
04:55for analyzing our qualitative data and such.
04:58And then I'll touch briefly at the end,
05:01summarizing some of the lessons we learned.
05:02This was really a training opportunity
05:04with some of our students.
05:06So I talk about that kind of throughout
05:08this presentation as well,
05:10but I think it's kind of cool to reflect on that as we go.
05:13And there's a couple of pictures throughout here of just,
05:16they'll be of the team or pictures I took in Liberia.
05:19So they're really to just enhance this visually
05:22because qualitative sometimes is just words,
05:24but giving a little bit of context of where we were.
05:28So why Liberia?
05:30I'll give a little brief background on it.
05:32And I wanna highlight that what I'm gonna focus on
05:35is really the reasons why the health system there is broken.
05:38So it's focusing on some of the tougher times
05:42and the harder hit parts of the health system.
05:46Liberia is great.
05:47There's a lot to offer from this country.
05:49So I'm not trying to just ground Liberia
05:52and being like this place of civil conflict and Ebola,
05:56but that's really what's weakened the health system
05:58to make this type of study.
06:01So between 1989 and 2003, I'm sure that people remember
06:06that there was civil conflict there.
06:08It was more by child soldiers and extreme poverty and fear.
06:1390% of the skilled health workers fled the country.
06:17So the health infrastructure was destroyed.
06:19And this really started a strong reliance
06:21on humanitarian organizations and international NGOs
06:25and foreign donors to kind of keep this system afloat.
06:29So then there was a brief period
06:31of health system rebuilding from 2006 until 2013.
06:36This is a picture of Ellen Johnson Sirleaf,
06:39the first female elected president
06:41on the continent of Africa.
06:43Very exciting.
06:44And she really, when she was sworn into power,
06:46she not only negotiated down and canceled a lot of debt,
06:49but she was able to renegotiate some investments
06:52from global donors.
06:52So she was really working on rebuilding
06:54a lot of these partnerships.
06:56And this chart here just shows the change
06:59in under five mortality rate,
07:01which is a strong indicator of a health system.
07:04So Liberia had started to turn the corner
07:07right around 2010, a little bit earlier in the early 2000s
07:13from when it was in civil conflict.
07:16And access to health facilities increased to 71% by 2013.
07:21In 2008, only 41% of people were within one hour walk
07:26of a health facility.
07:27And by 2013, it was up to 71%.
07:29So these are huge gains in a short amount of time.
07:33But then there was Ebola,
07:35which also many public health people probably remember.
07:39There was just over 5,000 or just under 5,000 deaths,
07:42including 8% of the skilled healthcare providers,
07:46which is the doctors, nurses, midwives.
07:49So routine healthcare service essentially collapsed
07:53once again, which was really unfortunate
07:56because as I noted in that period of rebuilding,
07:59they were really on a strong chart to go forward.
08:03So now what?
08:04You know, this timeline not only increased
08:09reliance on donors, it brought in a lot of donors.
08:12So many donors started engaging in Liberia
08:14with different aspects of strengthening
08:17the global health system.
08:18There was funding coming in from many partners.
08:22And again, you know, there's many more people
08:26that are just even highlighted on this slide.
08:29So in 2015, Liberia created this investment plan
08:34for building a resilient health system
08:36because clearly the system's taken some hits
08:38and it's how do we build up, make it even stronger?
08:40So again, this is just some examples
08:43of the numerous partners working in Liberia.
08:45And if you're thinking about the government
08:47as where this document is fitting,
08:50think about how many different negotiations
08:52and collaborations and partnerships are operating
08:54just over this document.
08:57And again, if that's the government,
08:59here's where the government sits now.
09:00You know, some of these organizations
09:03are dealing with one or two different arrows.
09:04The government is negotiating all of them.
09:06And, you know, just to highlight some of the arrows
09:09don't even connect directly to the document.
09:11So it's a really complex kind of daunting system
09:15to try to negotiate what these collaborations look like.
09:19So now I hope I've convinced you
09:20that Liberia is a very interesting place to study this.
09:23I'll talk a bit now about the study design and methodology
09:26and we'll talk about some of the influences
09:30for how this was designed.
09:32So the overview and goal of this particular study
09:35was to explore these dynamics
09:37of global health collaborations
09:38from the perspectives of those working in Liberia.
09:41So pictured here is Dr. Bernice Dahn.
09:44She's a former minister of health.
09:46She was chief medical officer
09:47for a time as well in the country.
09:50And she's just all around, you know,
09:53an expert on health system resilience in Liberia.
09:56And she gave a lecture at Yale,
09:58I believe it was in either the spring of 2022
10:02or the fall of 2021,
10:03titled The Unchecked Power of the Purse.
10:05And this talked about a lot of the inequities
10:07she's seen over her career working in global health.
10:10And from this lecture,
10:12there were two master's students and two undergrads
10:15along with Christina Talbert-Slagle
10:17who designs this qualitative study
10:19to explore these relationships in Liberia.
10:22So Dr. Talbert-Slagle was the co-PI along with Dr. Dahn.
10:27So we had the US representation affiliated with Yale
10:31and then strong Liberian representation as well.
10:34So there's this whole push that global health
10:37is inherently equitable
10:39when you talk about the word collaboration,
10:40but I think many of us kind of know that it's not.
10:43And from Dr. Dahn's experience,
10:45she's talked about how accountability and transparency
10:49are a one-way street.
10:50Essentially that the donors are controlling
10:52what accountability looks like,
10:54what is allowed to be transparent.
10:57She's talked about donors' priorities
10:59being favored over government needs
11:02and that there's a value of,
11:04there's claims of value of empowerment,
11:06but that's never actually transferred to the government.
11:10So it's leaving weak systems weak
11:11and perpetuating this corruption
11:14and ideas of perceived corruption.
11:16So really hearing these issues,
11:20knowing that Liberia is a great setting to study this,
11:22this is why a really qualitative study
11:24makes a lot of sense here.
11:26So if we're looking at,
11:27trying to understand the subjective meanings
11:29from social contexts, from perceptions and understandings
11:32and these actions and behaviors
11:34of people working in this setting,
11:36we can try to map out what this actually looks like.
11:39You can have as many structures and norms
11:41and frameworks as you want,
11:42but we really would like to understand
11:44and gain insight to the human experience here.
11:48So a couple of terminology,
11:49I know I've talked a lot about the word collaboration.
11:54So collaboration generally in this type of setting
11:56is a low resource country that needs support.
12:00Here we're talking really about financial support
12:02from a high resource setting.
12:03And this again can create a sort of reliance
12:07on needing that type of funding and support.
12:10And I think it's really important
12:11that collaboration is accepted as positive.
12:15It's not always positive.
12:17You really wanna understand the nuance
12:19of what's driving a collaboration
12:21or collaborative partnership.
12:22And this is kind of known
12:24in the field of global health and development,
12:26but there are calls to have more equitable collaborations
12:31and decrease this dependency, but is it really happening?
12:35So again, another push for why qualitative.
12:38By the end of this part,
12:39I'm gonna be convinced that qualitative is perfect
12:41to be studying this.
12:44And so, oops, my keys jumps forward a few slides.
12:50To sum up this notion of,
12:53we can have all of these calls and frameworks.
12:55We can have normative accounts of why equity is essential,
12:58and we can have practical guidelines,
12:59which are frameworks and norms
13:01of how we should be operating in these collaborations,
13:03but we need these empirical studies to inform action.
13:07And I think this quote from Bauer,
13:09it's a study of why considering equity
13:12in global health collaborations is necessary.
13:15This really just sums up all of that.
13:17I'm not gonna read the whole quote,
13:18but I'd like to focus on this idea
13:20that empirical studies can provide important insights
13:23from the experiences of those involved
13:25in developing equitable research collaborations.
13:28They could also inform policies, frameworks,
13:31and guidelines related to equitable research collaborations
13:34in global health.
13:35So tying to implementation science,
13:37we're really trying to build an evidence base
13:39of we have these frameworks,
13:40but are these frameworks actually doing
13:42what we think that they are doing?
13:47And just, again, I wanna reflect on this image.
13:50This is what we're trying to understand.
13:52We're trying to use qualitative data
13:54to make sense of these arrows
13:55and to discern some of the patterns
13:57and how individuals are acting in this system.
14:00And a lot of times in global health,
14:02in health system studies,
14:03we talk about the software and the hardware of the system.
14:06And briefly, I'll just talk about that.
14:09The hardware is generally the infrastructure,
14:12the finance, technology,
14:14interventions that are really easy to measure
14:15and see that we're making these investments.
14:18And then associated with that is the tangible software.
14:21So those are kind of go together.
14:23That's how you create structures and systems
14:25and hierarchies of how you're going to make change
14:28within health system.
14:30Now, what we're really curious about though,
14:32and especially in this study,
14:35are the values and norms and the relationships
14:38and communication and power that exists
14:41that aren't as easy to measure or to understand.
14:45You can't really put a quantitative measure on this.
14:47So by understanding the lived experience of it,
14:50we can try to map places to intervene and strengthen it.
14:54So all of these questions and all of this background
14:57now led to a study of this.
14:58So we wanted to examine these dynamics and perceptions
15:01through financial control, accountability,
15:04and decision-making.
15:06And the idea of focusing on these three within the study
15:09gave us our interview guide,
15:12some structure to follow along with.
15:14And we really wanted to keep it broad
15:16to talk about the partnerships in Liberia.
15:19It wasn't just financial donors
15:21and it wasn't just recipients.
15:22We talked to academics, we talked to NGOs
15:25to really try to get a rich experience of how this operates.
15:30So a few of the procedures.
15:32This here is a picture of some of the research team.
15:36I think we're missing a few people,
15:37but right in the center there is Dr. Talbert Slagle.
15:40And these are some of the students that were there
15:42over the summer representing Yale.
15:46So we obtained IRBs to start from both universities.
15:50So it was grounded in ethics in both Liberia
15:53and here at Yale.
15:55And the participants were recruited
15:56through purposeful sampling and then snowball sampling.
15:59So the purposeful part of it
16:00was reached out directly to by Dr. Dahn.
16:04And so she was able to work within her network as well.
16:08It was so strong and it would kind of have been a huge pass
16:10to not use her assistance with that.
16:14And then once we started the interviews,
16:15there was snowball sampling.
16:16So interviewees were able to recommend different people
16:19throughout the country
16:20that they felt would have powerful insight as well.
16:23So the interviews were in the summer of 2022
16:27between July and August.
16:28Then they were anywhere between 30 and 60 minutes
16:31either on Zoom or in-person in Liberia as possible.
16:34And to ensure kind of quality assurance,
16:40we had a Liberian data collector
16:42and an American data collector
16:44present at each data collection opportunity.
16:47And this was to make sure it was quality data collection,
16:50but also we were using this
16:51as a training opportunity for students.
16:53And so it's really good to be able to hear
16:56what your colleagues are doing
16:57or follow-up questions they might use
16:59or be able to flag a point where they missed the question.
17:01These were all novice data collectors.
17:03So it was great to have the teams.
17:05And then the students all transcribed the data themselves,
17:09which also helped embed them fully in this process
17:12because if anyone's done transcription,
17:15it is not the most fun of a process
17:17and it takes a lot of time.
17:19But they were great.
17:21And we ended up having a total of 38 interviews.
17:26We realized after the fact
17:29that we didn't do great training on note-taking
17:32if anyone declines to be interviewed.
17:34So we did eliminate three of those
17:36just on the idea that it didn't seem like
17:39we really had done adequate training
17:40for the data collectors.
17:42But we were still left with 35 interviews.
17:45We were really trying to have solid female representation
17:48because sometimes in global health, that can be very skewed.
17:50And I'd say 40% we did okay.
17:53And nationality,
17:54we definitely wanted the Liberian experience overall
17:57and what it's like to be working in Liberia.
17:59So that felt good as well.
18:02And then this idea of the classification of position.
18:05Many of these people have been working in this field
18:07for many, many years.
18:09And they had many different roles.
18:10Some had been Liberian government workers.
18:12Some had then switched into NGOs
18:15and some then had even shifted into the role of a donor.
18:18So in the process of reviewing the data,
18:21we did our own classification then
18:23of what was the most prominently discussed role.
18:26So that also forced the deeper reading of the data
18:28to be sure we were capturing that.
18:34So here we are now, we have these 35 interviews
18:37and we were trying to think like,
18:39what is the best way to be analyzing this data?
18:41And we were really guided
18:42by Brown and Clark's thematic analysis,
18:45which is a method for identifying, analyzing
18:48and reporting patterns or themes within data.
18:52And I know I have this as my last bullet point,
18:54but I think I will note it as my first.
18:56Thematic analysis is the most used
18:58but least well-defined method of qualitative analysis.
19:01So it was really important,
19:03especially for me having worked with qualitative data
19:06that I really wanted to document
19:07why we were making the decisions we were making
19:10and why this actually was the best way
19:13to be conducting this analysis.
19:15And again, this is a training opportunity.
19:17So this type of analysis fit well
19:19with being able to train students.
19:21It's a good intro to qualitative analysis.
19:25We could also go from picking a rich description
19:28of the dataset and doing a broad overview of it,
19:31or we were able to dive in with questions.
19:33And at this stage, I'll be honest,
19:34we weren't really sure what we were going to do.
19:37If that we thought we wanted to do a rich description,
19:39the more we engage with the data, it was huge.
19:42So we started asking more specific questions.
19:44And you'll see when we get to the analysis,
19:46we actually kind of did a layered like two-level analysis,
19:49which was really fun.
19:50So I've never done that either.
19:52But just to note that we actually started
19:55with grounded theory.
19:56We thought that's what we were going into.
19:58So just to highlight that this really was
20:00like a team engagement.
20:01We were going back and forth thinking
20:03what theories make sense.
20:04And grounded theory, we felt we
20:06actually already kind of knew
20:08what we were expecting from the data.
20:09We knew what patterns existed in this type of world,
20:13but we were trying to see how they manifested
20:15within Liberia.
20:16So we felt that there was enough of already theoretical,
20:19small T theoretical background
20:21of what we could be expecting with this data.
20:26So just a bit about the structure of the application
20:29of thematic analysis.
20:30It kind of is on a spectrum a little bit,
20:33starting with coding reliability,
20:34which really focused on getting this objective
20:37and unbiased coding.
20:38You're using a codebook that everyone agrees is defined
20:42and strongly used and reflective of the data
20:45and also what you expect to see.
20:47So the goal with that is that every team member
20:50could see an excerpt of data
20:52and know exactly what code that would go into.
20:55I mean, that's if you have perfect inter-rater reliability,
20:57which is a measure of how much you are actually
21:00as a team coding the same data the same way over and over.
21:04So this is great for teams, but it also is pretty rigid
21:08in how you're structuring your codebook.
21:10So you kind of have that set early on.
21:13And then on the other end, all the way on the flexible end,
21:17you are kind of, the idea is that the researcher embraces
21:21where they're sitting with the data.
21:23You're really acknowledging that like you are part
21:25of the tool that is working with this data.
21:28You're having like a big impact
21:30on how the data is shaped as well.
21:32There's open coding and no real structured codebook.
21:36You can have like notes and documentation throughout,
21:39but the goal is that your end result from coding is themes.
21:43It's not really creating a codebook to go back
21:45and reapply this codebook.
21:47So really that's kind of like an individual setting.
21:52And then somewhere in the middle, we have codebook
21:54and codebook uses kind of both of them a little bit.
21:57You're using a structured codebook to assist analysis,
21:59but that's not driving the objectivity of your results.
22:04Accuracy between coders is not the focus of the codebook.
22:07And the codebook can really evolve throughout the process.
22:09Again, with qualitative, as long as you're documenting,
22:12the goal is to really be documenting, documenting,
22:14documenting why you're doing what you're doing
22:17and why it makes sense.
22:18And it's really great for teams.
22:20Again, we were working with a team.
22:21It was a pretty big team by the time we got to coding.
22:24So we needed some kind of structure to assist us.
22:27But again, we were really trying to keep it
22:29as organic as possible when we were moving through the data.
22:33So I'd say we fell kind of about there.
22:35We were within the codebook structure,
22:37but there was a lot of reflexive aspects going on.
22:41I'll talk a bit more about the structure of the team,
22:44but I was reflexive of working
22:47between two different teams on this
22:48and also how the codebook was adapting
22:51and changing throughout.
22:54So there are six steps of thematic analysis.
22:57You start with familiarizing yourself
22:59and generating your initial codes.
23:01And that's kind of a phase
23:02where you're going back and forth a little bit.
23:06But we tended to kind of stay in this cycle for a while here
23:11before we moved on to themes and the other steps in it.
23:15But I'll focus on this for now,
23:17the process that we had five members now of the team coding.
23:20We had two Liberian citizens and two American, and then me.
23:26I was the fifth.
23:27So we started our first meeting.
23:30Both PIs were present as well.
23:32We all read through three interviews
23:34and created an initial codebook
23:37and thought, had many discussions
23:40of how does this codebook pan out?
23:42Does it work? Does it not?
23:43And I think, I know we had at least one two-hour meeting.
23:46I think we had two two-hour meetings
23:48to kind of get through this rough idea
23:49of what a codebook could look like.
23:51And then we came to an idea,
23:52like a consensus on what a draft was.
23:54And then we moved it to code two more interviews.
23:57And this ended up just being the coding team,
24:00the four students and myself.
24:02And then we came back and continued
24:04to refine this codebook and finalize it.
24:07So after our second meeting, then we split into two teams.
24:10I'll walk through that a little bit now.
24:12This is a screenshot of kind of how we organized it.
24:15And I think it'll walk through a little bit of this process.
24:18It says we did 10 weeks.
24:21That was over the course of many months.
24:22We were working with multiple schedules,
24:25multiple commitments, multiple time zones,
24:29and students in very different aspects of their careers.
24:32So we ended up having two meetings.
24:35Once we sat with, you know,
24:37this part of having the initial code team going,
24:41then we split into two teams, one and team two.
24:43I sat on both teams.
24:45So every time each team met, I met as well.
24:48And then we'd meet as a group every Friday
24:50to then go over the codebook
24:52and reassess and decide, you know,
24:54what changes are we making?
24:56And you can kind of see at the top,
24:57we had the deadline for when we wanted to complete it.
25:01And then in blue, it was when we actually completed it
25:04because some meetings took a lot longer.
25:06And then we highlighted which codebook we would be using
25:09because we were using many iterations of this codebook
25:12as it was defined throughout.
25:14And I'll do a quick screen share then
25:17to just highlight the evolution of the codebook.
25:20This is from the end of September last year,
25:22one of our first codes.
25:24You can see that we started with what we would have
25:27as like our parent code, how we defined it,
25:30how we had child codes.
25:32I'll just focus on accountability right now.
25:34You can see that we broke that down into many different,
25:38you know, D meant donor, R meant recipient.
25:41At this point, we were still classifying our interviews
25:43as donor and recipient, which then the more we read them,
25:47that's where we started to get, you know,
25:49some donors are, or some recipients
25:52are actually donors themselves.
25:55And, you know, it was a really,
25:56we had to actually classify the interviews then
25:59and it made it a lot easier.
26:00But you can see that this is like the first iteration.
26:02You can see notes, you can see changes.
26:05And then finally to the last one.
26:08So this was really just a screenshot
26:10of how, you know, accountability evolved.
26:13Then we, once we had realized we could classify people
26:16as to what position they were speaking about the most,
26:20it just came down to, was there a form of accountability
26:23or was there a lack of accountability?
26:24So then we could go into how are NGOs
26:27talking about forms of accountability?
26:29How are donors, foreign donors
26:32talking about forms of accountability?
26:34They aren't, if you want to know, in their own form.
26:38So it became a lot easier to figure out
26:41how to start organizing and structuring the data.
26:44Oh, I just saw the note that every time I turn my head,
26:48I'm muscling sound.
26:50So I will keep focused on the computer from now on.
26:54So yeah, the codebook, as you can also see
26:57at the bottom of the screen here,
26:58we went through so many iterations and we dated them
27:01so that as a team, we could go back to the shared document
27:04and know which is the most relevant codebook to be using.
27:12So here we are now coming back together.
27:14We're searching for themes.
27:15So imagine this is now,
27:17we've got through all 11 weeks of those meetings,
27:21which were 30 some meetings on my end
27:24of like getting these teams together.
27:26And we're starting to talk about themes.
27:27So all five of us met and discussed common themes
27:30and talked about our notes on what we saw
27:32kind of coming out of the data.
27:33And we started grouping codes based on where we thought
27:36codes were fitting into give development to these themes.
27:42And then from here, we did discuss as a big group,
27:46but then really only me and one of the team members
27:49continued to start actually writing our results up
27:53and refining and defining and naming the themes.
27:58And I'll be honest at this stage, it was just,
28:01we were doing a lot of wrangling with a lot of people.
28:04So it kind of made sense to just start focusing
28:06a little bit more on how to move forward
28:08now the data analysis part of it.
28:09And it was great.
28:11People, I think within that group,
28:13I'd say people were graduating,
28:15some were working on other projects.
28:18So it was never like people didn't want to be involved.
28:21They were always willing to give feedback
28:23and step in as needed.
28:24But this one student really had an aptitude for coding
28:28as well for qualitative analysis.
28:30And in the vein of being reflexive,
28:35I'd say maybe she and I worked well together.
28:37Maybe we approached the data the same,
28:39but at the same time, I was just,
28:41she was a student who had never taken qualitative before,
28:44but she was willing to just really discuss.
28:46And she was never afraid to kind of get knee deep in it
28:49and really talk through what definition she was seeing
28:52or to push back against something
28:54I or someone else was saying,
28:55or to be quick to jump in and give examples
28:59to bolster other people's feedback.
29:01So she just got it.
29:03So I'd say from there, it became her and I really working on
29:07how to move forward with this.
29:09So I'll present now some of the results that we saw.
29:14Again, a lot of this is focused on the analysis,
29:17but we do have results that I think were really powerful.
29:21We have two phases of the analysis.
29:23And after I go through some of the results,
29:25I'd like to talk a bit about just what it took
29:27to get to the images that we used,
29:30because I think also in qualitative work,
29:32it is so powerful to have images
29:33that can represent what you're trying to say.
29:36And it's kind of a quick takeaway for people as well.
29:40So starting here, we found this cycle of,
29:43it's pretty clear, whoever had control of the money
29:45was the person making the calls
29:48for how they were going to prioritize using that money.
29:51And then a lot of times, so we'll talk actually first a bit
29:55about just that pattern right there,
29:57who has the money and how that gets to be
29:59in that priority setting power.
30:01So we started this analysis really diving first
30:04into priority setting power.
30:06So we wanted to see kind of how that manifested
30:09within the system, but how that was being pushed along.
30:11So that's why we focus now a bit
30:13on what financial control really means.
30:17So the theme we discussed was that priority setting power
30:21is most strongly tied to whoever has the financial control.
30:24Not surprising, again,
30:26this was something we kind of expected to see.
30:29There was a strong hesitancy among those we interviewed
30:32to say no to donors.
30:33There was kind of a pressure that if you say no,
30:35you're going to damage a relationship
30:37to have funding for a long time.
30:39So it was tedious.
30:41And that also there was a feeling
30:42that these gestures of collaboration
30:44were not actually genuine.
30:45They were kind of just checking boxes.
30:47And it was kind of obvious to definitely
30:51on the side of being a recipient,
30:52but sometimes donors are mentioning that as well,
30:55that they were doing this because it was expected.
30:58And I'd like to read this quote.
30:59I think it's a great quote that sums up
31:01being engaged in this type of work right now.
31:05Most of the time,
31:06the international partners come with what they want to do.
31:09So they prioritize what they want to do
31:11in a particular environment or in a particular area.
31:15And in many instances, you either accept it
31:18or it is pushed down your throat.
31:21It is a bad process because when you are building a system,
31:24if you want to help,
31:25it is good to work with the people that you want to help
31:29to tell you what their priorities are.
31:31And you can work around it so that at the end of the day,
31:34they have an ownership to it.
31:36But if you decide on what you want to do for the recipient,
31:39there is no ownership.
31:41So it's just thrown on them.
31:43And I think that that push of it being,
31:47that notion of it being pushed down your throat,
31:49like this quote just stood out immediately
31:51that it was, you know, people are very graphically honest
31:57that they have no say in how this is being negotiated.
32:03So then now we'll focus a bit on the other side.
32:06You'll see that I like to really demarcate
32:09where I'm functioning in these images as well.
32:12But now there's this idea.
32:14So you set your priorities,
32:15you have how you want this work to get done,
32:18but then metrics are being consistently set
32:22to meet those priorities.
32:24So if you're meeting those priorities,
32:26you hit those metrics, the funders are seeing this
32:30and they're funding the same type of cycle.
32:32You know, that's just innate nature.
32:34It's a feedback loop,
32:35a reinforcing feedback loop of that exact process happening.
32:39So these implementation plans are driven by donor priorities
32:42with outputs that reflect these
32:44donors' measures and metrics.
32:47Sometimes they're not reflecting
32:48what the country members really even want to be seeing.
32:52So there's a stronger obligation back to home country
32:55than to the Liberian government.
32:57A lot of times we did hear quotes on taxpayer dollars
33:01and who needs to be having reports back
33:04on what's going on with their money abroad.
33:08And donors generally in this sense
33:09focused on short-term metrics.
33:11I'll talk about this a bit in the quote,
33:13but the Liberian government really wanted
33:15to see longer term sustainability interventions.
33:17And it's really hard to factor that in
33:19when you have to be working under someone else's priority.
33:23So then the idea was that these successful outcomes
33:25lead to successful funding,
33:26and that funding leads to, again,
33:27like I said, a reinforcing loop.
33:29So for this quote, it's not just priorities,
33:33but even the results are like,
33:35who decides what success looks like?
33:38Part of the problem is the way
33:39that the funders define success.
33:42The way that a lot of donor funded projects are measured,
33:44their success is measured by performance-based indicators.
33:48And I think, you know, an important measure of success
33:51is contribution towards building systems
33:53and a lot of long-term things that are difficult to measure
33:56within funding cycle of even five years.
33:59That also distorts how things are planned
34:02and how success is defined, and that's a cycle.
34:06I mean, this quote really just summarizes
34:08everything I showed within that cycle.
34:10And there were a few people who also spoke about this idea
34:13that when you come into a lot of these big
34:15international donor organizations, or even some of the NGOs,
34:19you're coming in on a five-year timeframe.
34:22And so what you wanna achieve in that five-year timeframe
34:24for your specific role
34:26is what's gonna get you to your next role.
34:28So people are naturally looking out for themselves too
34:31in their own career, but at this point,
34:33it's not creating an allegiance to building a bigger system
34:35that exists beyond them when they're there.
34:42So reinforcing here is the first phase of the analysis.
34:45We created this loop.
34:46We were left thinking though,
34:47like, why is this loop still existing?
34:50Is there a way that we can understand
34:51what's going on underneath it?
34:53So this is what we call the cyclical process
34:56of priority setting.
34:57It keeps operating in this cycle.
34:59So our goal was to then rotate it
35:02and think how can we discern what's going on underneath it?
35:06So we came up with this type of image.
35:09What's going on that's driving these types of patterns?
35:12And we wanted to really understand the underlying factors
35:14that are influencing this process.
35:16When we wanted to create it,
35:17it's kind of like a vortex in a way,
35:20or the idea of the iceberg model,
35:22where what you see on the surface,
35:23there's a lot more going underneath.
35:25And it's kind of the power inherently of qualitative work.
35:28But at the same time,
35:29we wanted to show that within this data,
35:32three major themes kind of emerged,
35:35that there was a history of prior engagement,
35:37a level of transparency and patterns of accountability.
35:39And some of these were reflective
35:42of what we asked in our guide, our interview guide.
35:45Some had emerged a bit more organically
35:47in how interviewees were talking about these topics.
35:51But we did want to note that there's likely more
35:53that exists underneath this as well, driving this pattern.
35:56But this is what we saw within this data.
36:01So the history of a prior engagement
36:03is referring to the ways that prior engagement in Liberia
36:06forms current collaboration.
36:08So as I had said, giving the background of Liberia,
36:11there was civil conflicts
36:13that kind of set the stage on reliance.
36:15Then there was Ebola.
36:17So it created this state of emergency and crisis
36:19that many donors continued to collaborate in this way.
36:24There's kind of this mentality of get in, get work done,
36:27and less of a focus
36:29on the long-term sustainability of a system.
36:32Donors did have a continued fear of corruption
36:34and mismanaged funds.
36:35And many Liberians were not ignoring that.
36:38They were not saying that that wasn't true,
36:40but that was perpetuating a fear
36:42of changing types of investment and ways to engage.
36:46And this wasn't just seen,
36:49I would assume this isn't only seen in Liberia.
36:52I think we also know from other studies and reports
36:55that external partners often operate
36:57in low resource settings
36:58in unethically, unethically questionable ways.
37:02They might not do in their home country
37:03or different types of settings.
37:05And that was definitely evident in Liberia
37:07in the way that some people spoke about this.
37:10So this quote summarizing would be,
37:12they, the donors, resist channeling their money
37:15to a recipient country government
37:17until it's running as well as their own.
37:20Like at that point, they don't need your money.
37:23The country's already running really well.
37:25So it's, if you want to wait for the country to be perfect
37:28before you take any risk,
37:30what that does, a preoccupation with fiduciary risk
37:33ends up becoming a strategic risk
37:35that your programs can fail.
37:37You're not helping the country
37:39to achieve its development goals and progress.
37:41So this idea of there's a constant risk state
37:45is not allowing there to be many different ways
37:47to consider how to fund or support or collaborate.
37:52Next, you'll remember in that diagram,
37:54I talked about that the donor
37:56is determining how transparent they are willing to be.
37:58And this was really specific to funds
37:59because that was a lot of what we were asking.
38:02But a lot of times, and a lot of us are in academia here,
38:06I'm assuming it's not all of us,
38:07indirect costs are not totally transparent,
38:10but especially members of the Liberian government
38:13are saying, we don't know how much money
38:14is coming through our country now.
38:15We don't know what the overhead costs are for NGOs.
38:18We don't know this.
38:20So sometimes, when a donor is filtering money
38:23through either NGOs or supporting partners,
38:27it's not clear what money is actually getting into Liberia.
38:31And this quote, I think sums that up perfectly
38:33where it's an imbalance of maybe
38:34what the public perception is of a huge investment,
38:38but then what really is happening in countries
38:40is not the same amount.
38:41So one thing that was of concern was the NGOs,
38:44not the donors, because the donors will channel funding
38:47through the international NGOs.
38:49The concern was that they will come
38:51and give the narrative of what needs to be done,
38:53but the actual financial display was kept secret.
38:56And I always told them, we know that you write proposals.
38:58We know that a certain percentage of money
39:01is there for your administrative costs.
39:03We know, but we also wanna know
39:05how much is there for service delivery.
39:07You cannot get up and tell Liberia,
39:09oh, we channel $2 million through external organization,
39:13and then you come and you use 50% of that money
39:15on administrative costs.
39:16It is wrong.
39:18There were a couple accounts too of different partners,
39:22either Liberians or based in Liberia
39:24or strong collaborations in Liberia
39:26saying that there is an inequity of how resources are used.
39:29Liberia doesn't even have stable electricity
39:31and fuel costs to run an office are astronomical,
39:35but sometimes there's rules of what would be funded,
39:38what wouldn't be funded
39:39when it's not in that organization's office.
39:42So it was really tricky to navigate
39:44some of the transparency on what was going on.
39:47And finally, at the bottom of this vortex I had shown,
39:51the donor is creating structures
39:53that are holding the recipient accountable
39:55by often overlooking their own accountability
39:57back to the Liberian government.
39:59So the government,
40:01we didn't ever explicitly ask about audits.
40:04We asked about accountability and auditing came up
40:07by almost every single Liberian government representative.
40:10And they were really proud of it
40:12because they said we would meet
40:13all of our audits in the end.
40:14We were hitting these milestones,
40:16but also there were reports of being audited
40:18like 17 times during a project or something.
40:21So it's continuous, continuous monitoring
40:24and reports being written.
40:25And that was never happening on the donor side.
40:28There were not audits on what the donors were doing
40:30or not audits on how, again,
40:32tying back to the financial aspect
40:34on where they were spending their money.
40:36So it really was, as Dr. Dunn had alluded to,
40:39a one-way street.
40:40And it was really hard for anyone
40:42other than who had that financial control to negotiate that.
40:46So we thought it was also interesting,
40:48and this was noted when we talked about the bigger circle,
40:52the original phase one of the analysis,
40:55that donors are really responsible
40:56only to report to their home countries.
40:59So you're seeing now that there's this discrepancy
41:01of kind of where there is kind of what you're owing
41:05when you're working in another setting.
41:08So again, there aren't really any systems for accountability
41:11in the other direction.
41:12That is, for a funder to even be really questioned
41:15for maybe withdrawing funding or changing their priorities
41:18or not providing the amount of funding originally promised
41:21or any number of things,
41:23the decisions they make about what can be paid for
41:25and what can't,
41:27the extent that they're really being held accountable,
41:29that's internal itself.
41:31They're self-regulated, really.
41:33So if there's no structure
41:34of who these donors are reporting to,
41:35remember back that document that I had representing
41:38how the Liberian government is trying to work
41:40with all these red arrows kind of shooting around it.
41:44There's no way to kind of make sense of that
41:46if there's no responsibility to report
41:48back to the government of what's going on.
41:52So I'd like to just reflect and really give,
41:57I know Christina Talbert-Slagle is on this call.
42:00We spent so much time trying to get to these images.
42:04I think it's also really important to talk about that
42:07in the process of qualitative work,
42:09because again, it's really words,
42:11but images can be powerful.
42:12So we went from something like this,
42:16trying to map out what these themes and codes
42:19are looking like and where they're fitting in.
42:21This looks like something out of like an electric box
42:23or something,
42:24where we finally then shift into this cyclical pattern.
42:27But then thinking of where does collaboration fit in?
42:30Is it within the priority setting
42:32to the implementation of the plan,
42:34or is it part of this cycle
42:36rather than where we kind of ended with it being
42:38underneath and embedded by supporting this cycle?
42:43We kind of went back to the square again,
42:46thinking about a lot of definitions
42:47of what is legacy of engagement?
42:51Who's being held accountable again?
42:54And then finally, we even did a causal loop diagram.
42:57So we really were going all over with trying to get to this,
43:01but as you can see,
43:02there's kind of this building to get to this point
43:04of it actually being a cyclical pattern
43:08that we can kind of start to dig underneath of.
43:12So then finally producing the final report.
43:16I'm excited that we're pretty much about to submit it.
43:20As I'm reading some of these quotes,
43:22I'm seeing spots where we had to clean them up
43:23a little bit too, but the report I think is great.
43:27I'm really excited.
43:28It's something that we've worked on for a long time
43:30and going through these methods
43:31are showing like how strategically and thoughtfully
43:34we thought about every step of the way
43:35in how to represent this data.
43:40And just some, you know,
43:41this ended up being embedded throughout,
43:43but just some highlights of using this
43:45as a training opportunity.
43:47The size of the team was big.
43:50And I think in a setting other than academia and training,
43:54it would have been like a little bit daunting
43:56to try to have a timeframe to do this,
43:58but it really allowed for a hands-on experience
44:01to learn these qualitative methods.
44:02And I think for students, that's really invaluable.
44:05So it would be kind of cool to think about ways
44:07that we can kind of implement something
44:10like training on the go with students
44:11in a way that is also standing true to the data
44:15and moving along projects.
44:16I really had fun being reflexive on my own aspect
44:21of working with this data,
44:22as well as trying to maintain reflexive
44:24of the process of working with multiple teams
44:26and editing a codebook and moving that part along.
44:30It was kind of unexpected that I would end up in that role.
44:33I don't think any of us knew that
44:34that would end up being my role,
44:35but I really enjoyed it and it was fun.
44:39And then also time is key.
44:41You can see this was 40 minutes of just talking through
44:43like some of the different meetings and discussions
44:46and considerations we had to make.
44:48But if we really wanted these methods to make sense,
44:51especially in qualitative,
44:52where I think that some of the rigor is always challenged,
44:55like we put so much time and effort
44:57into making sure this was sound.
44:59We wanted to make sure the codebook was inclusive.
45:01We wanted to make sure the approach made sense.
45:03So that takes a lot of time.
45:06And I loved the idea of thematic analysis
45:08where you don't have to do everything all at once.
45:11We did code the whole project at once,
45:14but then we started to realize,
45:15no, we wanna ask questions to really get further
45:18into these codes and develop the themes that way.
45:20And it felt a lot more manageable
45:22and a lot more interesting personally.
45:25And then also the idea that we didn't expect anything wild,
45:31like we kind of knew what we would see with this data,
45:33but I think with qualitative, it offers great insight
45:36and it offers great insight to,
45:38are these systems working the way
45:39that we think that they're working?
45:41And the human experience oftentimes might tell you not.
45:45So finally, I'd like to just thank
45:49Dr. Kristina Talbert-Slagle.
45:50She was the US Co-PI and Dr. Bernice Don
45:52who was our Liberian Co-PI.
45:55These are the four students and master's students,
45:58undergrad and recently graduated master's students
46:02who worked with us, Defne, Joseph, Antoinette and Johannah.
46:06And then again, there was a lot of support from CTLI,
46:10which is the Center for Teaching, Learning and Innovation
46:12at ULCHS, which is University of Liberia
46:16College of Health Sciences.
46:17It's a great team and supportive network to be.
46:19And overall, this was under BRIDGE-U Liberia,
46:22which is a USAID grant between here
46:25and the University of Liberia.
46:26So I think thank everyone for having me to do this seminar
46:30and please reach out with any questions or thoughts.
46:32And I will let you know the results of the impending paper.
46:37I'll take questions now or feedback, comments.
46:40I'm excited to engage.
46:46(attendee speaks faintly)
46:51<v Attendee>It looks like we entered</v>
46:54your toning and analysis process like remarkably fast.
46:59And I mean, I think that's because we're doing
47:01a rapid qualitative analysis in order to get results
47:05quickly from qualitative studies,
47:08especially in information science
47:10where we need the information to understand
47:13and change or tweak interventions.
47:17But on the tip of what you were doing this time,
47:19rapid analysis, but yet you sound like you're analyzing
47:24very large amount of data.
47:26It looks like it's two months period.
47:30<v Brigid>Oh, let me change that.</v>
47:34I'm gonna switch to.
47:36Is this a bit better to hear?
47:38We're engaging as a room?
47:41<v Attendee>One thing you could do</v>
47:42is just repeat the question.
47:43<v Brigid>Okay, yeah, I'll just repeat.</v>
47:44All right, let me get back to it, to my computer.
47:50I'll repeat the question here.
47:51So it was noted that it seems like the analysis
47:54was rather quick and I should have emphasized
47:57this a bit more that it wasn't.
47:59It took a long time.
48:02The timeline.
48:03So we did, I'd say we started with our analysis.
48:06It was, we had 10 weeks of meetings.
48:09So probably shouldn't be labeled as weeks.
48:11It should be labeled as meetings
48:12because some of the meetings took two weeks
48:15of us actually meeting to talk about that
48:18and get to the final results.
48:20And again, it was some of the weeks then
48:22if we had team one meeting and team two meeting,
48:25I would meet with both of those teams individually.
48:27And then we had to find a time
48:29for all of us to meet as a group.
48:32Sometimes that took a couple of weeks to implement as well.
48:35I think that if I could, if there were a need
48:40to have a rapid assessment or a rapid qualitative,
48:43something I would probably go back and do
48:44would be do a really initial, intense, deep dive
48:49into the data and then formulate questions
48:52worth kind of probing throughout the data with.
48:56But with this process, we kind of allowed that
48:58to emerge more of what the data was showing us
49:01could be different avenues to explore.
49:03So it really was exploratory, I would say.
49:07But we definitely, we had our first meeting
49:09with the data at the end of September
49:11and we wrapped up our final coding
49:12at the end of February, beginning of March.
49:16And from there, then we started diving
49:18into figuring out what our themes were.
49:22So it was a process.
49:28<v Attendee>In that early process,</v>
49:30you mentioned having the interviewers
49:33who did the transcripts then from recording,
49:36except the ones that rejected
49:37recording, so they were left out.
49:40What are your thoughts on using software
49:42to create initial transcripts and then do the coding?
49:46<v Brigid>I mean, that's how</v>
49:47I've always kind of worked previously.
49:51I worked in a mixed methods research lab
49:53and mostly my role was doing qualitative analysis
49:55for the quantitative ends of projects,
49:58but I didn't repeat the question, here I go.
50:02The question was, what are my thoughts on using
50:08like transcribing assistive technology
50:11and then going back and recleaning the data?
50:14And my response to that was, I think that's great.
50:17Really, I've used that before in other settings.
50:20It's really difficult in Liberia
50:22because of Liberian English is just hard
50:26to transcribe from settings.
50:28So you really have to listen.
50:31You can try using that assistive technology.
50:34I would have assumed that it would be not worth using.
50:39I know that a few of the students tried
50:41and it just did not work and they had to listen
50:43and do it on their own.
50:45But in previous studies, I've done that
50:47where you get the initial transcript
50:49and then it's easy to just listen to it
50:51and look at the words and do the editing,
50:53stopping the recording as it goes.
50:56<v Attendee>Yeah, I agree with that as well.</v>
50:57It's just a standard American tradition.
51:01Yeah, but anything helps.
51:05<v Brigid>Hey, there's a market.</v>
51:07I'm sure somebody wants to start thinking about that.
51:10I'll go quickly to a question we have online.
51:15How can we have a look at the final report
51:17and will it be possible or I'll say ethical
51:20to reproduce this analysis in other countries
51:22based on their work?
51:24This person works in Nigeria and the topic is so familiar.
51:27I definitely will be able to look at the final report.
51:31We're submitting it to journals,
51:33like I wanna say Monday or Tuesday.
51:37Kristina's probably laughing on this call
51:39as soon as possible,
51:40but definitely would love to disseminate that.
51:43And I think it's absolutely possible to reproduce elsewhere.
51:47I mean, and I would love to talk about how to do that.
51:50I don't think the results are gonna be reproduced,
51:52but this process,
51:53and we could have more feedback on refining this process,
51:56but how do you kind of dig below
51:58to understand what these collaborations look like?
52:00I think, again, this is my bias.
52:02I think this could be done everywhere
52:04that these collaborations are happening.
52:06And I think it would really give light
52:08to how to go forward to structure them a bit more equitably.
52:13Questions in the room?
52:16(attendee speaks faintly)
52:21So the question was, aside from journals,
52:23what are we hoping to do with the results
52:26outside of submitting them to journals?
52:29That is a great question.
52:31And that's something that I think,
52:33especially thinking about peer-reviewed journals
52:36that also comes with a specific audience
52:39who's reading those.
52:41I did present this at a conference already.
52:44That was the original causal loop,
52:46one of the green loop diagrams.
52:48And again, that's also a very specific audience.
52:51But the goal would be to, I think,
52:53try to write even commentaries
52:55or something that's gonna start getting attention
52:57that this is worth being investigated in a different way.
53:00There's different documents, like USAID and WHO,
53:04reports about what collaborative partnerships look like.
53:07And if we could get attention of that,
53:10if you have ideas too, I'd love to hear.
53:12But I think getting that attention of like,
53:14how can we make this routine to monitor as well?
53:18<v Attendee>So I just want to follow up on</v>
53:19some stuff that you guys have.
53:23So, and now, what was the experience of donors
53:27for this type of action?
53:29What was the push of power on the donors?
53:33And I was wondering if you were finding
53:36a secondary search that you were looking for on that,
53:41like from the risk-taking actionable things
53:44that you were looking for,
53:46or were there other things that happened?
53:48<v Brigid>So the question was that a lot of the themes</v>
53:50were donor-centric and things that the donors
53:52could be doing, but are there ways that we could see
53:55what the recipients could be doing
53:57to change this pattern as well?
53:59I think that could be,
54:02the way I would envision that for follow-up collection,
54:05I don't think we could see that clearly
54:06in this data as much.
54:08There's definitely room for it.
54:09I think people would have so much feedback.
54:11I think it would be cool to structure workshops around that.
54:14Like, what would that look like to implement change
54:17and how, I mean, another facet of my research
54:20is I like future studies and foresight strategic planning,
54:23and I'm doing that with a project in Stellenbosch right now,
54:26and thinking their whole focus
54:29is what does the future of development research look like?
54:32And how can you kind of plan that
54:34and set the stage for that?
54:36And I think that's like, that type of futures planning
54:38is like, all right, we have an idea of where we wanna be,
54:40and how can we work backwards to get there?
54:42I think that that's gonna be something
54:44that needs to be a collaborative meeting.
54:47I mean, regardless, we can't just say,
54:50donors, you're kicked out.
54:51Like, that would be ideal, but in reality,
54:53that's probably not going to happen.
54:54So how to have this be a collaborative meeting
54:57where there is that agency to really put forward ideas
55:01that are centered on what Liberians think need to happen,
55:05I think could be awesome.
55:08<v Attendee>I mean, I think, so the answer to this question</v>
55:13was to ask what role people thought could be done
55:18to improve equity and positive impact research.
55:22So it seems like it's an out-of-touch question.
55:26But the other question was,
55:29it seems to me that people in this board of controllers
55:33clearly have a different idea of government people,
55:40Liberian government people
55:43have a different perspective than Liberians.
55:46So, I mean, I'm thinking of different elements,
55:50but what are the differences in perspective
55:54that you've noticed? (speaks faintly)
56:03And I think that's a big thing to kind of have.
56:07<v Brigid>Yeah, so the second part of the question,</v>
56:10I'm not gonna look.
56:11Yeah, the second part of the question had to do with it.
56:14But it was looking at the different perspectives.
56:17Like we had these, the groups classified as,
56:20Liberian academic, Liberian government donor and NGOs.
56:25And it's kind of like understanding
56:26the nuance of perspectives between those.
56:29And we did try to do that.
56:30And there definitely was,
56:31especially between Liberian academic and government,
56:36because Liberian universities are so tied to the government
56:39to its government funding, they have very similar views too.
56:42And I mean, a quote that's kind of, yeah.
56:46But a quote that's even standing out to me
56:48was one of the academics saying like,
56:51you can sit down with people
56:52and you show up with your list of like 12 things
56:55that you really want done.
56:56And you're hoping that maybe four of them get funded.
56:59And with whatever gets funded,
57:00you cobble together what you need for your program
57:02because you need that money.
57:04And that's frustrating, right?
57:06Like that on that list of 12,
57:09maybe number two, number eight, number four,
57:13and 12 get funded.
57:14And you're like, well, how do you structure?
57:16We have this whole plan of a program.
57:18How are we structuring those four specific things?
57:22And then the first question I believe was like,
57:25what could be done?
57:25Did we ask people?
57:27We kind of did.
57:29How could you envision like these collaborations
57:31be more equitable?
57:32And it was pretty standard answers
57:33that were reflecting what the issues were,
57:36stronger systems of accountability or stronger reporting.
57:39And if we were to do this again,
57:41I would say like, dig into that question,
57:43like really dig further and have more.
57:46And again, this is,
57:48we were working with novice data collectors
57:50who did a great job,
57:51but even it highlights like training for people
57:55how to probe,
57:56training on how to circle back to a previous question.
57:59It's like kind of hard.
58:01So I think that could be a next,
58:03a really great next study as well.
58:09<v Ashley>So Brigid, I just want to note</v>
58:11that it's one o'clock.
58:13And so I think several folks will probably need to hop off,
58:19but I just want to thank you so much
58:22on behalf of CMIPS and others that for this talk,
58:29it was really phenomenal.
58:30Thank you for digging into so much of the analytic process.
58:34I had a question in the chat
58:35that maybe we could chat about later.
58:37You can kind of talk about later,
58:39but it seems like there's a lot of implications
58:41for new NIH requirements and policies,
58:44particularly the one that requires all foreign projects
58:48and foreign agencies to submit their lab, like lab notes,
58:54which is just like a phenomenal amount of work in general.
58:57So I think it's all wrapped up
59:00in everything you're talking about,
59:02about trust and accountability and all of those things.
59:06So anyway, thank you so much.
59:10And I'll say goodbye to everyone here and take care.
59:15Thanks everyone.
59:16<v Brigid>I had a lot of fun chatting about this.</v>
59:24Okay.