Using Qualitative Thematic Analysis to Explore Financial Donor and Recipient Collaboration in Liberia

Transcript

• 00:00<v Ashley>Okay, it's two after,</v>
• 00:02so I think we'll go ahead and get started.
• 00:08My name is Ashley Hagaman.
• 00:11I'm a faculty member
• 00:13in our Social and Behavioral Sciences Department
• 00:17and the Director of the Qualitative Methods
• 00:19Innovation Program within our Center for Methods
• 00:22and Implementation and Prevention Science here at Yale.
• 00:26And we are delighted to welcome Dr. Cakouros
• 00:34as our seminar speaker today.
• 00:39She is a global health scientist and mixed methodologist
• 00:43with expertise in health systems.
• 00:46I met Bridget a couple of different ways.
• 00:49One was through one of our
• 00:51Master's in Public Health students
• 00:53and another was an email just asking
• 00:57about access to qualitative methods training
• 01:00and methods just like more generally here at Yale.
• 01:04And then of course our shared interest in global health.
• 01:06So we had been in some common spaces together
• 01:10and I was just immediately impressed
• 01:12with her wealth of experience
• 01:14working on complex systems studies
• 01:18and in sort of implementing programs around the world.
• 01:22She has a really interesting and really important focus
• 01:25on developing and development research more broadly.
• 01:30Global health research and global health development
• 01:33are topics that I engage with every day
• 01:35as a part of my own career.
• 01:37And my teams and I often think about the larger dynamics
• 01:41that we have with our composition
• 01:43and our positionality and relationality
• 01:46both within our team and our project is being implemented.
• 01:49But I hadn't thought about how we understand that
• 01:55or what we do with that and kind of what we can learn
• 01:58about these dynamics more broadly.
• 02:03And so Dr. Cakouros' work is so important
• 02:06because it will help us not only do better
• 02:09global health work and health systems work
• 02:11but it'll help global health programs be more successful
• 02:15which is really all health programs.
• 02:16And so at Yale, she is a postdoctoral fellow
• 02:20with Professor Talbert-Slagle, the Associate Director
• 02:23of our Yale Institute for Global Health.
• 02:25And today she's going to talk about qualitative methods
• 02:29to study global health collaboration
• 02:32anchoring in her collaborative work in Liberia.
• 02:36And so thank you so much for joining us Dr. Cakouros
• 02:40and I'll hand it over to you.
• 02:42And then just a note, I can kind of monitor the chat as well
• 02:46in case there were any questions
• 02:47but let us know if you want to take questions
• 02:49in the middle or if you just want to wait till the end.
• 03:04(Brigid speaks faintly)
• 03:11(audio distortion)
• 03:35<v Brigid>I'm just going to go to my camera now.</v>
• 03:37Here's my computer, there you're welcome.
• 03:40<v Ashley>Hey Brigid, just to stop you for a second.</v>
• 03:43It sounds very muffled on our end.
• 03:45<v Brigid>How about now?</v>
• 03:46<v Ashley>Ooh, that's great.</v>
• 03:47<v Brigid>Okay, we switch it to my computer.</v>
• 03:49So when we start maybe in chat as a group.
• 03:53<v Ashley>Okay, thank you.</v>
• 03:55<v Brigid>Perfect.</v>
• 03:56All right.
• 03:58So to summarize what I was saying,
• 03:59I'm just really excited to talk about this project.
• 04:02I've been working on it for about a year now,
• 04:04a little over a year.
• 04:06And I'm really excited about qualitative methodology
• 04:10and especially in global health
• 04:11and what you can learn about systems
• 04:13and how systems operate using qualitative methods.
• 04:17It's kind of a fun frontier to be working on
• 04:20and I'm looking forward to sharing this project with you.
• 04:23So this is titled using qualitative thematic analysis
• 04:26to explore financial donor
• 04:28and recipient collaboration in Liberia.
• 04:32And a little overview, there we go,
• 04:37for what I'll be talking about today.
• 04:39I'll talk a little bit about why Liberia,
• 04:41why it makes sense to be thinking
• 04:43about collaborations in Liberia, kind of what sets the stage
• 04:46for the importance of studying this topic there.
• 04:48And then I'll talk about the study design and methodology,
• 04:52why we chose the methods we did,
• 04:53the processes we went through
• 04:55for analyzing our qualitative data and such.
• 04:58And then I'll touch briefly at the end,
• 05:01summarizing some of the lessons we learned.
• 05:02This was really a training opportunity
• 05:04with some of our students.
• 05:06So I talk about that kind of throughout
• 05:08this presentation as well,
• 05:10but I think it's kind of cool to reflect on that as we go.
• 05:13And there's a couple of pictures throughout here of just,
• 05:16they'll be of the team or pictures I took in Liberia.
• 05:19So they're really to just enhance this visually
• 05:22because qualitative sometimes is just words,
• 05:24but giving a little bit of context of where we were.
• 05:28So why Liberia?
• 05:30I'll give a little brief background on it.
• 05:32And I wanna highlight that what I'm gonna focus on
• 05:35is really the reasons why the health system there is broken.
• 05:38So it's focusing on some of the tougher times
• 05:42and the harder hit parts of the health system.
• 05:46Liberia is great.
• 05:47There's a lot to offer from this country.
• 05:49So I'm not trying to just ground Liberia
• 05:52and being like this place of civil conflict and Ebola,
• 05:56but that's really what's weakened the health system
• 05:58to make this type of study.
• 06:01So between 1989 and 2003, I'm sure that people remember
• 06:06that there was civil conflict there.
• 06:08It was more by child soldiers and extreme poverty and fear.
• 06:1390% of the skilled health workers fled the country.
• 06:17So the health infrastructure was destroyed.
• 06:19And this really started a strong reliance
• 06:21on humanitarian organizations and international NGOs
• 06:25and foreign donors to kind of keep this system afloat.
• 06:29So then there was a brief period
• 06:31of health system rebuilding from 2006 until 2013.
• 06:36This is a picture of Ellen Johnson Sirleaf,
• 06:39the first female elected president
• 06:41on the continent of Africa.
• 06:43Very exciting.
• 06:44And she really, when she was sworn into power,
• 06:46she not only negotiated down and canceled a lot of debt,
• 06:49but she was able to renegotiate some investments
• 06:52from global donors.
• 06:52So she was really working on rebuilding
• 06:54a lot of these partnerships.
• 06:56And this chart here just shows the change
• 06:59in under five mortality rate,
• 07:01which is a strong indicator of a health system.
• 07:04So Liberia had started to turn the corner
• 07:07right around 2010, a little bit earlier in the early 2000s
• 07:13from when it was in civil conflict.
• 07:16And access to health facilities increased to 71% by 2013.
• 07:21In 2008, only 41% of people were within one hour walk
• 07:26of a health facility.
• 07:27And by 2013, it was up to 71%.
• 07:29So these are huge gains in a short amount of time.
• 07:33But then there was Ebola,
• 07:35which also many public health people probably remember.
• 07:39There was just over 5,000 or just under 5,000 deaths,
• 07:42including 8% of the skilled healthcare providers,
• 07:46which is the doctors, nurses, midwives.
• 07:49So routine healthcare service essentially collapsed
• 07:53once again, which was really unfortunate
• 07:56because as I noted in that period of rebuilding,
• 07:59they were really on a strong chart to go forward.
• 08:03So now what?
• 08:04You know, this timeline not only increased
• 08:09reliance on donors, it brought in a lot of donors.
• 08:12So many donors started engaging in Liberia
• 08:14with different aspects of strengthening
• 08:17the global health system.
• 08:18There was funding coming in from many partners.
• 08:22And again, you know, there's many more people
• 08:26that are just even highlighted on this slide.
• 08:29So in 2015, Liberia created this investment plan
• 08:34for building a resilient health system
• 08:36because clearly the system's taken some hits
• 08:38and it's how do we build up, make it even stronger?
• 08:40So again, this is just some examples
• 08:43of the numerous partners working in Liberia.
• 08:45And if you're thinking about the government
• 08:47as where this document is fitting,
• 08:50think about how many different negotiations
• 08:52and collaborations and partnerships are operating
• 08:54just over this document.
• 08:57And again, if that's the government,
• 08:59here's where the government sits now.
• 09:00You know, some of these organizations
• 09:03are dealing with one or two different arrows.
• 09:04The government is negotiating all of them.
• 09:06And, you know, just to highlight some of the arrows
• 09:09don't even connect directly to the document.
• 09:11So it's a really complex kind of daunting system
• 09:15to try to negotiate what these collaborations look like.
• 09:19So now I hope I've convinced you
• 09:20that Liberia is a very interesting place to study this.
• 09:23I'll talk a bit now about the study design and methodology
• 09:26and we'll talk about some of the influences
• 09:30for how this was designed.
• 09:32So the overview and goal of this particular study
• 09:35was to explore these dynamics
• 09:37of global health collaborations
• 09:38from the perspectives of those working in Liberia.
• 09:41So pictured here is Dr. Bernice Dahn.
• 09:44She's a former minister of health.
• 09:46She was chief medical officer
• 09:47for a time as well in the country.
• 09:50And she's just all around, you know,
• 09:53an expert on health system resilience in Liberia.
• 09:56And she gave a lecture at Yale,
• 09:58I believe it was in either the spring of 2022
• 10:02or the fall of 2021,
• 10:03titled The Unchecked Power of the Purse.
• 10:05And this talked about a lot of the inequities
• 10:07she's seen over her career working in global health.
• 10:10And from this lecture,
• 10:12there were two master's students and two undergrads
• 10:15along with Christina Talbert-Slagle
• 10:17who designs this qualitative study
• 10:19to explore these relationships in Liberia.
• 10:22So Dr. Talbert-Slagle was the co-PI along with Dr. Dahn.
• 10:27So we had the US representation affiliated with Yale
• 10:31and then strong Liberian representation as well.
• 10:34So there's this whole push that global health
• 10:37is inherently equitable
• 10:39when you talk about the word collaboration,
• 10:40but I think many of us kind of know that it's not.
• 10:43And from Dr. Dahn's experience,
• 10:45she's talked about how accountability and transparency
• 10:49are a one-way street.
• 10:50Essentially that the donors are controlling
• 10:52what accountability looks like,
• 10:54what is allowed to be transparent.
• 10:57She's talked about donors' priorities
• 10:59being favored over government needs
• 11:02and that there's a value of,
• 11:04there's claims of value of empowerment,
• 11:06but that's never actually transferred to the government.
• 11:10So it's leaving weak systems weak
• 11:11and perpetuating this corruption
• 11:14and ideas of perceived corruption.
• 11:16So really hearing these issues,
• 11:20knowing that Liberia is a great setting to study this,
• 11:22this is why a really qualitative study
• 11:24makes a lot of sense here.
• 11:26So if we're looking at,
• 11:27trying to understand the subjective meanings
• 11:29from social contexts, from perceptions and understandings
• 11:32and these actions and behaviors
• 11:34of people working in this setting,
• 11:36we can try to map out what this actually looks like.
• 11:39You can have as many structures and norms
• 11:41and frameworks as you want,
• 11:42but we really would like to understand
• 11:44and gain insight to the human experience here.
• 11:48So a couple of terminology,
• 11:49I know I've talked a lot about the word collaboration.
• 11:54So collaboration generally in this type of setting
• 11:56is a low resource country that needs support.
• 12:00Here we're talking really about financial support
• 12:02from a high resource setting.
• 12:03And this again can create a sort of reliance
• 12:07on needing that type of funding and support.
• 12:10And I think it's really important
• 12:11that collaboration is accepted as positive.
• 12:15It's not always positive.
• 12:17You really wanna understand the nuance
• 12:19of what's driving a collaboration
• 12:21or collaborative partnership.
• 12:22And this is kind of known
• 12:24in the field of global health and development,
• 12:26but there are calls to have more equitable collaborations
• 12:31and decrease this dependency, but is it really happening?
• 12:35So again, another push for why qualitative.
• 12:38By the end of this part,
• 12:39I'm gonna be convinced that qualitative is perfect
• 12:41to be studying this.
• 12:44And so, oops, my keys jumps forward a few slides.
• 12:50To sum up this notion of,
• 12:53we can have all of these calls and frameworks.
• 12:55We can have normative accounts of why equity is essential,
• 12:58and we can have practical guidelines,
• 12:59which are frameworks and norms
• 13:01of how we should be operating in these collaborations,
• 13:03but we need these empirical studies to inform action.
• 13:07And I think this quote from Bauer,
• 13:09it's a study of why considering equity
• 13:12in global health collaborations is necessary.
• 13:15This really just sums up all of that.
• 13:17I'm not gonna read the whole quote,
• 13:18but I'd like to focus on this idea
• 13:20that empirical studies can provide important insights
• 13:23from the experiences of those involved
• 13:25in developing equitable research collaborations.
• 13:28They could also inform policies, frameworks,
• 13:31and guidelines related to equitable research collaborations
• 13:34in global health.
• 13:35So tying to implementation science,
• 13:37we're really trying to build an evidence base
• 13:39of we have these frameworks,
• 13:40but are these frameworks actually doing
• 13:42what we think that they are doing?
• 13:47And just, again, I wanna reflect on this image.
• 13:50This is what we're trying to understand.
• 13:52We're trying to use qualitative data
• 13:54to make sense of these arrows
• 13:55and to discern some of the patterns
• 13:57and how individuals are acting in this system.
• 14:00And a lot of times in global health,
• 14:02in health system studies,
• 14:03we talk about the software and the hardware of the system.
• 14:06And briefly, I'll just talk about that.
• 14:09The hardware is generally the infrastructure,
• 14:12the finance, technology,
• 14:14interventions that are really easy to measure
• 14:15and see that we're making these investments.
• 14:18And then associated with that is the tangible software.
• 14:21So those are kind of go together.
• 14:23That's how you create structures and systems
• 14:25and hierarchies of how you're going to make change
• 14:28within health system.
• 14:30Now, what we're really curious about though,
• 14:32and especially in this study,
• 14:35are the values and norms and the relationships
• 14:38and communication and power that exists
• 14:41that aren't as easy to measure or to understand.
• 14:45You can't really put a quantitative measure on this.
• 14:47So by understanding the lived experience of it,
• 14:50we can try to map places to intervene and strengthen it.
• 14:54So all of these questions and all of this background
• 14:57now led to a study of this.
• 14:58So we wanted to examine these dynamics and perceptions
• 15:01through financial control, accountability,
• 15:04and decision-making.
• 15:06And the idea of focusing on these three within the study
• 15:09gave us our interview guide,
• 15:12some structure to follow along with.
• 15:14And we really wanted to keep it broad
• 15:16to talk about the partnerships in Liberia.
• 15:19It wasn't just financial donors
• 15:21and it wasn't just recipients.
• 15:22We talked to academics, we talked to NGOs
• 15:25to really try to get a rich experience of how this operates.
• 15:30So a few of the procedures.
• 15:32This here is a picture of some of the research team.
• 15:36I think we're missing a few people,
• 15:37but right in the center there is Dr. Talbert Slagle.
• 15:40And these are some of the students that were there
• 15:42over the summer representing Yale.
• 15:46So we obtained IRBs to start from both universities.
• 15:50So it was grounded in ethics in both Liberia
• 15:53and here at Yale.
• 15:55And the participants were recruited
• 15:56through purposeful sampling and then snowball sampling.
• 15:59So the purposeful part of it
• 16:00was reached out directly to by Dr. Dahn.
• 16:04And so she was able to work within her network as well.
• 16:08It was so strong and it would kind of have been a huge pass
• 16:10to not use her assistance with that.
• 16:14And then once we started the interviews,
• 16:15there was snowball sampling.
• 16:16So interviewees were able to recommend different people
• 16:19throughout the country
• 16:20that they felt would have powerful insight as well.
• 16:23So the interviews were in the summer of 2022
• 16:27between July and August.
• 16:28Then they were anywhere between 30 and 60 minutes
• 16:31either on Zoom or in-person in Liberia as possible.
• 16:34And to ensure kind of quality assurance,
• 16:40we had a Liberian data collector
• 16:42and an American data collector
• 16:44present at each data collection opportunity.
• 16:47And this was to make sure it was quality data collection,
• 16:50but also we were using this
• 16:51as a training opportunity for students.
• 16:53And so it's really good to be able to hear
• 16:56what your colleagues are doing
• 16:57or follow-up questions they might use
• 16:59or be able to flag a point where they missed the question.
• 17:01These were all novice data collectors.
• 17:03So it was great to have the teams.
• 17:05And then the students all transcribed the data themselves,
• 17:09which also helped embed them fully in this process
• 17:12because if anyone's done transcription,
• 17:15it is not the most fun of a process
• 17:17and it takes a lot of time.
• 17:19But they were great.
• 17:21And we ended up having a total of 38 interviews.
• 17:26We realized after the fact
• 17:29that we didn't do great training on note-taking
• 17:32if anyone declines to be interviewed.
• 17:34So we did eliminate three of those
• 17:36just on the idea that it didn't seem like
• 17:39we really had done adequate training
• 17:40for the data collectors.
• 17:42But we were still left with 35 interviews.
• 17:45We were really trying to have solid female representation
• 17:48because sometimes in global health, that can be very skewed.
• 17:50And I'd say 40% we did okay.
• 17:53And nationality,
• 17:54we definitely wanted the Liberian experience overall
• 17:57and what it's like to be working in Liberia.
• 17:59So that felt good as well.
• 18:02And then this idea of the classification of position.
• 18:05Many of these people have been working in this field
• 18:07for many, many years.
• 18:09And they had many different roles.
• 18:10Some had been Liberian government workers.
• 18:12Some had then switched into NGOs
• 18:15and some then had even shifted into the role of a donor.
• 18:18So in the process of reviewing the data,
• 18:21we did our own classification then
• 18:23of what was the most prominently discussed role.
• 18:26So that also forced the deeper reading of the data
• 18:28to be sure we were capturing that.
• 18:34So here we are now, we have these 35 interviews
• 18:37and we were trying to think like,
• 18:39what is the best way to be analyzing this data?
• 18:41And we were really guided
• 18:42by Brown and Clark's thematic analysis,
• 18:45which is a method for identifying, analyzing
• 18:48and reporting patterns or themes within data.
• 18:52And I know I have this as my last bullet point,
• 18:54but I think I will note it as my first.
• 18:56Thematic analysis is the most used
• 18:58but least well-defined method of qualitative analysis.
• 19:01So it was really important,
• 19:03especially for me having worked with qualitative data
• 19:06that I really wanted to document
• 19:07why we were making the decisions we were making
• 19:10and why this actually was the best way
• 19:13to be conducting this analysis.
• 19:15And again, this is a training opportunity.
• 19:17So this type of analysis fit well
• 19:19with being able to train students.
• 19:21It's a good intro to qualitative analysis.
• 19:25We could also go from picking a rich description
• 19:28of the dataset and doing a broad overview of it,
• 19:31or we were able to dive in with questions.
• 19:33And at this stage, I'll be honest,
• 19:34we weren't really sure what we were going to do.
• 19:37If that we thought we wanted to do a rich description,
• 19:39the more we engage with the data, it was huge.
• 19:42So we started asking more specific questions.
• 19:44And you'll see when we get to the analysis,
• 19:46we actually kind of did a layered like two-level analysis,
• 19:49which was really fun.
• 19:50So I've never done that either.
• 19:52But just to note that we actually started
• 19:55with grounded theory.
• 19:56We thought that's what we were going into.
• 19:58So just to highlight that this really was
• 20:00like a team engagement.
• 20:01We were going back and forth thinking
• 20:03what theories make sense.
• 20:04And grounded theory, we felt we
• 20:06actually already kind of knew
• 20:08what we were expecting from the data.
• 20:09We knew what patterns existed in this type of world,
• 20:13but we were trying to see how they manifested
• 20:15within Liberia.
• 20:16So we felt that there was enough of already theoretical,
• 20:19small T theoretical background
• 20:21of what we could be expecting with this data.
• 20:26So just a bit about the structure of the application
• 20:29of thematic analysis.
• 20:30It kind of is on a spectrum a little bit,
• 20:33starting with coding reliability,
• 20:34which really focused on getting this objective
• 20:37and unbiased coding.
• 20:38You're using a codebook that everyone agrees is defined
• 20:42and strongly used and reflective of the data
• 20:45and also what you expect to see.
• 20:47So the goal with that is that every team member
• 20:50could see an excerpt of data
• 20:52and know exactly what code that would go into.
• 20:55I mean, that's if you have perfect inter-rater reliability,
• 20:57which is a measure of how much you are actually
• 21:00as a team coding the same data the same way over and over.
• 21:04So this is great for teams, but it also is pretty rigid
• 21:08in how you're structuring your codebook.
• 21:10So you kind of have that set early on.
• 21:13And then on the other end, all the way on the flexible end,
• 21:17you are kind of, the idea is that the researcher embraces
• 21:21where they're sitting with the data.
• 21:23You're really acknowledging that like you are part
• 21:25of the tool that is working with this data.
• 21:28You're having like a big impact
• 21:30on how the data is shaped as well.
• 21:32There's open coding and no real structured codebook.
• 21:36You can have like notes and documentation throughout,
• 21:39but the goal is that your end result from coding is themes.
• 21:43It's not really creating a codebook to go back
• 21:45and reapply this codebook.
• 21:47So really that's kind of like an individual setting.
• 21:52And then somewhere in the middle, we have codebook
• 21:54and codebook uses kind of both of them a little bit.
• 21:57You're using a structured codebook to assist analysis,
• 21:59but that's not driving the objectivity of your results.
• 22:04Accuracy between coders is not the focus of the codebook.
• 22:07And the codebook can really evolve throughout the process.
• 22:09Again, with qualitative, as long as you're documenting,
• 22:12the goal is to really be documenting, documenting,
• 22:14documenting why you're doing what you're doing
• 22:17and why it makes sense.
• 22:18And it's really great for teams.
• 22:20Again, we were working with a team.
• 22:21It was a pretty big team by the time we got to coding.
• 22:24So we needed some kind of structure to assist us.
• 22:27But again, we were really trying to keep it
• 22:29as organic as possible when we were moving through the data.
• 22:33So I'd say we fell kind of about there.
• 22:35We were within the codebook structure,
• 22:37but there was a lot of reflexive aspects going on.
• 22:41I'll talk a bit more about the structure of the team,
• 22:44but I was reflexive of working
• 22:47between two different teams on this
• 22:48and also how the codebook was adapting
• 22:51and changing throughout.
• 22:54So there are six steps of thematic analysis.
• 22:57You start with familiarizing yourself
• 22:59and generating your initial codes.
• 23:01And that's kind of a phase
• 23:02where you're going back and forth a little bit.
• 23:06But we tended to kind of stay in this cycle for a while here
• 23:11before we moved on to themes and the other steps in it.
• 23:15But I'll focus on this for now,
• 23:17the process that we had five members now of the team coding.
• 23:20We had two Liberian citizens and two American, and then me.
• 23:26I was the fifth.
• 23:27So we started our first meeting.
• 23:30Both PIs were present as well.
• 23:32We all read through three interviews
• 23:34and created an initial codebook
• 23:37and thought, had many discussions
• 23:40of how does this codebook pan out?
• 23:42Does it work? Does it not?
• 23:43And I think, I know we had at least one two-hour meeting.
• 23:46I think we had two two-hour meetings
• 23:48to kind of get through this rough idea
• 23:49of what a codebook could look like.
• 23:51And then we came to an idea,
• 23:52like a consensus on what a draft was.
• 23:54And then we moved it to code two more interviews.
• 23:57And this ended up just being the coding team,
• 24:00the four students and myself.
• 24:02And then we came back and continued
• 24:04to refine this codebook and finalize it.
• 24:07So after our second meeting, then we split into two teams.
• 24:10I'll walk through that a little bit now.
• 24:12This is a screenshot of kind of how we organized it.
• 24:15And I think it'll walk through a little bit of this process.
• 24:18It says we did 10 weeks.
• 24:21That was over the course of many months.
• 24:22We were working with multiple schedules,
• 24:25multiple commitments, multiple time zones,
• 24:29and students in very different aspects of their careers.
• 24:32So we ended up having two meetings.
• 24:35Once we sat with, you know,
• 24:37this part of having the initial code team going,
• 24:41then we split into two teams, one and team two.
• 24:43I sat on both teams.
• 24:45So every time each team met, I met as well.
• 24:48And then we'd meet as a group every Friday
• 24:50to then go over the codebook
• 24:52and reassess and decide, you know,
• 24:54what changes are we making?
• 24:56And you can kind of see at the top,
• 24:57we had the deadline for when we wanted to complete it.
• 25:01And then in blue, it was when we actually completed it
• 25:04because some meetings took a lot longer.
• 25:06And then we highlighted which codebook we would be using
• 25:09because we were using many iterations of this codebook
• 25:12as it was defined throughout.
• 25:14And I'll do a quick screen share then
• 25:17to just highlight the evolution of the codebook.
• 25:20This is from the end of September last year,
• 25:22one of our first codes.
• 25:24You can see that we started with what we would have
• 25:27as like our parent code, how we defined it,
• 25:30how we had child codes.
• 25:32I'll just focus on accountability right now.
• 25:34You can see that we broke that down into many different,
• 25:38you know, D meant donor, R meant recipient.
• 25:41At this point, we were still classifying our interviews
• 25:43as donor and recipient, which then the more we read them,
• 25:47that's where we started to get, you know,
• 25:49some donors are, or some recipients
• 25:52are actually donors themselves.
• 25:55And, you know, it was a really,
• 25:56we had to actually classify the interviews then
• 25:59and it made it a lot easier.
• 26:00But you can see that this is like the first iteration.
• 26:02You can see notes, you can see changes.
• 26:05And then finally to the last one.
• 26:08So this was really just a screenshot
• 26:10of how, you know, accountability evolved.
• 26:13Then we, once we had realized we could classify people
• 26:16as to what position they were speaking about the most,
• 26:20it just came down to, was there a form of accountability
• 26:23or was there a lack of accountability?
• 26:24So then we could go into how are NGOs
• 26:27talking about forms of accountability?
• 26:29How are donors, foreign donors
• 26:32talking about forms of accountability?
• 26:34They aren't, if you want to know, in their own form.
• 26:38So it became a lot easier to figure out
• 26:41how to start organizing and structuring the data.
• 26:44Oh, I just saw the note that every time I turn my head,
• 26:48I'm muscling sound.
• 26:50So I will keep focused on the computer from now on.
• 26:54So yeah, the codebook, as you can also see
• 26:57at the bottom of the screen here,
• 26:58we went through so many iterations and we dated them
• 27:01so that as a team, we could go back to the shared document
• 27:04and know which is the most relevant codebook to be using.
• 27:12So here we are now coming back together.
• 27:14We're searching for themes.
• 27:15So imagine this is now,
• 27:17we've got through all 11 weeks of those meetings,
• 27:21which were 30 some meetings on my end
• 27:24of like getting these teams together.
• 27:26And we're starting to talk about themes.
• 27:27So all five of us met and discussed common themes
• 27:30and talked about our notes on what we saw
• 27:32kind of coming out of the data.
• 27:33And we started grouping codes based on where we thought
• 27:36codes were fitting into give development to these themes.
• 27:42And then from here, we did discuss as a big group,
• 27:46but then really only me and one of the team members
• 27:49continued to start actually writing our results up
• 27:53and refining and defining and naming the themes.
• 27:58And I'll be honest at this stage, it was just,
• 28:01we were doing a lot of wrangling with a lot of people.
• 28:04So it kind of made sense to just start focusing
• 28:06a little bit more on how to move forward
• 28:08now the data analysis part of it.
• 28:09And it was great.
• 28:11People, I think within that group,
• 28:13I'd say people were graduating,
• 28:15some were working on other projects.
• 28:18So it was never like people didn't want to be involved.
• 28:21They were always willing to give feedback
• 28:23and step in as needed.
• 28:24But this one student really had an aptitude for coding
• 28:28as well for qualitative analysis.
• 28:30And in the vein of being reflexive,
• 28:35I'd say maybe she and I worked well together.
• 28:37Maybe we approached the data the same,
• 28:39but at the same time, I was just,
• 28:41she was a student who had never taken qualitative before,
• 28:44but she was willing to just really discuss.
• 28:46And she was never afraid to kind of get knee deep in it
• 28:49and really talk through what definition she was seeing
• 28:52or to push back against something
• 28:54I or someone else was saying,
• 28:55or to be quick to jump in and give examples
• 28:59to bolster other people's feedback.
• 29:01So she just got it.
• 29:03So I'd say from there, it became her and I really working on
• 29:07how to move forward with this.
• 29:09So I'll present now some of the results that we saw.
• 29:14Again, a lot of this is focused on the analysis,
• 29:17but we do have results that I think were really powerful.
• 29:21We have two phases of the analysis.
• 29:23And after I go through some of the results,
• 29:25I'd like to talk a bit about just what it took
• 29:27to get to the images that we used,
• 29:30because I think also in qualitative work,
• 29:32it is so powerful to have images
• 29:33that can represent what you're trying to say.
• 29:36And it's kind of a quick takeaway for people as well.
• 29:40So starting here, we found this cycle of,
• 29:43it's pretty clear, whoever had control of the money
• 29:45was the person making the calls
• 29:48for how they were going to prioritize using that money.
• 29:51And then a lot of times, so we'll talk actually first a bit
• 29:55about just that pattern right there,
• 29:57who has the money and how that gets to be
• 29:59in that priority setting power.
• 30:01So we started this analysis really diving first
• 30:04into priority setting power.
• 30:06So we wanted to see kind of how that manifested
• 30:09within the system, but how that was being pushed along.
• 30:11So that's why we focus now a bit
• 30:13on what financial control really means.
• 30:17So the theme we discussed was that priority setting power
• 30:21is most strongly tied to whoever has the financial control.
• 30:24Not surprising, again,
• 30:26this was something we kind of expected to see.
• 30:29There was a strong hesitancy among those we interviewed
• 30:32to say no to donors.
• 30:33There was kind of a pressure that if you say no,
• 30:35you're going to damage a relationship
• 30:37to have funding for a long time.
• 30:39So it was tedious.
• 30:41And that also there was a feeling
• 30:42that these gestures of collaboration
• 30:44were not actually genuine.
• 30:45They were kind of just checking boxes.
• 30:47And it was kind of obvious to definitely
• 30:51on the side of being a recipient,
• 30:52but sometimes donors are mentioning that as well,
• 30:55that they were doing this because it was expected.
• 30:58And I'd like to read this quote.
• 30:59I think it's a great quote that sums up
• 31:01being engaged in this type of work right now.
• 31:05Most of the time,
• 31:06the international partners come with what they want to do.
• 31:09So they prioritize what they want to do
• 31:11in a particular environment or in a particular area.
• 31:15And in many instances, you either accept it
• 31:18or it is pushed down your throat.
• 31:21It is a bad process because when you are building a system,
• 31:24if you want to help,
• 31:25it is good to work with the people that you want to help
• 31:29to tell you what their priorities are.
• 31:31And you can work around it so that at the end of the day,
• 31:34they have an ownership to it.
• 31:36But if you decide on what you want to do for the recipient,
• 31:39there is no ownership.
• 31:41So it's just thrown on them.
• 31:43And I think that that push of it being,
• 31:47that notion of it being pushed down your throat,
• 31:49like this quote just stood out immediately
• 31:51that it was, you know, people are very graphically honest
• 31:57that they have no say in how this is being negotiated.
• 32:03So then now we'll focus a bit on the other side.
• 32:06You'll see that I like to really demarcate
• 32:09where I'm functioning in these images as well.
• 32:12But now there's this idea.
• 32:14So you set your priorities,
• 32:15you have how you want this work to get done,
• 32:18but then metrics are being consistently set
• 32:22to meet those priorities.
• 32:24So if you're meeting those priorities,
• 32:26you hit those metrics, the funders are seeing this
• 32:30and they're funding the same type of cycle.
• 32:32You know, that's just innate nature.
• 32:34It's a feedback loop,
• 32:35a reinforcing feedback loop of that exact process happening.
• 32:39So these implementation plans are driven by donor priorities
• 32:42with outputs that reflect these
• 32:44donors' measures and metrics.
• 32:47Sometimes they're not reflecting
• 32:48what the country members really even want to be seeing.
• 32:52So there's a stronger obligation back to home country
• 32:55than to the Liberian government.
• 32:57A lot of times we did hear quotes on taxpayer dollars
• 33:01and who needs to be having reports back
• 33:04on what's going on with their money abroad.
• 33:08And donors generally in this sense
• 33:09focused on short-term metrics.
• 33:11I'll talk about this a bit in the quote,
• 33:13but the Liberian government really wanted
• 33:15to see longer term sustainability interventions.
• 33:17And it's really hard to factor that in
• 33:19when you have to be working under someone else's priority.
• 33:23So then the idea was that these successful outcomes
• 33:25lead to successful funding,
• 33:26and that funding leads to, again,
• 33:27like I said, a reinforcing loop.
• 33:29So for this quote, it's not just priorities,
• 33:33but even the results are like,
• 33:35who decides what success looks like?
• 33:38Part of the problem is the way
• 33:39that the funders define success.
• 33:42The way that a lot of donor funded projects are measured,
• 33:44their success is measured by performance-based indicators.
• 33:48And I think, you know, an important measure of success
• 33:51is contribution towards building systems
• 33:53and a lot of long-term things that are difficult to measure
• 33:56within funding cycle of even five years.
• 33:59That also distorts how things are planned
• 34:02and how success is defined, and that's a cycle.
• 34:06I mean, this quote really just summarizes
• 34:08everything I showed within that cycle.
• 34:10And there were a few people who also spoke about this idea
• 34:13that when you come into a lot of these big
• 34:15international donor organizations, or even some of the NGOs,
• 34:19you're coming in on a five-year timeframe.
• 34:22And so what you wanna achieve in that five-year timeframe
• 34:24for your specific role
• 34:26is what's gonna get you to your next role.
• 34:28So people are naturally looking out for themselves too
• 34:31in their own career, but at this point,
• 34:33it's not creating an allegiance to building a bigger system
• 34:35that exists beyond them when they're there.
• 34:42So reinforcing here is the first phase of the analysis.
• 34:45We created this loop.
• 34:46We were left thinking though,
• 34:47like, why is this loop still existing?
• 34:50Is there a way that we can understand
• 34:51what's going on underneath it?
• 34:53So this is what we call the cyclical process
• 34:56of priority setting.
• 34:57It keeps operating in this cycle.
• 34:59So our goal was to then rotate it
• 35:02and think how can we discern what's going on underneath it?
• 35:06So we came up with this type of image.
• 35:09What's going on that's driving these types of patterns?
• 35:12And we wanted to really understand the underlying factors
• 35:14that are influencing this process.
• 35:16When we wanted to create it,
• 35:17it's kind of like a vortex in a way,
• 35:20or the idea of the iceberg model,
• 35:22where what you see on the surface,
• 35:23there's a lot more going underneath.
• 35:25And it's kind of the power inherently of qualitative work.
• 35:28But at the same time,
• 35:29we wanted to show that within this data,
• 35:32three major themes kind of emerged,
• 35:35that there was a history of prior engagement,
• 35:37a level of transparency and patterns of accountability.
• 35:39And some of these were reflective
• 35:42of what we asked in our guide, our interview guide.
• 35:45Some had emerged a bit more organically
• 35:47in how interviewees were talking about these topics.
• 35:51But we did want to note that there's likely more
• 35:53that exists underneath this as well, driving this pattern.
• 35:56But this is what we saw within this data.
• 36:01So the history of a prior engagement
• 36:03is referring to the ways that prior engagement in Liberia
• 36:06forms current collaboration.
• 36:08So as I had said, giving the background of Liberia,
• 36:11there was civil conflicts
• 36:13that kind of set the stage on reliance.
• 36:15Then there was Ebola.
• 36:17So it created this state of emergency and crisis
• 36:19that many donors continued to collaborate in this way.
• 36:24There's kind of this mentality of get in, get work done,
• 36:27and less of a focus
• 36:29on the long-term sustainability of a system.
• 36:32Donors did have a continued fear of corruption
• 36:34and mismanaged funds.
• 36:35And many Liberians were not ignoring that.
• 36:38They were not saying that that wasn't true,
• 36:40but that was perpetuating a fear
• 36:42of changing types of investment and ways to engage.
• 36:46And this wasn't just seen,
• 36:49I would assume this isn't only seen in Liberia.
• 36:52I think we also know from other studies and reports
• 36:55that external partners often operate
• 36:57in low resource settings
• 36:58in unethically, unethically questionable ways.
• 37:02They might not do in their home country
• 37:03or different types of settings.
• 37:05And that was definitely evident in Liberia
• 37:07in the way that some people spoke about this.
• 37:10So this quote summarizing would be,
• 37:12they, the donors, resist channeling their money
• 37:15to a recipient country government
• 37:17until it's running as well as their own.
• 37:20Like at that point, they don't need your money.
• 37:23The country's already running really well.
• 37:25So it's, if you want to wait for the country to be perfect
• 37:28before you take any risk,
• 37:30what that does, a preoccupation with fiduciary risk
• 37:33ends up becoming a strategic risk
• 37:35that your programs can fail.
• 37:37You're not helping the country
• 37:39to achieve its development goals and progress.
• 37:41So this idea of there's a constant risk state
• 37:45is not allowing there to be many different ways
• 37:47to consider how to fund or support or collaborate.
• 37:52Next, you'll remember in that diagram,
• 37:54I talked about that the donor
• 37:56is determining how transparent they are willing to be.
• 37:58And this was really specific to funds
• 37:59because that was a lot of what we were asking.
• 38:02But a lot of times, and a lot of us are in academia here,
• 38:06I'm assuming it's not all of us,
• 38:07indirect costs are not totally transparent,
• 38:10but especially members of the Liberian government
• 38:13are saying, we don't know how much money
• 38:14is coming through our country now.
• 38:15We don't know what the overhead costs are for NGOs.
• 38:18We don't know this.
• 38:20So sometimes, when a donor is filtering money
• 38:23through either NGOs or supporting partners,
• 38:27it's not clear what money is actually getting into Liberia.
• 38:31And this quote, I think sums that up perfectly
• 38:33where it's an imbalance of maybe
• 38:34what the public perception is of a huge investment,
• 38:38but then what really is happening in countries
• 38:40is not the same amount.
• 38:41So one thing that was of concern was the NGOs,
• 38:44not the donors, because the donors will channel funding
• 38:47through the international NGOs.
• 38:49The concern was that they will come
• 38:51and give the narrative of what needs to be done,
• 38:53but the actual financial display was kept secret.
• 38:56And I always told them, we know that you write proposals.
• 38:58We know that a certain percentage of money
• 39:01is there for your administrative costs.
• 39:03We know, but we also wanna know
• 39:05how much is there for service delivery.
• 39:07You cannot get up and tell Liberia,
• 39:09oh, we channel $2 million through external organization,
• 39:13and then you come and you use 50% of that money
• 39:15on administrative costs.
• 39:16It is wrong.
• 39:18There were a couple accounts too of different partners,
• 39:22either Liberians or based in Liberia
• 39:24or strong collaborations in Liberia
• 39:26saying that there is an inequity of how resources are used.
• 39:29Liberia doesn't even have stable electricity
• 39:31and fuel costs to run an office are astronomical,
• 39:35but sometimes there's rules of what would be funded,
• 39:38what wouldn't be funded
• 39:39when it's not in that organization's office.
• 39:42So it was really tricky to navigate
• 39:44some of the transparency on what was going on.
• 39:47And finally, at the bottom of this vortex I had shown,
• 39:51the donor is creating structures
• 39:53that are holding the recipient accountable
• 39:55by often overlooking their own accountability
• 39:57back to the Liberian government.
• 39:59So the government,
• 40:01we didn't ever explicitly ask about audits.
• 40:04We asked about accountability and auditing came up
• 40:07by almost every single Liberian government representative.
• 40:10And they were really proud of it
• 40:12because they said we would meet
• 40:13all of our audits in the end.
• 40:14We were hitting these milestones,
• 40:16but also there were reports of being audited
• 40:18like 17 times during a project or something.
• 40:21So it's continuous, continuous monitoring
• 40:24and reports being written.
• 40:25And that was never happening on the donor side.
• 40:28There were not audits on what the donors were doing
• 40:30or not audits on how, again,
• 40:32tying back to the financial aspect
• 40:34on where they were spending their money.
• 40:36So it really was, as Dr. Dunn had alluded to,
• 40:39a one-way street.
• 40:40And it was really hard for anyone
• 40:42other than who had that financial control to negotiate that.
• 40:46So we thought it was also interesting,
• 40:48and this was noted when we talked about the bigger circle,
• 40:52the original phase one of the analysis,
• 40:55that donors are really responsible
• 40:56only to report to their home countries.
• 40:59So you're seeing now that there's this discrepancy
• 41:01of kind of where there is kind of what you're owing
• 41:05when you're working in another setting.
• 41:08So again, there aren't really any systems for accountability
• 41:11in the other direction.
• 41:12That is, for a funder to even be really questioned
• 41:15for maybe withdrawing funding or changing their priorities
• 41:18or not providing the amount of funding originally promised
• 41:21or any number of things,
• 41:23the decisions they make about what can be paid for
• 41:25and what can't,
• 41:27the extent that they're really being held accountable,
• 41:29that's internal itself.
• 41:31They're self-regulated, really.
• 41:33So if there's no structure
• 41:34of who these donors are reporting to,
• 41:35remember back that document that I had representing
• 41:38how the Liberian government is trying to work
• 41:40with all these red arrows kind of shooting around it.
• 41:44There's no way to kind of make sense of that
• 41:46if there's no responsibility to report
• 41:48back to the government of what's going on.
• 41:52So I'd like to just reflect and really give,
• 41:57I know Christina Talbert-Slagle is on this call.
• 42:00We spent so much time trying to get to these images.
• 42:04I think it's also really important to talk about that
• 42:07in the process of qualitative work,
• 42:09because again, it's really words,
• 42:11but images can be powerful.
• 42:12So we went from something like this,
• 42:16trying to map out what these themes and codes
• 42:19are looking like and where they're fitting in.
• 42:21This looks like something out of like an electric box
• 42:23or something,
• 42:24where we finally then shift into this cyclical pattern.
• 42:27But then thinking of where does collaboration fit in?
• 42:30Is it within the priority setting
• 42:32to the implementation of the plan,
• 42:34or is it part of this cycle
• 42:36rather than where we kind of ended with it being
• 42:38underneath and embedded by supporting this cycle?
• 42:43We kind of went back to the square again,
• 42:46thinking about a lot of definitions
• 42:47of what is legacy of engagement?
• 42:51Who's being held accountable again?
• 42:54And then finally, we even did a causal loop diagram.
• 42:57So we really were going all over with trying to get to this,
• 43:01but as you can see,
• 43:02there's kind of this building to get to this point
• 43:04of it actually being a cyclical pattern
• 43:08that we can kind of start to dig underneath of.
• 43:12So then finally producing the final report.
• 43:16I'm excited that we're pretty much about to submit it.
• 43:20As I'm reading some of these quotes,
• 43:22I'm seeing spots where we had to clean them up
• 43:23a little bit too, but the report I think is great.
• 43:27I'm really excited.
• 43:28It's something that we've worked on for a long time
• 43:30and going through these methods
• 43:31are showing like how strategically and thoughtfully
• 43:34we thought about every step of the way
• 43:35in how to represent this data.
• 43:40And just some, you know,
• 43:41this ended up being embedded throughout,
• 43:43but just some highlights of using this
• 43:45as a training opportunity.
• 43:47The size of the team was big.
• 43:50And I think in a setting other than academia and training,
• 43:54it would have been like a little bit daunting
• 43:56to try to have a timeframe to do this,
• 43:58but it really allowed for a hands-on experience
• 44:01to learn these qualitative methods.
• 44:02And I think for students, that's really invaluable.
• 44:05So it would be kind of cool to think about ways
• 44:07that we can kind of implement something
• 44:10like training on the go with students
• 44:11in a way that is also standing true to the data
• 44:15and moving along projects.
• 44:16I really had fun being reflexive on my own aspect
• 44:21of working with this data,
• 44:22as well as trying to maintain reflexive
• 44:24of the process of working with multiple teams
• 44:26and editing a codebook and moving that part along.
• 44:30It was kind of unexpected that I would end up in that role.
• 44:33I don't think any of us knew that
• 44:34that would end up being my role,
• 44:35but I really enjoyed it and it was fun.
• 44:39And then also time is key.
• 44:41You can see this was 40 minutes of just talking through
• 44:43like some of the different meetings and discussions
• 44:46and considerations we had to make.
• 44:48But if we really wanted these methods to make sense,
• 44:51especially in qualitative,
• 44:52where I think that some of the rigor is always challenged,
• 44:55like we put so much time and effort
• 44:57into making sure this was sound.
• 44:59We wanted to make sure the codebook was inclusive.
• 45:01We wanted to make sure the approach made sense.
• 45:03So that takes a lot of time.
• 45:06And I loved the idea of thematic analysis
• 45:08where you don't have to do everything all at once.
• 45:11We did code the whole project at once,
• 45:14but then we started to realize,
• 45:15no, we wanna ask questions to really get further
• 45:18into these codes and develop the themes that way.
• 45:20And it felt a lot more manageable
• 45:22and a lot more interesting personally.
• 45:25And then also the idea that we didn't expect anything wild,
• 45:31like we kind of knew what we would see with this data,
• 45:33but I think with qualitative, it offers great insight
• 45:36and it offers great insight to,
• 45:38are these systems working the way
• 45:39that we think that they're working?
• 45:41And the human experience oftentimes might tell you not.
• 45:45So finally, I'd like to just thank
• 45:49Dr. Kristina Talbert-Slagle.
• 45:50She was the US Co-PI and Dr. Bernice Don
• 45:52who was our Liberian Co-PI.
• 45:55These are the four students and master's students,
• 45:58undergrad and recently graduated master's students
• 46:02who worked with us, Defne, Joseph, Antoinette and Johannah.
• 46:06And then again, there was a lot of support from CTLI,
• 46:10which is the Center for Teaching, Learning and Innovation
• 46:12at ULCHS, which is University of Liberia
• 46:16College of Health Sciences.
• 46:17It's a great team and supportive network to be.
• 46:19And overall, this was under BRIDGE-U Liberia,
• 46:22which is a USAID grant between here
• 46:25and the University of Liberia.
• 46:26So I think thank everyone for having me to do this seminar
• 46:30and please reach out with any questions or thoughts.
• 46:32And I will let you know the results of the impending paper.
• 46:37I'll take questions now or feedback, comments.
• 46:40I'm excited to engage.
• 46:46(attendee speaks faintly)
• 46:51<v Attendee>It looks like we entered</v>
• 46:54your toning and analysis process like remarkably fast.
• 46:59And I mean, I think that's because we're doing
• 47:01a rapid qualitative analysis in order to get results
• 47:05quickly from qualitative studies,
• 47:08especially in information science
• 47:10where we need the information to understand
• 47:13and change or tweak interventions.
• 47:17But on the tip of what you were doing this time,
• 47:19rapid analysis, but yet you sound like you're analyzing
• 47:24very large amount of data.
• 47:26It looks like it's two months period.
• 47:30<v Brigid>Oh, let me change that.</v>
• 47:34I'm gonna switch to.
• 47:36Is this a bit better to hear?
• 47:38We're engaging as a room?
• 47:41<v Attendee>One thing you could do</v>
• 47:42is just repeat the question.
• 47:43<v Brigid>Okay, yeah, I'll just repeat.</v>
• 47:44All right, let me get back to it, to my computer.
• 47:50I'll repeat the question here.
• 47:51So it was noted that it seems like the analysis
• 47:54was rather quick and I should have emphasized
• 47:57this a bit more that it wasn't.
• 47:59It took a long time.
• 48:02The timeline.
• 48:03So we did, I'd say we started with our analysis.
• 48:06It was, we had 10 weeks of meetings.
• 48:09So probably shouldn't be labeled as weeks.
• 48:11It should be labeled as meetings
• 48:12because some of the meetings took two weeks
• 48:15of us actually meeting to talk about that
• 48:18and get to the final results.
• 48:20And again, it was some of the weeks then
• 48:22if we had team one meeting and team two meeting,
• 48:25I would meet with both of those teams individually.
• 48:27And then we had to find a time
• 48:29for all of us to meet as a group.
• 48:32Sometimes that took a couple of weeks to implement as well.
• 48:35I think that if I could, if there were a need
• 48:40to have a rapid assessment or a rapid qualitative,
• 48:43something I would probably go back and do
• 48:44would be do a really initial, intense, deep dive
• 48:49into the data and then formulate questions
• 48:52worth kind of probing throughout the data with.
• 48:56But with this process, we kind of allowed that
• 48:58to emerge more of what the data was showing us
• 49:01could be different avenues to explore.
• 49:03So it really was exploratory, I would say.
• 49:07But we definitely, we had our first meeting
• 49:09with the data at the end of September
• 49:11and we wrapped up our final coding
• 49:12at the end of February, beginning of March.
• 49:16And from there, then we started diving
• 49:18into figuring out what our themes were.
• 49:22So it was a process.
• 49:28<v Attendee>In that early process,</v>
• 49:30you mentioned having the interviewers
• 49:33who did the transcripts then from recording,
• 49:36except the ones that rejected
• 49:37recording, so they were left out.
• 49:40What are your thoughts on using software
• 49:42to create initial transcripts and then do the coding?
• 49:46<v Brigid>I mean, that's how</v>
• 49:47I've always kind of worked previously.
• 49:51I worked in a mixed methods research lab
• 49:53and mostly my role was doing qualitative analysis
• 49:55for the quantitative ends of projects,
• 49:58but I didn't repeat the question, here I go.
• 50:02The question was, what are my thoughts on using
• 50:08like transcribing assistive technology
• 50:11and then going back and recleaning the data?
• 50:14And my response to that was, I think that's great.
• 50:17Really, I've used that before in other settings.
• 50:20It's really difficult in Liberia
• 50:22because of Liberian English is just hard
• 50:26to transcribe from settings.
• 50:28So you really have to listen.
• 50:31You can try using that assistive technology.
• 50:34I would have assumed that it would be not worth using.
• 50:39I know that a few of the students tried
• 50:41and it just did not work and they had to listen
• 50:43and do it on their own.
• 50:45But in previous studies, I've done that
• 50:47where you get the initial transcript
• 50:49and then it's easy to just listen to it
• 50:51and look at the words and do the editing,
• 50:53stopping the recording as it goes.
• 50:56<v Attendee>Yeah, I agree with that as well.</v>
• 50:57It's just a standard American tradition.
• 51:01Yeah, but anything helps.
• 51:05<v Brigid>Hey, there's a market.</v>
• 51:07I'm sure somebody wants to start thinking about that.
• 51:10I'll go quickly to a question we have online.
• 51:15How can we have a look at the final report
• 51:17and will it be possible or I'll say ethical
• 51:20to reproduce this analysis in other countries
• 51:22based on their work?
• 51:24This person works in Nigeria and the topic is so familiar.
• 51:27I definitely will be able to look at the final report.
• 51:31We're submitting it to journals,
• 51:33like I wanna say Monday or Tuesday.
• 51:37Kristina's probably laughing on this call
• 51:39as soon as possible,
• 51:40but definitely would love to disseminate that.
• 51:43And I think it's absolutely possible to reproduce elsewhere.
• 51:47I mean, and I would love to talk about how to do that.
• 51:50I don't think the results are gonna be reproduced,
• 51:52but this process,
• 51:53and we could have more feedback on refining this process,
• 51:56but how do you kind of dig below
• 51:58to understand what these collaborations look like?
• 52:00I think, again, this is my bias.
• 52:02I think this could be done everywhere
• 52:04that these collaborations are happening.
• 52:06And I think it would really give light
• 52:08to how to go forward to structure them a bit more equitably.
• 52:13Questions in the room?
• 52:16(attendee speaks faintly)
• 52:21So the question was, aside from journals,
• 52:23what are we hoping to do with the results
• 52:26outside of submitting them to journals?
• 52:29That is a great question.
• 52:31And that's something that I think,
• 52:33especially thinking about peer-reviewed journals
• 52:36that also comes with a specific audience
• 52:39who's reading those.
• 52:41I did present this at a conference already.
• 52:44That was the original causal loop,
• 52:46one of the green loop diagrams.
• 52:48And again, that's also a very specific audience.
• 52:51But the goal would be to, I think,
• 52:53try to write even commentaries
• 52:55or something that's gonna start getting attention
• 52:57that this is worth being investigated in a different way.
• 53:00There's different documents, like USAID and WHO,
• 53:04reports about what collaborative partnerships look like.
• 53:07And if we could get attention of that,
• 53:10if you have ideas too, I'd love to hear.
• 53:12But I think getting that attention of like,
• 53:14how can we make this routine to monitor as well?
• 53:18<v Attendee>So I just want to follow up on</v>
• 53:19some stuff that you guys have.
• 53:23So, and now, what was the experience of donors
• 53:27for this type of action?
• 53:29What was the push of power on the donors?
• 53:33And I was wondering if you were finding
• 53:36a secondary search that you were looking for on that,
• 53:41like from the risk-taking actionable things
• 53:44that you were looking for,
• 53:46or were there other things that happened?
• 53:48<v Brigid>So the question was that a lot of the themes</v>
• 53:50were donor-centric and things that the donors
• 53:52could be doing, but are there ways that we could see
• 53:55what the recipients could be doing
• 53:57to change this pattern as well?
• 53:59I think that could be,
• 54:02the way I would envision that for follow-up collection,
• 54:05I don't think we could see that clearly
• 54:06in this data as much.
• 54:08There's definitely room for it.
• 54:09I think people would have so much feedback.
• 54:11I think it would be cool to structure workshops around that.
• 54:14Like, what would that look like to implement change
• 54:17and how, I mean, another facet of my research
• 54:20is I like future studies and foresight strategic planning,
• 54:23and I'm doing that with a project in Stellenbosch right now,
• 54:26and thinking their whole focus
• 54:29is what does the future of development research look like?
• 54:32And how can you kind of plan that
• 54:34and set the stage for that?
• 54:36And I think that's like, that type of futures planning
• 54:38is like, all right, we have an idea of where we wanna be,
• 54:40and how can we work backwards to get there?
• 54:42I think that that's gonna be something
• 54:44that needs to be a collaborative meeting.
• 54:47I mean, regardless, we can't just say,
• 54:50donors, you're kicked out.
• 54:51Like, that would be ideal, but in reality,
• 54:53that's probably not going to happen.
• 54:54So how to have this be a collaborative meeting
• 54:57where there is that agency to really put forward ideas
• 55:01that are centered on what Liberians think need to happen,
• 55:05I think could be awesome.
• 55:08<v Attendee>I mean, I think, so the answer to this question</v>
• 55:13was to ask what role people thought could be done
• 55:18to improve equity and positive impact research.
• 55:22So it seems like it's an out-of-touch question.
• 55:26But the other question was,
• 55:29it seems to me that people in this board of controllers
• 55:33clearly have a different idea of government people,
• 55:40Liberian government people
• 55:43have a different perspective than Liberians.
• 55:46So, I mean, I'm thinking of different elements,
• 55:50but what are the differences in perspective
• 55:54that you've noticed? (speaks faintly)
• 56:03And I think that's a big thing to kind of have.
• 56:07<v Brigid>Yeah, so the second part of the question,</v>
• 56:10I'm not gonna look.
• 56:11Yeah, the second part of the question had to do with it.
• 56:14But it was looking at the different perspectives.
• 56:17Like we had these, the groups classified as,
• 56:20Liberian academic, Liberian government donor and NGOs.
• 56:25And it's kind of like understanding
• 56:26the nuance of perspectives between those.
• 56:29And we did try to do that.
• 56:30And there definitely was,
• 56:31especially between Liberian academic and government,
• 56:36because Liberian universities are so tied to the government
• 56:39to its government funding, they have very similar views too.
• 56:42And I mean, a quote that's kind of, yeah.
• 56:46But a quote that's even standing out to me
• 56:48was one of the academics saying like,
• 56:51you can sit down with people
• 56:52and you show up with your list of like 12 things
• 56:55that you really want done.
• 56:56And you're hoping that maybe four of them get funded.
• 56:59And with whatever gets funded,
• 57:00you cobble together what you need for your program
• 57:02because you need that money.
• 57:04And that's frustrating, right?
• 57:06Like that on that list of 12,
• 57:09maybe number two, number eight, number four,
• 57:13and 12 get funded.
• 57:14And you're like, well, how do you structure?
• 57:16We have this whole plan of a program.
• 57:18How are we structuring those four specific things?
• 57:22And then the first question I believe was like,
• 57:25what could be done?
• 57:25Did we ask people?
• 57:27We kind of did.
• 57:29How could you envision like these collaborations
• 57:31be more equitable?
• 57:32And it was pretty standard answers
• 57:33that were reflecting what the issues were,
• 57:36stronger systems of accountability or stronger reporting.
• 57:39And if we were to do this again,
• 57:41I would say like, dig into that question,
• 57:43like really dig further and have more.
• 57:46And again, this is,
• 57:48we were working with novice data collectors
• 57:50who did a great job,
• 57:51but even it highlights like training for people
• 57:55how to probe,
• 57:56training on how to circle back to a previous question.
• 57:59It's like kind of hard.
• 58:01So I think that could be a next,
• 58:03a really great next study as well.
• 58:09<v Ashley>So Brigid, I just want to note</v>
• 58:11that it's one o'clock.
• 58:13And so I think several folks will probably need to hop off,
• 58:19but I just want to thank you so much
• 58:22on behalf of CMIPS and others that for this talk,
• 58:29it was really phenomenal.
• 58:30Thank you for digging into so much of the analytic process.
• 58:34I had a question in the chat
• 58:35that maybe we could chat about later.
• 58:37You can kind of talk about later,
• 58:39but it seems like there's a lot of implications
• 58:41for new NIH requirements and policies,
• 58:44particularly the one that requires all foreign projects
• 58:48and foreign agencies to submit their lab, like lab notes,
• 58:54which is just like a phenomenal amount of work in general.
• 58:57So I think it's all wrapped up
• 59:00in everything you're talking about,
• 59:02about trust and accountability and all of those things.
• 59:06So anyway, thank you so much.
• 59:10And I'll say goodbye to everyone here and take care.
• 59:15Thanks everyone.
• 59:16<v Brigid>I had a lot of fun chatting about this.</v>
• 59:24Okay.