Costing and Economic Evaluations in Implementation Science

Transcript

• 00:02<v ->Prior to joining the faculty at YSPH,</v>
• 00:06Dr. Cameron was a member
• 00:08of the Global Health Cost Consortium,
• 00:10an initiative funded
• 00:11by the Bill and Melinda Gates Foundation
• 00:14to provide decision makers
• 00:16with improved resources
• 00:17to estimate the cost
• 00:19of HIV and tuberculosis programs,
• 00:21and to improve the availability, quality, timing,
• 00:27and policy relevance of cost information.
• 00:30He holds an MA in international development
• 00:33from American University and a PhD in health policy
• 00:37from the University of California Berkeley
• 00:39School of Public Health.
• 00:41The title of his talk is
• 00:42"Costing and Economic Evaluations
• 00:45in Implementation Science."
• 00:46He'll speak for about 45 minutes,
• 00:48and then we'll open the floor for questions and discussion.
• 00:52There you go, drew.
• 00:54<v ->Thanks very much for that introduction, Debbie,</v>
• 00:56and thank you to those who were kind enough
• 00:59to join the call today.
• 01:02I won't introduce myself any further
• 01:04than to say as a member
• 01:06of the Center for Methods
• 01:08and Implementation and Prevention Science,
• 01:10this is a pretty sort of exciting opportunity
• 01:13to share some of the work
• 01:14that I've been involved with in the past.
• 01:16This talk though, I'm gonna focus largely on, you know,
• 01:21what might be for many of you a review of topics
• 01:25as they relate to economic evaluations and costing.
• 01:28So please do bear with me on that front.
• 01:31I'll try to get to some novel research findings
• 01:34towards the end of the talk,
• 01:36but this is sort of designed as a, anyone can participate.
• 01:40You don't have to have much of any background
• 01:42in economic evaluation or costing.
• 01:44So without much more,
• 01:47let me get into the topics here.
• 01:51Okay, so this talk is divided
• 01:53into three main sections.
• 01:54First, I'm gonna discuss some key economics concepts
• 01:57that kind of lay the groundwork for the rest of the talk.
• 01:59I'll briefly cover some different types
• 02:01of economic evaluation and decision analyses
• 02:05and how costing
• 02:07in particular is an important
• 02:08and often overlooked component of implementation science.
• 02:12Second, we'll get into the nitty gritty
• 02:14of costs and costing.
• 02:15So what are they, who uses them?
• 02:17How do you do it?
• 02:19And what are some of the key considerations
• 02:20that you have to make along the way
• 02:22when you are conducting a costing study?
• 02:27If you have a great deal of experience with costing already,
• 02:29some of you may find some of this to be familiar,
• 02:31so I'll apologize in advance,
• 02:33but I'm still eager for your feedback.
• 02:35And then finally,
• 02:36I'm gonna briefly discuss the efforts
• 02:39of the Global Health Cost Consortium,
• 02:40which includes a website
• 02:41with a ton of great costing resources.
• 02:43So if you're more interested
• 02:44in costing and economic evaluation,
• 02:47I can direct you towards those at the end of the talk.
• 02:50And as part of that effort,
• 02:51I'll share some of the results
• 02:52of a somewhat recent publication,
• 02:54a systematic review of costing literature on HIV
• 02:57in Sub-Saharan Africa,
• 02:58and another study that's in progress.
• 03:02So let me move ahead here.
• 03:05So there are four main takeaways.
• 03:06You know, if you take nothing else from this talk,
• 03:10and I wouldn't blame you if you didn't.
• 03:13Four things that I'd like you to walk away from this talk,
• 03:17considering like number one is perspective.
• 03:20I'll get into what perspective is, and timeframe.
• 03:24These are two concepts that matter,
• 03:26I think the absolute most in terms of economic evaluations
• 03:30and costing and how they're conducted and the findings,
• 03:33you know, that you were able to generalize
• 03:35from these studies.
• 03:37I wanna also leave you with the impression
• 03:40that although expenditures are real,
• 03:42they're real things,
• 03:43things you can quantify,
• 03:45costs really aren't.
• 03:47They're conceptual in nature,
• 03:49and they depend on a lot of different assumptions.
• 03:52So, you know, we'll get into the nitty gritty of that.
• 03:55I'd also like to leave you with the idea
• 03:57that perfect is often the enemy of the good.
• 04:00I think that's a concept that's very familiar
• 04:01to anybody who's ever published a paper, for example.
• 04:04But on the flip side,
• 04:06little details in your decision making and costing
• 04:10and data collection can matter a lot.
• 04:13So there's a balance that we try to strike
• 04:16when we do these types of studies.
• 04:18And then finally,
• 04:19reporting standards and quality are two areas
• 04:22that have a lot of room to improve in this field.
• 04:24And I'll show you some examples of why.
• 04:27So what is economic evaluation?
• 04:29Economic evaluation is an often sort of misunderstood term.
• 04:33You've probably heard of monitoring and evaluation,
• 04:35which typically refers to monitoring processes
• 04:37for a program's implementation and ongoing operation.
• 04:41If you work in the social sciences,
• 04:42you've probably heard of impact evaluation,
• 04:44which typically refers to evaluating, say, health, social,
• 04:48or economic impacts of the program.
• 04:50These are all separate but related fields and methods.
• 04:54Economic evaluation is sort of an umbrella term.
• 04:58This includes a broad range of comparative methods
• 05:00to help to evaluate
• 05:01choices or trade-offs between costs and benefits.
• 05:05Another familiar term might be decision analysis.
• 05:08At the heart though of economic evaluation
• 05:10and decision analysis is a concept that, you know,
• 05:13we spoke about or that I'll speak about a little further
• 05:18into the talk.
• 05:18And that is opportunity costs.
• 05:20So when we talk about economic evaluation,
• 05:23what we typically wanna understand
• 05:24is what's the best value for money?
• 05:26And in the most basic sense,
• 05:28cost is what you pay and value is what you get.
• 05:31It's the benefit.
• 05:33So it won't always indicate a clear choice,
• 05:36but it will help to evaluate options quantitatively
• 05:40based on a defined set of decisions.
• 05:43These processes, they aim to capture opportunity costs
• 05:47of doing one thing over another.
• 05:48So for example,
• 05:49do we invest in home visits
• 05:52or do we make mass media platforms
• 05:55to disseminate behavior change communication
• 05:57for improved, say infant and young child feeding practices,
• 06:02or do we promote home gardens or vitamin A capsule
• 06:05supplementations to address micronutrient deficiencies?
• 06:11So lastly, I'll leave you
• 06:12with the thought that economic evaluations
• 06:14are comparative by nature.
• 06:15So not only do they compare costs and benefits,
• 06:18but a comprehensive evaluation
• 06:20will compare two or more different options.
• 06:23So let's look at a couple of different types
• 06:25of economic evaluations.
• 06:26So you may have come across these in the past.
• 06:30There's really a large family
• 06:31of economic evaluation methods,
• 06:33but it's important to note that all of these different types
• 06:36of analyses and tools at their core
• 06:39are measuring costs in dollars.
• 06:40So that that column down the middle.
• 06:43And that's the main focus of the rest of this talk.
• 06:46But a key difference across these different types
• 06:50of evaluations is gonna be whose costs it is
• 06:54that we're taking into account
• 06:55as well as how we're measuring benefits.
• 06:58So all decision analyses are comparative,
• 07:00as I mentioned,
• 07:01we're gonna compare one or more options
• 07:03in relation to another.
• 07:04So that other option could just be
• 07:06what the current standard
• 07:07of care is or the status quo.
• 07:09It could be doing nothing
• 07:11or it could be another active intervention.
• 07:13So one of the first things that we need to do
• 07:15in any such analysis
• 07:16is to understand where a comparator is
• 07:18and the most basic type of economic evaluation
• 07:21is the one on the top there.
• 07:23We'll start with cost efficiency analysis,
• 07:25which is sort of a partial form of economic appraisal
• 07:28'cause it looks only at the costs
• 07:29of the program and it doesn't
• 07:31really provide any information on let's say agriculture,
• 07:34nutrition, or health outcome of interest.
• 07:37This typically reports out on the cost per output achieved.
• 07:40So another example of this might be
• 07:41like a budget impact analysis where you're looking
• 07:44at the cost per output, per patient served,
• 07:47or per vaccine administered.
• 07:51Now the more common type of economic evaluation
• 07:54for evaluating health interventions
• 07:56is cost effectiveness analysis.
• 07:58And CEAs, they compare the cost and outcomes of two or more
• 08:02alternatives or compare a new intervention
• 08:06or treatment with the status quo.
• 08:07CEAs relate the cost associated
• 08:10with the health outcome such as cost per disease avoided,
• 08:14cost per death avoided
• 08:15or additional expected life year for example.
• 08:19And then there's some other types of analysis
• 08:21that we won't get into too much.
• 08:23Cost utility analysis,
• 08:25you know, includes analyses with say disability
• 08:29or quality adjusted life years
• 08:32or other standardized measures of preference.
• 08:37And then there's cost benefit analysis,
• 08:39which is a method in which both the costs and the benefits
• 08:42of the intervention are expressed in monetary terms.
• 08:45So in agriculture for example,
• 08:47this might be the value of increased crop production.
• 08:51While in health it would be the dollar value associated
• 08:54with the life years that were gained
• 08:56because of an intervention.
• 08:58There's several other types of decision analysis
• 09:01that I haven't included in this table
• 09:03and they exist, so sorry.
• 09:06So those include return on investment analysis,
• 09:10cost of illness analysis and others.
• 09:13So as I've suggested,
• 09:14different sectors kind of use different of these approaches.
• 09:18So for example,
• 09:19global health and education disciplines
• 09:21have promoted the use of cost effectiveness analysis.
• 09:25Implementation science probably naturally falls
• 09:28under this category
• 09:30or in the cost efficiency analysis category
• 09:33with a little bit more emphasis
• 09:34on perhaps cost efficiency
• 09:36depending on the scope of the evaluation taking place.
• 09:39And I'll get into that in a moment.
• 09:42Meanwhile, you might see agriculture
• 09:45or large scale investment
• 09:46in infrastructure being evaluated using
• 09:49say a cost benefit analysis.
• 09:53So zooming out to sort of the level that we often interact
• 10:00with these types of evaluations,
• 10:02say an aim three of your NIH proposal,
• 10:05depending of course on your desired evaluation type,
• 10:09you'll often wanna arrive
• 10:11at really a simple division problem.
• 10:13The cost of a program or policy or intervention
• 10:16divided by the outcome in question.
• 10:18For cost effectiveness analysis,
• 10:20the outcome is some measure of effectiveness,
• 10:23particularly a health outcome.
• 10:27For budget impact analysis and cost efficiency.
• 10:29This might be an intermediate outcome
• 10:32per person vaccinated or per patients served.
• 10:36But at its core,
• 10:37all the economic evaluations boil down
• 10:39to this simple division problem.
• 10:41And I wanna mention one more
• 10:43slightly more complicated
• 10:45but still simple division problem
• 10:47that you may have come across at some point, you know,
• 10:51in the process of of putting together a proposal.
• 10:54And that is one that builds very simply on the previous,
• 10:58and that's for cost effectiveness analysis
• 11:01that compares the cost effectiveness
• 11:03of the intervention and question
• 11:05to some particular alternative.
• 11:06So that's, that's the ICER.
• 11:08And this stands for incremental cost effectiveness ratio.
• 11:12So usually the comparator here is a status quo
• 11:15or a standard of care,
• 11:17but it might be some new alternative you're considering too.
• 11:21<v Person>That is very useful for developers.</v>
• 11:24<v ->I dunno if I'm getting a question here or not,</v>
• 11:27but I'll just continue about that.
• 11:30You can also use ICERs to compare
• 11:32between multiple different
• 11:33alternative interventions or configurations.
• 11:36You might end up with like a list of different interventions
• 11:38that you want to choose between
• 11:40using a process like a shopping spree scenario.
• 11:45I won't get into the details of that,
• 11:46but there's lots of different ways
• 11:47that you can use this tool.
• 11:52Lemme move on.
• 11:53In this talk though,
• 11:54I'm mostly concerned with the numerator.
• 11:56So how do we choose what counts and what doesn't count?
• 11:59What goes into the process of collecting data
• 12:01to inform these numbers
• 12:03and how might we begin to distinguish
• 12:05between studies that do that well or do that poorly?
• 12:08So I wanna get through a couple of key concepts here
• 12:10before we get started.
• 12:12And those are scarcity, opportunity costs,
• 12:15types of efficiency and perspective.
• 12:18And I'll linger on perspective a bit longer
• 12:20than the others.
• 12:21Scarcity is one of the most important concepts in economics
• 12:24and that's simply stated, resources are limited.
• 12:27So whether we have relatively more resources
• 12:30as within the US or say other OECD countries,
• 12:33or relatively fewer resources
• 12:35as many low and middle income countries find,
• 12:39there's always trade offs to be made in terms of investments
• 12:42and competing programs and activities.
• 12:44So it's a major consideration.
• 12:47The most powerful implication of the concept of scarcity
• 12:50is of course opportunity costs.
• 12:52So this refers to the simple truth that the use of resources
• 12:55for any one purpose precludes their use for another.
• 12:58And I have to say that opportunity cost is is a really great
• 13:01concept for focusing the mind by counteracting
• 13:05our own personal tendencies towards inertia,
• 13:08which is to say the tendency to say unthinkingly continue
• 13:12this year, what worked or what we did last year,
• 13:15or for that matter,
• 13:16what didn't work particularly well last year.
• 13:19So thinking about opportunity costs stimulates
• 13:22some creativity about the best use of resources
• 13:25given the needs and alternatives that exist.
• 13:29And the problem of resource allocation
• 13:30then becomes where can we get
• 13:33the greatest benefit for a given expenditure.
• 13:36And cost effectiveness analysis provides
• 13:38the sort of conceptual framework and methods
• 13:41to help make that determination.
• 13:43Efficiency.
• 13:44I'll just touch on briefly,
• 13:46there's two major types.
• 13:47Allocative efficiency,
• 13:48which con concerns the division of funds
• 13:51among different interventions.
• 13:53So HIV prevention versus treatment.
• 13:56This often involves complex choices,
• 13:59even if the health gains achievable in one area are greater
• 14:02than in another because costs also differ.
• 14:05And then there's technical efficiency,
• 14:07which is about how we implement an intervention
• 14:08that we've decided upon.
• 14:10So for example,
• 14:11what's the best program designed
• 14:14to deliver ARV therapy?
• 14:17Both kinds of efficiency require good cost data
• 14:20in order to properly assess them.
• 14:22However, technical efficiency is only about costs,
• 14:24whereas allocative efficiency
• 14:26concerns health impacts as well.
• 14:29And then finally, perspective,
• 14:30and I'll spend a little more time here
• 14:31'cause I think this is a really key thing
• 14:34that we all need to be on the same level with.
• 14:37This is perhaps the most important concept.
• 14:41So different stakeholders
• 14:42have different experiences of cost.
• 14:45There's the societal perspective,
• 14:46which is what we consider the most inclusive
• 14:49and is often recommended for most policy analyses
• 14:54in low and middle income countries.
• 14:57In this case,
• 14:58all costs are considered regardless of who incurs them
• 15:01within a society.
• 15:03However, considering the perspective
• 15:06of specific stakeholders is particularly useful
• 15:09in helping to address those stakeholder concerns.
• 15:13So will a payer or an insurance provider lose money
• 15:17even though a society gains in net savings?
• 15:20Or should we align financial incentives
• 15:22and how do we go about accomplishing that?
• 15:26So here's another sort of useful graphic
• 15:29for thinking about perspectives
• 15:30and in the context of a cost effectiveness analysis,
• 15:33these are the areas in which you might consider your choice
• 15:35of inputs for costs and benefits
• 15:37for a programmer intervention.
• 15:40So you'll see that in the rows of the table here.
• 15:43So many decision analyses are gonna be taking a societal
• 15:46perspective to start,
• 15:49but let's drill down a little bit more
• 15:52where we're talking about the hospital perspective
• 15:55or a payer perspective.
• 15:57So this is sort of the narrowest definition.
• 16:01They'll have their own particular interests
• 16:03and we can also think about this being done
• 16:06from the perspective of an employer,
• 16:08how we might think about the healthcare system as a whole.
• 16:12The next level out would be the payer perspective,
• 16:19and that captures the insurer
• 16:22and that that may be a government entity.
• 16:25So Medicare, Medicaid,
• 16:27and then zooming out even further
• 16:30the societal perspective.
• 16:32And that captures, you know, this broader range of costs.
• 16:36Okay?
• 16:40I wanna mention then the question arises,
• 16:44which of these perspectives should we choose?
• 16:47So in implementation science,
• 16:51it often depends on the priorities
• 16:57of the those on the ground partners
• 17:01and the institutions that you're working with
• 17:02when you roll out one of these studies.
• 17:05So in low and middle income country settings,
• 17:08the guidance from traditional reference case examples,
• 17:13and I'll touch on those in a bit,
• 17:15tend to encourage including
• 17:17this broader set of perspectives.
• 17:20So the societal
• 17:21or the health system perspective.
• 17:24However, narrower perspectives
• 17:25are usually these are more directed at the interests
• 17:29of key stakeholders,
• 17:30and that may be the number one priority
• 17:33in any one of these examples.
• 17:37So it really, you know,
• 17:39in short it really depends on the priorities
• 17:41of your partner agencies
• 17:45and those who are rolling out
• 17:47whatever this intervention happens to be.
• 17:51You know, there's a really great quote
• 17:53from this paper
• 17:54by Eisman and colleagues
• 17:55that just came out last year,
• 17:57and really it's a pragmatic answer.
• 17:59"The perspective of the stakeholder and the decision makers
• 18:01who will be informed by the analysis
• 18:04should probably be prioritized."
• 18:06But you can also conduct analysis at multiple levels,
• 18:10you know, as long as you're able
• 18:11to distinguish between those different levels
• 18:14when you write things up.
• 18:15So just keep in mind in this process,
• 18:19the organizations that are adopting evidence-based policies,
• 18:22they really need to know what it's gonna cost them
• 18:24in their setting, given their resources.
• 18:28But then there's a trade off
• 18:29and that is whose reality counts
• 18:32and whose being missed
• 18:33by these types of analyses?
• 18:35I don't know if those on the call have ever read anything
• 18:39by Robert Chambers, but you know,
• 18:41I think a lot about
• 18:44one of his early books, "Whose Reality Counts,"
• 18:48when it comes to development interventions
• 18:49when I think about this idea of perspective.
• 18:55Another consideration that's closely related
• 18:58to perspective is timeframe.
• 19:00And so that is not just what costs and benefits
• 19:03should we collect, but like when and where
• 19:05in the project life cycle itself
• 19:07is it appropriate to collect 'em?
• 19:09So one option, you know, and we'll start
• 19:12with the first in that list there
• 19:13is to focus solely on the short term healthcare costs,
• 19:16which could include the costs of the intervention,
• 19:19costs of downstream stream healthcare.
• 19:22This approach is consistent with,
• 19:24I think, you know,
• 19:25most budget impact analyses, frameworks,
• 19:28particularly those mentioned
• 19:30in the CHEERS guidelines,
• 19:33which I'll provide some resources for later.
• 19:36And in the context of costing,
• 19:38this is usually on the order of about six months
• 19:41after a program's been implemented.
• 19:43So it's once that program is reached maturity,
• 19:46you know, what does it cost to continue to implement?
• 19:50But that ignores another set of questions,
• 19:52which is, you know, number two here,
• 19:55the intervention startup.
• 19:57And it might be extremely valuable,
• 19:58the organizations and key stakeholders
• 20:00who are carrying out
• 20:01that evidence-based program
• 20:04to include these costs
• 20:05in order to capture some sense
• 20:07of what it would take for a system
• 20:09to start to integrate new practices,
• 20:10how long it might take
• 20:11for those systems to reach maturity.
• 20:15Another option,
• 20:16the third in that list
• 20:18is to include additional research costs,
• 20:21research costs included in the design and development
• 20:24of implementing a new evidence-based practice.
• 20:28So that might be adapting that evidence-based practice
• 20:33to a local setting or an intervention.
• 20:35And it could also include some of the costs of say,
• 20:41monitoring and getting feedback
• 20:42from participants in order
• 20:44to adapt and to improve,
• 20:49I guess the implementation of that process.
• 20:52And then finally,
• 20:53there's this question
• 20:53of future downstream costs and benefits.
• 20:55And that's particularly to patients
• 20:57and to program beneficiaries.
• 20:58So that's number four in the image that you see.
• 21:01And this is often included, you know,
• 21:03as I mentioned before,
• 21:04in perspectives that are taken
• 21:05from the societal level,
• 21:08but all too often they fall outside
• 21:12of the scope of interest
• 21:14for implementing partners
• 21:15and particularly for payers.
• 21:17You know, and I'll just mention
• 21:18that this is one of the main major downsides to many,
• 21:21if not most economic evaluations
• 21:24that are conducted domestically.
• 21:25So when it, you know,
• 21:26this is a setting where the payer
• 21:28is often a private health insurer who has no monetary stake
• 21:31in the future health state
• 21:32of their clients because
• 21:34in most cases those customers might age out
• 21:39and move into a new health insurer,
• 21:40so Medicare, for example, when you turn 65.
• 21:44So I'll just mention that an analyst may choose
• 21:46to include any number of these different components
• 21:49of an intervention or these different parts
• 21:51of the timeframe.
• 21:52But in any case,
• 21:53it's really essential to be able to distinguish
• 21:55between these different sets of costs and benefits
• 21:57and to be explicit about the scope,
• 22:00particularly for generalizability
• 22:02and understanding limitations.
• 22:05So lemme go into costs and costing,
• 22:07I've kinda droned on for a long time here,
• 22:10but let's talk about what costs are.
• 22:15So there's definitely,
• 22:16there's many different sources of costs
• 22:19from donors and government providers,
• 22:21international and local NGOs,
• 22:23participant costs from households and individuals.
• 22:27There's also many different ways to describe
• 22:29and categorize costs,
• 22:31say by the type of costs.
• 22:33Are they real world costs or are they costs
• 22:35of a randomized control trial?
• 22:37Are they full or incremental?
• 22:41Or by cost categories,
• 22:43so different types of inputs
• 22:45or organized by activities.
• 22:48Or, you know, as I mentioned even by the timing of costs.
• 22:52If you remember my first major takeaway
• 22:53from the presentation,
• 22:55you should be starting to get an idea
• 22:56of why I say
• 22:57that there was my second takeaway, rather,
• 23:01you start to get an idea here
• 23:03of the notion that expenditures exist,
• 23:06but that costs,
• 23:07they're a little bit fuzzier.
• 23:09They're frequently abstract
• 23:10and they depend on a lot of different factors.
• 23:14So two different kinds of costs to be able
• 23:17to distinguish between are economic and financial costs.
• 23:22So financial costs are those appearing potentially
• 23:25in the program expenditures documents.
• 23:30Meanwhile, economic costs are costs
• 23:33that are included or that include
• 23:35the value of all resources,
• 23:37regardless of who's paying.
• 23:38So the difference here is resources contributed
• 23:42by other entities.
• 23:43So for example, the local hospital,
• 23:46the clinical staff,
• 23:47volunteers or subsidized costs,
• 23:51such as from an international agency paying for commodities.
• 23:56So note that when you see an economic costing exercise
• 24:01referred to in the literature,
• 24:03it typically subsumes financial costing
• 24:06when it's, you know, thought of in this Venn diagram.
• 24:13So we have a sense of what costs are
• 24:16and why they're important,
• 24:17but what exactly is costing.
• 24:19So we use this term costing as a shorthand
• 24:22to describe the estimation of the costs
• 24:25of producing health services, for example.
• 24:29So costing places of value on the total resources used
• 24:32to produce a good or service.
• 24:34It requires measurement of the amount of each resource
• 24:37as well as information about the price
• 24:39of each of those resources.
• 24:42So costs can vary by context.
• 24:45So when one is costing,
• 24:48it's important to gather information about location,
• 24:50about time period, population, and a host of other factors.
• 24:55Some of the different types of resources
• 24:57that are used to produce health services
• 24:59might include human resources,
• 25:03drugs and supplies,
• 25:04medical or other non-medical equipment
• 25:08or even patient costs.
• 25:10So what exactly do we use costing for?
• 25:13Well, costing is used
• 25:14in a range of research evaluations, programs,
• 25:18and in planning,
• 25:20primarily it's in the areas of financial planning
• 25:23and budgeting and priority setting
• 25:25that cost data is the most sort of visible we'll say.
• 25:30So for example,
• 25:31cost data are used to evaluate the effectiveness
• 25:34of an intervention or to understand
• 25:36the efficiency of health delivery,
• 25:39but what are the cost drivers
• 25:41and how do these vary over time?
• 25:43So in in ART,
• 25:44which is the example in the slide here,
• 25:48this figure is from a systematic review
• 25:51of the costing literature.
• 25:52This is one that I'll touch on later
• 25:54for antiretroviral treatment
• 25:56in low and middle income countries,
• 25:59it shows the different drivers
• 26:01of the average unit costs across five different settings.
• 26:06And this helps to show
• 26:07which cost inputs contribute most to the overall unit cost.
• 26:12So this figure I think does
• 26:13a really great job of showing the absolute difference
• 26:16in unit costs between settings.
• 26:18And in this case it's it's by country
• 26:20and it includes a lot of different settings
• 26:22within countries and doing so over time.
• 26:28Oops, excuse me.
• 26:34So cost data are also used, you know,
• 26:36as I mentioned in efficiency analyses.
• 26:39So for example,
• 26:39this might be to examine how costs vary
• 26:43with different levels of service delivery.
• 26:45So understanding unit costs at different coverage levels
• 26:49is really important to predicting costs as programs tend
• 26:52to scale up or to grow.
• 26:54So scale is a really important component here.
• 26:57Now these dynamics can be explored
• 27:00through the use of cost functions.
• 27:02So here the figure shows
• 27:04this sort of interesting dynamic relationship
• 27:06between scale and unit cost for antiretroviral treatments
• 27:10across a large number of sites
• 27:12in sub-Saharan African countries.
• 27:14So you'll note that the relationship here isn't linear.
• 27:18It's a little noisy,
• 27:19but that makes it important to estimate cost functions
• 27:22and then to utilize different costs
• 27:24for different levels of coverage.
• 27:26So in this case we can see
• 27:28some suggestive evidence of economies of scale,
• 27:32which is to say that per patient ART services
• 27:36within facilities with larger numbers
• 27:39of annual patients tend
• 27:40to be slightly cheaper than facilities
• 27:42with fewer annual patients.
• 27:46So one thing to remember here
• 27:48is that although there's a wide range
• 27:50of different analyses that need cost data,
• 27:52they don't always use or need the same types of data.
• 27:58The purpose of the analysis
• 27:59in this case sort of dictates
• 28:01the type of cost that is required.
• 28:03And so it's really important
• 28:06that the purpose be identified at the outset.
• 28:10So, you know,
• 28:11we have to consider budgeting, forecasting,
• 28:14efficiency analysis, priority setting, among others.
• 28:21Move forward.
• 28:22So before we go into costs further,
• 28:25I think it's also important to get some terminology right.
• 28:29In addition to the types of costs,
• 28:32there's also different measures of cost.
• 28:34So being clear about what each means, I think,
• 28:37is pretty central to understanding
• 28:41and interpreting cost information better as a field.
• 28:44So here are four different types
• 28:46of costing terms that you're probably familiar with
• 28:49or, you know, you may have encountered some,
• 28:52but what do they mean?
• 28:53So the first is total cost,
• 28:55which is as you'd imagine the total cost
• 28:58of producing a service.
• 29:00And in healthcare this is often presented as an annual cost.
• 29:03For example, the total annual cost of VMMC services
• 29:07at a clinic in a particular setting.
• 29:11The second term is average unit cost.
• 29:14And so this is the total cost per unit of output.
• 29:17And you'll notice that the units of output
• 29:19can be measured in different ways,
• 29:21even within the same service.
• 29:23So for example,
• 29:24there's a cost per person contacted
• 29:26or a cost per person treated.
• 29:28And I'll return to this a little later.
• 29:31The third term is marginal cost,
• 29:32which is a concept that's frequently used in economics
• 29:35and it's really critical to efficiency analysis.
• 29:38And this is the additional cost of producing
• 29:40just one more unit of output.
• 29:42So for example,
• 29:44the cost of testing one more person
• 29:46or vaccinating one additional patient.
• 29:49And then the final term here is incremental cost,
• 29:53which also examines change.
• 29:55In this case, however, the focus is on the cost
• 29:58of adding a completely new level
• 29:59or type of service rather
• 30:00than adding one more unit of output
• 30:03to an existing service.
• 30:04So for example,
• 30:05the additional cost of adding counseling
• 30:08and linkage to care
• 30:09to current HIV testing.
• 30:12Most costing exercises though that you'll encounter tend to
• 30:16refer to either average or to unit costs.
• 30:21I wanna give you just a quick example here
• 30:23of what a program costing table might look like.
• 30:27So this slide comes from a presentation by Dr. Jim Kahn,
• 30:31who is a mentor of mine at Cal.
• 30:36The example here is a small rural HIV clinic
• 30:41that was set up by a colleague of Dr. Khan at UCSF,
• 30:46which I'll have to say
• 30:47is where he spent most of his career.
• 30:51And the two in this case conducted
• 30:55a costing analysis of this facility.
• 30:58So as you can see,
• 30:58the summary divides costs
• 31:00into four different categories.
• 31:02There's personnel at the top,
• 31:05antiretroviral medications,
• 31:06which are the dominant supply
• 31:08that were used by the program,
• 31:10lab tests, which is the main service that was provided
• 31:14by the program,
• 31:15and then there's this fourth sort of residual category
• 31:18of less expensive items.
• 31:20So administration, general supplies,
• 31:23vehicles, storage, utilities.
• 31:26You might find pencils in here.
• 31:29So the standard input categories here
• 31:31were adapted to conform
• 31:32to how clinicians think about major components
• 31:35of antiretroviral treatment.
• 31:38In this case, the columns show the units
• 31:40that were tested,
• 31:41the costs per unit,
• 31:44the resulting monthly cost for the entire activity
• 31:47and the entire costs per patient
• 31:50for each line item.
• 31:52So in this case, for example,
• 31:54the full-time doctor costs $1083 per month
• 31:58and $52 per patient year.
• 32:02So one interesting takeaway here
• 32:04is that the authors found
• 32:06that once the clinic had reached fully operational capacity,
• 32:10it was able to deliver an ART
• 32:12at it looks like about $550 per patient per year.
• 32:16And they also found, which isn't shown here,
• 32:18that the cost could drop by about a third
• 32:21with less expensive medications
• 32:24as well as lower wage scales
• 32:25and modest increases in patient load.
• 32:28So just an interesting example
• 32:31to kind of put some of this into context.
• 32:34A couple of key costing principles here.
• 32:37And these are major takeaways
• 32:38I think from Jim as well.
• 32:41But in theory you can do some really solid costing work
• 32:45if you adhere to really these three main principles.
• 32:48So the most important is that your goal
• 32:50is to quantify the resources required
• 32:52for program operation and the associated cost,
• 32:56and that means developing an inventory of resources
• 33:00and assigning appropriate costs.
• 33:02So this task is quite different
• 33:05than simply accepting a program's budget
• 33:07or their expenditure report
• 33:09because that report might include costs
• 33:11that aren't required for routine program operations,
• 33:13like special evaluation needs of a funder.
• 33:17Also, even if the costs are accurate,
• 33:20you've likely learned nothing
• 33:21about how the program operates.
• 33:23So specifically, what types of inputs might be available?
• 33:27The number two point, you know,
• 33:29that Jim makes often is that you should be systematic
• 33:33and thoughtfully thorough.
• 33:35And by systematic, you know, we mean
• 33:38that the search for resources
• 33:40should include all of the usual input categories
• 33:42like personnel and equipment,
• 33:45the range of activities, such as marketing
• 33:47and service delivery.
• 33:49But, by being thoughtfully thorough, you know,
• 33:53what Jim refers to is that all of the significant resources
• 33:57should be included, quantified and costed.
• 34:00But minor costs,
• 34:02truly minor things like routine stationary supplies
• 34:05or routine services, pencils and erasers,
• 34:10they can often be safely ignored if the data collection
• 34:13in order to obtain those is too onerous.
• 34:16So of course that's like a judgment call.
• 34:19And one suggestion, you know,
• 34:21that Jim makes is to use a table
• 34:24and kind of order resources
• 34:26from the most to the least expensive
• 34:28so that you can empirically identify,
• 34:30say a cutoff for each of the resources
• 34:33that are considered maybe too minor
• 34:35or that may not play into the overall costs
• 34:39as much as you might be concerned about.
• 34:41So, you know, this is really getting into fine, fine detail.
• 34:44Do you really need to cost that last pencil?
• 34:48And then finally, you know, as we mentioned before,
• 34:51adopt the appropriate perspective.
• 34:53So I've talked about, you know, this at some length,
• 34:58but it's worth reiterating.
• 35:04I'll go over, I think I might skip past some
• 35:06of the typical cost and categories except to say, you know,
• 35:10these are some examples they may often
• 35:14go as far as to include patient costs.
• 35:18They may also vary by program and over time.
• 35:26The timeframe, you know, which I mentioned earlier,
• 35:29typically when we think about a costing exercise,
• 35:32it's about 12 months or it's multiples thereof.
• 35:36And you know, the reason for this
• 35:37for annual costing
• 35:38is that it has the hallmark of, you know,
• 35:42conforming to our pervasive and programmatic record keeping,
• 35:45which is really nice,
• 35:46but it can also help to deal with distortions,
• 35:50seasonal effects and shocks.
• 35:51It's smooths as a lot of random variation.
• 35:54So that can be for like the calendar year
• 35:56or the fiscal year, typically.
• 35:59Portraying costs by month,
• 36:01if it's easier,
• 36:01might indicate how costs vary over the course of a year.
• 36:04So that's another consideration.
• 36:07And you know, again,
• 36:09if we're talking about fully operational programs,
• 36:12we wanna make sure that the program is actually matured,
• 36:16but again, that might ignore some of the startup costs
• 36:20or other costs that I discussed earlier.
• 36:23One other little piece of terminology that I want to cover
• 36:27is gross versus macro, and top down versus bottom up.
• 36:30So you might have come across this a number of times,
• 36:32you know, in proposals or mentioned
• 36:34sort of casually in the literature,
• 36:37micro-costing focuses on really granular accounting
• 36:40of inputs as opposed to to macro-costing,
• 36:45which is, or sorry, gross costing.
• 36:50I always get them confused.
• 36:53A gross costing approach that might simply estimate
• 36:55all of the relevant costs,
• 36:57typically from program expenditure data,
• 36:59and then dividing the associated outputs.
• 37:02So like patient episodes.
• 37:05In contrast, we've got the bottom up or top down,
• 37:08which refers to the way in which each resource is allocated
• 37:12to the total unit cost being estimated.
• 37:14So top down costing divides
• 37:17overall program costs or expenditures,
• 37:20often including those,
• 37:22which are above the service delivery level
• 37:27by the number of outputs in order
• 37:28to calculate the unit cost,
• 37:31whereas bottom up costing
• 37:33measures input quantities at the client or activity level.
• 37:38So gross costing, for example,
• 37:39is commonly done top down, for example.
• 37:45And then micro-costing kind of has a bottom up.
• 37:50I'll skip over.
• 37:51Well, let's see, Debbie, we're at 43 after.
• 37:57I wanna mention pricing and valuation very briefly
• 38:01before moving into some of our other findings
• 38:05simply because I think there's one example
• 38:09in the popular press right now
• 38:12that's really worth consideration,
• 38:13particularly if you're operating
• 38:15in low and middle income countries.
• 38:16And I'll just point to the fourth bullet here, inflation.
• 38:21So inflation comes up a lot
• 38:25in the popular press
• 38:26because we're experiencing rapid inflation
• 38:28in the US right now,
• 38:30but I wanna linger on it for a second, simply to say,
• 38:34so the operation that you use
• 38:40in order to account for inflation
• 38:43in different country settings matters a great deal.
• 38:47So the example here is,
• 38:48let's say that you've collected
• 38:51a piece of critical cost data
• 38:52from a program in Tanzania,
• 38:54the unit cost of a service
• 38:55in the year 2000 to say a $100.
• 38:59So, you know, the typical strategy
• 39:02is to use a US price inflation tool,
• 39:08so typically the GDP price deflator
• 39:12and convert that into US dollars
• 39:15in order to inflate over time.
• 39:18There's some recommendations in the literature
• 39:21that this should be done
• 39:22in a different order of operations,
• 39:24and that would be to actually inflate
• 39:26in local inflation before converting into US dollars.
• 39:31And the implications for that.
• 39:32So in the next slide here are pretty substantial.
• 39:36So, you know, in Tanzania over that same period,
• 39:39inflation was quite rampant and so it it really changes
• 39:43the overall dollar amount
• 39:46that you're looking at
• 39:47in the last period of the cycle there.
• 39:50And it's even more volatile
• 39:52when you include other countries
• 39:54in sub-Saharan Africa, period.
• 39:57So, your choice of inflating first before exchanging
• 40:02into US dollars or exchanging into US dollars
• 40:05and then using a US inflation process
• 40:08can have some really substantial implications.
• 40:12I just wanted to highlight that.
• 40:13I'm gonna push through some of these final notes here
• 40:19because I'd like to get to some of our findings.
• 40:23I do wanna leave you with three key resources at the end,
• 40:26and those are the consolidated health economics
• 40:29evaluation reporting standards,
• 40:30or CHEERS standards.
• 40:31There's a new update to those standards
• 40:33that was published in 2022.
• 40:35And then for those working
• 40:37in low and middle income country settings,
• 40:39Vasal and colleagues reference case
• 40:42for estimating the cost
• 40:44of global health services and interventions
• 40:46and the Bill and Melinda Gates Foundation 2014 guidelines
• 40:49that come through their reference case.
• 40:52And I'm happy to share those resources after the talk,
• 40:55but let me get into costing and practice here
• 40:57because I am running quite low on time.
• 40:59So the Global Health Cost Consortium,
• 41:02as Debbie mentioned at the top
• 41:03of the hour was funded
• 41:05by the Bill and Melinda Gates Foundation.
• 41:09I was involved with this project for about four years,
• 41:12and two of the outputs from the project
• 41:16were our unit cost study repository,
• 41:18which I'd invite you to visit.
• 41:21And that's really a one-stop shop for all
• 41:24of the published cost data
• 41:26from HIV and TB interventions
• 41:28that were conducted
• 41:29in low and middle income countries settings,
• 41:30really over the last 20 years or so.
• 41:33So we went through a pretty long painstaking process
• 41:36of extracting primary cost data,
• 41:39that is costs that were not modeled
• 41:43from the published literature,
• 41:45standardizing them across
• 41:48several hundred different variables,
• 41:51and then making them sort of available to the public
• 41:56at large in a pretty easy
• 41:58to use dropdown type of database.
• 42:01So if you're ever doing any program planning
• 42:05for HIV or TB interventions,
• 42:07I'd invite you to visit that site
• 42:10and see if you can get any utility
• 42:11from some of the efforts that we put forward.
• 42:14Another big output was the reference case
• 42:16that I mentioned before,
• 42:18guidelines for estimating costs
• 42:19for Global Health Interventions.
• 42:22And again, happy to share those after the talk.
• 42:26Another major output here was we were able
• 42:29to produce several different publications.
• 42:33So I'll present a little bit of the findings from that.
• 42:36So over several years, our team
• 42:38conducted the systematic search
• 42:40and screening of peer reviewed and grey literature.
• 42:43We had a few different goals here.
• 42:46We wanted to describe the quantity and characteristics
• 42:48of public published costing data
• 42:51to identify production patterns over time,
• 42:54so geographic location,
• 42:56publication venues like journals
• 42:58and authorship and to look into key methods
• 43:02and reporting standards to try to identify gaps
• 43:05in the costing literature.
• 43:07So I'll go over some of those findings.
• 43:09So figure one here shows the results
• 43:12of that search and screening process.
• 43:15So we covered articles that were published starting
• 43:18in January, 2006 through October, 2017.
• 43:23So it's a couple years old now.
• 43:26We use six databases,
• 43:27so PubMed, M Base, Web Science, Cochrane,
• 43:32and a number of others.
• 43:34I'm gonna give you high level content here.
• 43:37So searching also included grey literature,
• 43:40which you see on the right hand side.
• 43:42And we identified over 23,000 possible titles,
• 43:46including nearly 500 known costing studies
• 43:48that we had started with.
• 43:50So from these we were able to screen down
• 43:52to 217 relevant titles
• 43:54that got included in that unit cost study repository,
• 43:58that database that I mentioned before.
• 44:01And of those we published one study among 159
• 44:11published studies that had taken place
• 44:12in Sub-Saharan Africa.
• 44:14So I'm gonna discuss really briefly two different studies
• 44:18that came out of the effort.
• 44:19And the first was that 2019 study that I just described.
• 44:22And the second is a study that's in progress
• 44:24where we're reviewing findings
• 44:26from the detailed extraction
• 44:28of the 243 studies that we started with
• 44:32in the lines above.
• 44:34So I might jump a little bit between findings
• 44:36in these two different venues.
• 44:37So the first study focused on that evidence
• 44:41from Sub-Saharan Africa.
• 44:42This gave us a really nice picture
• 44:44of where costing data is and isn't collected
• 44:47on the continent.
• 44:49The studies we identified took place in 25 countries,
• 44:52so mostly East Africa and Southern Africa.
• 44:55And although this sort of geographic variation
• 44:58is broad, you know,
• 44:59we found that the quality of evidence
• 45:01within many of these countries is,
• 45:03or the quantity rather of evidence within these countries
• 45:05is quite sparse.
• 45:07So west Africa doesn't have
• 45:10a great deal of primary cost data
• 45:13in the published literature.
• 45:14A lot of it, you know, is often modeled.
• 45:19In terms of who is publishing studies
• 45:21that report on HIV intervention costs.
• 45:24There's a few notable peer-reviewed journals.
• 45:26So PLoS One, J AIDS and AIDS are kind of our leaders.
• 45:33But I think the big takeaway here
• 45:35is that there's an awful lot
• 45:36of potential destinations
• 45:38for published primary cost data
• 45:42and more so than I think we realized
• 45:45when we set out in this exercise.
• 45:48So you can see the large list
• 45:50of different published studies on the left
• 45:53or sorry journal destinations
• 45:55for published studies on the left
• 45:58and the table in the right.
• 46:00These are grey literature resources.
• 46:02So these are the sources of grey lit,
• 46:05a few key players here, USAID, PEPFAR,
• 46:09the Futures Group, Health Policy Project
• 46:12and PANGEA and Bill and Melinda Gates
• 46:15had their fingers
• 46:16in a lot of these different published studies.
• 46:19But these are the destinations.
• 46:23One thing that I do wanna mention here
• 46:25is that although we're examining
• 46:26this subset of primary cost data that was collected
• 46:29from studies in Sub-Saharan Africa,
• 46:32there's a little bit of a disconnect here
• 46:33in terms of where we find that primary cost evidence
• 46:37is being published and where leaders in the field,
• 46:40at least nine years ago
• 46:42when the first CHEERS guidelines paper was published,
• 46:46that there's, you know,
• 46:48in that paper there was an explicit effort
• 46:53to identify leading journals as landing places
• 46:57for health economic evaluations.
• 46:59But what we find is that of the 10 that they identified
• 47:03in that CHEERS statement paper,
• 47:05only three were we able to find
• 47:08in our larger systematic search of the HIV literature.
• 47:12And this is, you know, beyond Sub-Saharan Africa.
• 47:14This is that paper too that I mentioned.
• 47:17So this isn't necessarily an indictment of the list
• 47:20that was generated by the CHEERS authors,
• 47:23but it does highlight,
• 47:24I think a large and varied number of journal sources
• 47:27where these data are published.
• 47:29And it emphasizes the need to be sure
• 47:30that these standards, you know,
• 47:32like the ones laid out by the CHEERS document are being
• 47:35upheld across different publications.
• 47:39So, you know, a large growth in publications over time,
• 47:44a difference in the time to publication.
• 47:46So peer reviewed studies tended to take 2.8 years on average
• 47:51to get published,
• 47:53whereas those in the grey literature 1.4 years.
• 47:56So this is much more quickly disseminated
• 48:01into the zeitgeist, you know,
• 48:04the ability for folks to actually start
• 48:06to use these data
• 48:07for their own program planning purposes.
• 48:13Scale, I'll mention.
• 48:14This is probably our most substantial finding
• 48:17and you don't need to spend
• 48:18a lot of time pouring over this table,
• 48:19but this was actually quite damning.
• 48:22If you don't take anything else away
• 48:24from this presentation,
• 48:25I think, I hope that that this resonates,
• 48:29that reporting of scale
• 48:31is, you know, it's a really critical, important component
• 48:34of any costing exercise.
• 48:36But what we found is that in, you know,
• 48:40the six of the intervention types of all the studies
• 48:44through which we were able to collect data,
• 48:46the average number of unit costs
• 48:50in those published studies,
• 48:54scale was reported below 50% of the time.
• 48:58And in some areas as low as 35 or 26% of the time.
• 49:04So in total, I think the average, you know,
• 49:06that we found was something
• 49:08like 44% of unit costs have
• 49:11explicit measurement of scale associated
• 49:14with those reports.
• 49:16So that's a problem,
• 49:19that's something that needs to improve in the field.
• 49:22And then I'll also mention perspectives.
• 49:24So despite I think recommendations and urging from say,
• 49:30the Gates Foundation's reference case,
• 49:32which I cited earlier,
• 49:34which emphasizes that authors should focus
• 49:36on societal perspectives as much as possible,
• 49:39we actually find here in terms
• 49:41of what folks in these articles report
• 49:46as their given perspective
• 49:48and when we clean it up and we, you know,
• 49:51we figure out among those who didn't report
• 49:53what their perspective was, by the way,
• 49:5515.6% of of studies
• 49:58that we uncover don't report a perspective,
• 50:00which is again, a problem for reporting standards,
• 50:05only seven or about 2.9% of the studies
• 50:08that we identified take a societal perspective.
• 50:12So, you know, most of this is done
• 50:13at the level of the provider.
• 50:15So, you know, it gives you a little bit of context,
• 50:17I think for where there's room for growth.
• 50:21So in conclusion, on these points, you know,
• 50:24there's, I wanna say that there's, you know,
• 50:27a few final recommendations.
• 50:31We recommend that that future costing
• 50:32and cost effectiveness studies closely follow
• 50:35the Gates and the GHCC reference case.
• 50:40And you know,
• 50:42that there needs to be more detailed reporting
• 50:45on a number of different items
• 50:47that I wasn't able to touch on in this talk.
• 50:50But, you know, most of all,
• 50:51I think scale is something that's really missing
• 50:54in the literature.
• 50:56So I won't touch on this too much
• 51:00other than to say there's some future research
• 51:01that we're working on
• 51:03to determine the quality of some of these cost data.
• 51:05So be on the lookout for publication on that in the future.
• 51:10And now that I've ran through all of our Q and A time,
• 51:13I do wanna thank you for attending
• 51:15and I'm happy to take any questions.
• 51:20<v ->Yeah.</v>
• 51:23Thank you for the great talk.
• 51:24So I had a question,
• 51:25comment about, you know, the societal perspective.
• 51:28I think in certain scenarios or situation,
• 51:31this could be a little bit tricky.
• 51:33For example, if you take vaccine
• 51:35for some infectious disease in low income countries,
• 51:39usually the price is heavily negotiated
• 51:42between the pharmaceutical companies and donors and NGOs.
• 51:47And if you take the societal perspective,
• 51:49when you are doing the cost effectiveness analysis,
• 51:52including, you know, the productivity loss,
• 51:55the income loss, the vaccine becomes extremely,
• 52:00you know, cost effective or even cost saving.
• 52:03So even if they price the vaccine
• 52:05to $50, $100,
• 52:07still it becomes cost saving, which is, you know,
• 52:10it becomes an excuse in hands of, you know,
• 52:14the pharmaceutical companies to jump up the price of vaccine
• 52:17because such a cost effective strategy.
• 52:19So I dunno if you have thought about that.
• 52:21I usually, I have purposefully avoided
• 52:24taking societal perspective in those contexts
• 52:27because I just don't want to put these numbers out,
• 52:30but I don't know if you have comments, thoughts.
• 52:34<v ->Yeah, no, I mean that's a great point.</v>
• 52:37I often don't think far enough along the causal chain
• 52:40to consider sort of the potential negative externalities
• 52:43of price gouging, you know, in those contexts.
• 52:49I guess I'll say, you know,
• 52:52at least in the context of what we found, you know,
• 52:55in our search of the literature and you know,
• 52:57there were a few of those different intervention types
• 52:59that dealt with pharmaceuticals,
• 53:02you know, and the costs of those being, you know,
• 53:04not insignificant drivers of overall program
• 53:08and unit costs, you know, for HIV care
• 53:12that most of the studies that we identified,
• 53:20you know, really do focus in, you know,
• 53:22don't tend to focus in on the societal level.
• 53:25So it's not, you know,
• 53:27perhaps that that type of crisis
• 53:30hasn't borne out necessarily
• 53:31as a result of some of this in the literature,
• 53:34but it's definitely not something that's discussed
• 53:36as often as I think it should be.
• 53:37So you raise an excellent point and, you know,
• 53:39and one that I hope, you know,
• 53:41gets added to further discussions
• 53:43in published literature and forums like this.
• 53:49So I don't have a good, you know,
• 53:49I don't have a good anecdotal example to share,
• 53:54but I really, yeah, I like the point.
• 53:56Yeah. Jeremy.
• 53:58<v ->Yeah, Drew, thanks so much.</v>
• 53:59This was really informative.
• 54:02I actually wonder maybe building
• 54:03on the previous question,
• 54:05if you could go back to the slide with the table
• 54:08from that rural Uganda study,
• 54:10because my question was about sort of was this a financial
• 54:15or economic perspective that was being presented
• 54:18and I was honing in on the ART
• 54:20because presumably, you know,
• 54:22through PEPFAR from the programs perspective,
• 54:25ART should have been without cost
• 54:29and also without cost to the patient.
• 54:31So then, so which made me think,
• 54:33well maybe this is an economic perspective,
• 54:36but then there are other,
• 54:37all those other pieces of deliver of getting the medicine
• 54:39to the patient,
• 54:40the whole supply chain and whatnot.
• 54:42And so I was just wondering if you have a sense of like,
• 54:44was this trying to describe programmatic
• 54:49or economic perspectives
• 54:51and, you know, when, if this is economic,
• 54:54should we truly be thinking about like that whole process?
• 54:57Like where in the supply chain do you start counting?
• 55:01<v ->Yeah, you know, I wasn't involved</v>
• 55:04in this particular study
• 55:05and so I'm happy to field that question to Jim
• 55:07'cause he uses this example in his talks.
• 55:10My sense is that this is programmatic, you know.
• 55:13It does seem fairly narrow in scope,
• 55:16but you do, you raise a really good point
• 55:17I think, you know, that Reza raised as well,
• 55:19which is, you know,
• 55:21it depends on who's providing that,
• 55:24where the supply chain is coming from
• 55:27and you know, and that obviously
• 55:28has downstream implications
• 55:29for, you know, how you deal with the costing procedures
• 55:35in, you know, these different income country settings,
• 55:39how you'd inflate and how you'd adjust.
• 55:42And so supply chains don't often make it into the discussion
• 55:47in these write-ups, but they need to, right?
• 55:49Because that's a really critical component.
• 55:53I don't know if that answers your underlying question.
• 55:56<v ->Yeah, I think it gets to it for sure.</v>
• 55:59<v ->Okay.</v>
• 56:00Yeah.
• 56:00And you know, like I said,
• 56:01I'm happy to reach out and and find out
• 56:03a little this particular example,
• 56:05but it's question.
• 56:06<v ->Great, thanks.</v>
• 56:08<v ->Yeah.</v>
• 56:09I do realize we're at the top of the hour.
• 56:11I'm not able to see the chat very well.
• 56:15So if there's anyone that would like to raise anything
• 56:17from there that I'm missing,
• 56:19I'm happy to field those.
• 56:21<v ->Drew.</v>
• 56:22I would just say that before David Palteo ran away,
• 56:26he said that for what it's worth,
• 56:29his concern is the mirror opposite of Reza's.
• 56:32When we abandon the societal perspective,
• 56:35interventions that have broad public benefit
• 56:38can be greatly undervalued
• 56:40because they're not germane to the narrow
• 56:41decision maker and real economic resource use
• 56:45such as volunteer time and effort
• 56:47can be ignored because it looks free
• 56:49to the narrow stakeholder.
• 56:51So he's, and I would also say
• 56:54that my question,
• 56:55and this is one not as an economist
• 56:58that I've wondered about
• 56:59for a couple of years,
• 57:01is that from an equity perspective,
• 57:04I actually wonder about the household level perspective
• 57:07as opposed to the individual level perspective.
• 57:13I've never seen one of these presentations
• 57:15that looks at the household level
• 57:18and yet it's at the household level
• 57:20that cushioning of resources
• 57:23and equity balances often happen
• 57:26not at the individual level.
• 57:28And so I'm just curious from your experience,
• 57:31have you ever seen work done at the household cost level?
• 57:37And what do you think
• 57:38about those as an equity perspective?
• 57:41<v ->Yeah, I mean I think there's,</v>
• 57:43I think there's some downstream costs there that, you know,
• 57:45in theory would get
• 57:46some subsumed into a societal perspective.
• 57:50But I struggle to think of specific examples
• 57:53that kind of cast the umbrella wide enough
• 57:56or cast a net wide enough
• 57:57to include other individuals within the household.
• 58:01So I think that's a really,
• 58:03that's a really great point.
• 58:05You know, and it speaks
• 58:06to the demand side part of this question
• 58:12in a way that's really, you know,
• 58:14potentially quite meaningful.
• 58:16And I would tend to think that as you suspect, you know,
• 58:23we'd be moving towards even greater downstream benefits
• 58:28at the household level.
• 58:29So I am working on one study right now
• 58:31in Uganda actually
• 58:32with Jeremy who I think might have jumped off
• 58:35where we are investigating patient costs
• 58:40as a component of that intervention
• 58:43and we do tend to ask in the scope
• 58:46of collecting patient costs data about opportunity costs
• 58:52that are germane to the rest of the household.
• 58:53So we'll often ask questions about household expenditures
• 58:59and household income generating activities
• 59:03as well as durable assets of the household
• 59:07and ways that households can sort of deal
• 59:09with shocks over time.
• 59:12'Cause those are all really important questions
• 59:13I think that play into, you know,
• 59:16what the experience of a patient is like,
• 59:18particularly in a lower middle income country setting
• 59:20where one of these interventions
• 59:21might have a really meaningful downstream impact
• 59:24on, you know, on the rest of the household.
• 59:27So it's something that I spend a lot of time thinking about
• 59:29'cause I kind of started in that world I think
• 59:31before coming into the economic evaluation world.
• 59:34And it's one that I think doesn't currently receive
• 59:37a great deal of attention, but ought to.
• 59:40But again, to, you know, to Reza's warning,
• 59:42I think you have to be careful about, you know,
• 59:45how much generalizability you pull,
• 59:49and try to tout from different studies like this
• 59:51because, right, they have different implications
• 59:55for different parties
• 59:56And so, you know,
• 59:57I'd like to see a standard,
• 60:00not where one perspective is chosen
• 01:00:01over another ad nauseum,
• 01:00:04but where they're both incorporated
• 01:00:05and where there's a societal
• 01:00:07and a financial or a programmatic perspective presented
• 01:00:11within the same paper, you know.
• 01:00:14And readers of those pieces of evidence
• 01:00:17can then decide for themselves,
• 01:00:19like which is the perspective
• 01:00:20that's the most meaningful?
• 01:00:21What are the trade-offs, you know, along the way?
• 01:00:24'Cause it, you know, the value add of that is substantial,
• 01:00:28to add in a societal perspective piece
• 01:00:30to a publish economic evaluation.
• 01:00:33<v ->So I think we do need to wrap up.</v>
• 01:00:35But thank you Drew.
• 01:00:37I think that's very helpful
• 01:00:39and we will have the video available for people.
• 01:00:45And really appreciate your time
• 01:00:48in preparing all of this material
• 01:00:50and sharing it with all of us.
• 01:00:54<v ->Appreciate the invitation</v>
• 01:00:56and thanks to those attended today.
• 01:00:58It's nice to see some faces.
• 01:01:01<v ->Okay.</v>