YSPH Biostatistics Seminar: “Does Real-World Evidence have a Role in Precision Oncology?”December 07, 2021
Brian P. Hobbs, PhD, Associate Professor, Department of Population Health, University of Texas at Austin
December 7, 2021
To CiteDCA Citation Guide
- 00:00(people chattering)
- 00:10<v Man>(indistinct) Biostatistics</v>
- 00:13at the University of Minnesota.
- 00:15And he's currently attending or an associate professor
- 00:19at the University of the Texas Dell Medical School.
- 00:23Dr. Hobbes is a library recognized as an expert
- 00:27in clinical oncology and research (indistinct).
- 00:31Among his many accomplishments,
- 00:35in 2017, Dr. Hobbes (indistinct)
- 00:40of The National Cancer Institute, clinical trial
- 00:44(indistinct) a national consensus recommendations
- 00:50or (indistinct).
- 00:55In 2019, Dr. Hobbes (indistinct).
- 01:08In 2021, Dr. Hobbes (indistinct)
- 01:47<v Brian Hobbes>Thank you, thank you very much,</v>
- 01:48and for that long and generous introduction.
- 01:54I'm excited to give this talk today.
- 01:55I wish I could visit in-person.
- 01:58I was fortunate to have that opportunity a few years ago,
- 02:00so thank you for inviting me back.
- 02:05Okay, so I got tired of giving talks
- 02:10that were very technical and very specific
- 02:13to a specific problem.
- 02:15Because, if you don't have an understanding of the problem,
- 02:18you don't have an understand that the biomarkers,
- 02:19or you don't work in a particular area of methodology,
- 02:23I think it becomes very, you know, what do you wanna say?
- 02:26I think people lose interest pretty quickly.
- 02:29And so I decided to start giving talks
- 02:31that had to do overview a subject
- 02:34that I think is very relevant in the field right now.
- 02:38So recently, I'm an external advisor on a grant
- 02:43that Genentech has got from the FDA
- 02:46for developing clinical trials that use real-world data.
- 02:50I've worked with Flatiron in the last few years,
- 02:52as well as the CancerLinQ.
- 02:54And I've been watching this field,
- 02:56sort of the discussions in this field
- 02:58about real-world evidence,
- 02:59and where does it fit-in?
- 03:00and specifically in the context of cancer drug development.
- 03:03So I decided to talk about that today.
- 03:06So yeah, I think this is what I'm gonna do.
- 03:10So does real-world evidence
- 03:11have a role in cancer drug development?
- 03:14So if you'll see,
- 03:15there's a question mark at the end of the statement.
- 03:18So I'm gonna talk about this,
- 03:20and I'm gonna give you the perspective that I have,
- 03:23which comes from a methodologist
- 03:25that really wants real-world evidence
- 03:27to have a role in cancer drug development,
- 03:28because, databases are growing.
- 03:33Data science becomes more relevant
- 03:34if those databases are useful.
- 03:37We all want to write algorithms
- 03:38and do, you know, causal inference on database.
- 03:41We want to unlock those databases with our intelligence,
- 03:44for drug development.
- 03:45Drug development is incredibly expensive.
- 03:48Patients need access to therapies
- 03:52that are gonna save their lives.
- 03:53We have refractory patients enrolling in clinical trials.
- 03:56There's probably not enough clinical trials.
- 03:58And we do have advances in biology
- 04:00that have manifest themselves in precision therapeutics.
- 04:05So we want all of this to work together.
- 04:08We want this to be true.
- 04:09On the other hand,
- 04:11I have designed hundreds of clinical trials
- 04:14and I continue to,
- 04:14most of my collaborations continue
- 04:16with MD Anderson in this space.
- 04:19I've worked with oncologists for over a decade now.
- 04:22I've worked with translational researchers in oncology,
- 04:25and I see the issues that are presented.
- 04:29I mean, maybe I should say the challenges,
- 04:31the challenges that we confront
- 04:33when we think about this space.
- 04:35So I'm gonna talk about this.
- 04:38And I think if I was the 30-year-old version of myself,
- 04:42Brian Hobbs, the 30-year-old,
- 04:44there would not be a question mark here.
- 04:46I would be saying we can use real-world evidence,
- 04:49and this is how.
- 04:50But now that I'm 40 years old, there's a question mark.
- 04:54And I think that, you know okay,
- 04:57so when you've seen other talks about this,
- 04:59I don't know if you're experiencing the same thing I have,
- 05:01but, I've seen several talks at seminars, conferences,
- 05:05where people are presenting very specific cases.
- 05:09Specific cases where they could use real-world evidence
- 05:12and it made sense,
- 05:13or it was the only thing that could be done in that context.
- 05:16So I've seen a lot of talks like that.
- 05:18I'm gonna take this from the other perspective,
- 05:20I'm gonna talk about what's going on in oncology right now.
- 05:22What are the most important developments happening
- 05:24in oncology?
- 05:25And then I'm gonna ask the question,
- 05:27can we use real-world evidence to help augment
- 05:31our trial designs and drug development in general?
- 05:35So to begin with what is real-world evidence?
- 05:37So there's different definitions of this.
- 05:40It tends to be a very broad definition.
- 05:45That's, you know, that different people use this.
- 05:49I've taken this diagram from the CancerLinQ,
- 05:51which is a nonprofit organization that works
- 05:54with the American Society of Clinical Oncology.
- 05:57They are massing and organizing a large database,
- 06:00I collaborate with Elizabeth Garrett-Mayer at CancerLinQ
- 06:03who's at ASCO, who's great.
- 06:05Who's have a PhD statistician working on this.
- 06:08So this diagram, you know,
- 06:10what we often think about as real-world evidence,
- 06:13we think about as the electronic medical health records
- 06:15that are in sort of community hospital systems, right?
- 06:19We think about data that's acquired from routine care
- 06:22or from claims that's not on patients
- 06:27that are in a clinical study.
- 06:29We tend to think about that as real-world evidence.
- 06:31And so CancerLinQ says Real-World Evidence has a capability,
- 06:35data tools, processes, organization, underpinning functions
- 06:38to drive business intelligence.
- 06:39So that's kind of, you know, very broad.
- 06:42They also tell us that there's other things
- 06:45that should count as real-world evidence
- 06:46beyond the EMR data.
- 06:48Okay, observational data
- 06:50as well as historical randomized controlled data.
- 06:52Okay, that makes sense.
- 06:54Pharmacy data, mortality registries, hospital visits,
- 06:57lab values, claim databases, social media,
- 07:02they put on this diagram as well.
- 07:05So you know maybe, right?
- 07:08But I think that we're at a place right now
- 07:11where people are excited about using
- 07:12these sources of information in research,
- 07:15but somebody really needs to develop a framework
- 07:17for each of these.
- 07:18There's not a single framework that says,
- 07:20this is how you use all of these
- 07:22in a clinical research program.
- 07:24If you're gonna use social media in clinical study
- 07:27for research purposes, you know,
- 07:28there needs to be a framework for how to do it,
- 07:30especially in the context of precision oncology.
- 07:33But so we have this,
- 07:34and we have groups that are working on these databases,
- 07:38they want to make this a realization.
- 07:40What are the regulators saying?
- 07:42Well, so real-world data and real-world evidence
- 07:44really got a boost from the 21st Century Cures Act
- 07:47signed into law in 2016.
- 07:49They advocated for the use of real-world evidence
- 07:51to support new indications for approved drugs.
- 07:54Of course, the US Government wants the innovations
- 07:57that we have in biology to translate into therapeutics
- 08:01for patients.
- 08:01And we have a very, you know,
- 08:04I think forward looking approach when it comes to that,
- 08:06if the drug is relatively safe
- 08:08and can demonstrate some efficacy
- 08:10it gets to the market, it gets to patients.
- 08:13So that there's a guidance document
- 08:15about the use of real-world evidence
- 08:16to support regulatory decision-making,
- 08:18which was initially for devices.
- 08:20There's another one for biologics in 2019,
- 08:23there's actually a website you can go to,
- 08:25which is the framework they discussed in 2018.
- 08:29If you go to that website, and this was done,
- 08:33they have quotes from Scott Gottlieb here.
- 08:36You can see that a little more of a definition,
- 08:39real-world data can be used to improve efficiency
- 08:41of clinical trials, even if it's not used
- 08:43for product effectiveness.
- 08:45So the FDA is still saying,
- 08:47we don't want to use real-world data as a control arm
- 08:49to replace a randomized control, for example,
- 08:53but we could use it to generate hypothesis, right?
- 08:56What is the expected event rate
- 08:57for this population that we're enrolling?
- 09:00How many events do we expected
- 09:01to have in a certain timeframe?
- 09:03How likely is it that we can roll that population.
- 09:06Trial feasibility and forming prior distributions
- 09:09in Bayesian models.
- 09:10So I liked that observation,
- 09:11but, you know, what is our expectation?
- 09:15Maybe we're not starting from nothing.
- 09:16And then prognostic indicators.
- 09:19Are there things we should stratify for
- 09:20or account for an analysis that could be imbalanced,
- 09:23especially when we don't randomize?
- 09:25So this is what regulators are saying,
- 09:28but they also say the standard
- 09:30for drug approval remains the same.
- 09:32And this is an important statement.
- 09:34The basis of approval remains the same.
- 09:36Substantial evidence that the drug will have the effect,
- 09:38and adequate well-controlled clinical investigations.
- 09:41So. and I was just at a meeting at UNC
- 09:44with Genentech and FDA and people from the EMA,
- 09:48and they're standing firm on this.
- 09:52While we're discussing
- 09:53how you could potentially augment a randomized-control
- 09:56with real-world controls,
- 09:58there's no sort of interest in replacing
- 10:02of randomized-control right now.
- 10:05Not unless there's absolutely no ethical way
- 10:08you could randomize.
- 10:11So they say that, you know,
- 10:12there's more flexibility when the disease is rare,
- 10:14and the patient population lacks a suitable control.
- 10:19So what about the CancerLinQ?
- 10:20So these slides are a little dated
- 10:22as of the last year, March of 2020,
- 10:24but they had at that time
- 10:26over two and a half million patients in their database.
- 10:30So they have worked on data codes
- 10:32and structuring outcomes, structuring CONMED data.
- 10:37They've done a lot with this database,
- 10:39and I used it at Cleveland Clinic.
- 10:43So this is growing as a resource.
- 10:46Also what happened is that Flatiron,
- 10:51which has over 2 million active patients in their database.
- 10:54Of course, this is an industry group
- 10:57that's partially owned by Roche.
- 11:00They have partnered with Foundation Medicine,
- 11:02and now there's an intersection of Flatiron patients
- 11:05that also have genetic testing from Foundation Medicine.
- 11:09And they're calling this the Clinical Genomic Database.
- 11:12And at the time that I took this slide,
- 11:14they had over 40,000 patients
- 11:15that had the real-world data matched to the molecular data.
- 11:21I think that's very interesting,
- 11:22and I think that's very important.
- 11:25One of the main issues with real-world evidence
- 11:28in the oncology setting
- 11:30is that we don't have a real-world tumor response.
- 11:34So for those of you that work in oncology,
- 11:36of course, you know that phase one, phase two trials
- 11:40are designed on the basis of endpoints
- 11:43that measure reductions in tumor burden.
- 11:46So for solid tumors, this is done through scans.
- 11:49So patients are scanned at baseline.
- 11:50They're scanned regularly at follow-up intervals
- 11:53after every visit or every cycle of therapy.
- 11:57Those scans go for an adjudication process,
- 12:00which is done by more than one person
- 12:02where they actually measure how much reduction
- 12:04in tumor burden happens after treatment.
- 12:06And then we look at that longitudinally,
- 12:08we take the best reduction
- 12:10or the most reduction that we saw,
- 12:13we consider did they have distant migration of disease?
- 12:16So for example, if they had a brain tumor,
- 12:18did they also come in with tumors in their liver?
- 12:22And then we come up, we have a four point ordinal scale,
- 12:25and it tells us whether the patient has a complete response,
- 12:29which means the tumor burden's gone, right?
- 12:31The lesions are gone,
- 12:32or the blast counts in their blood are gone,
- 12:35if they have leukemia.
- 12:36They had a partial response.
- 12:38That means there was a reduction in their tumor size,
- 12:40and it was a clinically meaningful reduction.
- 12:43They had stable disease,
- 12:44which means that there could have been a reduction,
- 12:46but it wasn't clinically meaningful,
- 12:48and it didn't really increase.
- 12:50And progressive disease, the tumor burden is much higher
- 12:53than it was at baseline.
- 12:55So this process is critical for understanding
- 12:59and making decisions in phase two trials,
- 13:02as well as now the phase one trials
- 13:03that we have in oncology, which are very large.
- 13:07This forms the basis for many go-decisions
- 13:09of whether you continue to develop a drug.
- 13:11Did it have a local effect on the tumor burden?
- 13:14It's very expensive to do this.
- 13:15It's very difficult to do this.
- 13:18So now we have to think
- 13:19about how can we get this information from an EMR?
- 13:23Certainly patients may have scans in an EMR
- 13:26that we could use,
- 13:28but there's several issues with that.
- 13:30So if we're going to use scans in a database
- 13:33to assess a patient's tumor burden,
- 13:36number one, those scans don't go for a central review.
- 13:40The process by which the community
- 13:42or the non-trial evaluation of those scans
- 13:45is very different than the clinical trial process.
- 13:49They don't really have an ordinal scale
- 13:51like this that they use.
- 13:54Certainly, I think you could distinguish progressive disease
- 13:58from complete response.
- 13:59I think it'd be very difficult
- 14:00to distinguish partial response from stable disease.
- 14:03So we have groups that are saying they can do this, right?
- 14:05They're going back to the clinical annotations
- 14:07and the writing algorithms that look
- 14:09at the clinical annotations that says,
- 14:11well, if the notes say the lesions are all gone,
- 14:14then they had a complete response, right?
- 14:16If there was an increase, overall increase,
- 14:20or there was new lesions, they had progressive disease.
- 14:23So if the annotations are good enough,
- 14:25I guess, you could get to progressive disease
- 14:27versus complete response.
- 14:30However, there are several issues with this.
- 14:33Everything in oncology is based on the line of therapy.
- 14:36Patients come in, they get a sequence of treatments.
- 14:39Usually, they progress and go to a second line of therapy.
- 14:43Or they progress again
- 14:44and they go to a third line of therapy.
- 14:46The expectations for tumor response as both survival
- 14:49are very different by line of therapy.
- 14:51So if you're gonna go into the EMR,
- 14:52you have to now make sure
- 14:55that the scans you're getting align with the line of therapy
- 14:58that you're enrolling in your clinical study.
- 15:01So most clinical studies in oncology require
- 15:04a specific line of therapy.
- 15:05So first-line therapy means patients
- 15:07that haven't been treated previously.
- 15:09Second-line therapy means patients
- 15:10that have progressed on a prior treatment,
- 15:12and now they're trying a subsequent treatment.
- 15:15So the expectations are very different for response by that.
- 15:18So you would have to know that this is the first,
- 15:20if you're using a first-line therapy study,
- 15:22you would have to know
- 15:23that this is this patient's first line of therapy
- 15:25and these scans correspond to that.
- 15:27Not only that, you'd have to make sure
- 15:29the scans reasonably aligned with the timeframe
- 15:31by which the clinical trial
- 15:32is actually going to acquire their energy.
- 15:37Beyond that, you'd have to, you know,
- 15:40there are several other issues with that, right?
- 15:43Patients may not be scanned in the community setting.
- 15:47And working with oncologists for a long time,
- 15:49I know that there's a certain point
- 15:52where if a patient fails a few lines of therapy,
- 15:55they may not wanna risk the patient getting nephrotoxicity
- 15:58from the contrast that are used in the scans.
- 16:01So if a patient doesn't have a lot of good treatment options
- 16:04or they're reasonably unhealthy,
- 16:06where there's concern about kidney or liver issues,
- 16:09they don't scan the patients in the community.
- 16:12So up till now, I think that the consensus has been,
- 16:16there is no real-world tumor response right now.
- 16:19We don't have that.
- 16:20And I think that's difficult because
- 16:23we want to use real-world data to sort of augment
- 16:26or supplement the areas
- 16:27where we don't have a lot of information, right?
- 16:30And that is the early phase studies, right?
- 16:33Once you go to phase three,
- 16:34you've kind of established that the drug may be promising
- 16:37and you're gonna run a seven-year trial.
- 16:39And over that seven years,
- 16:40you're gonna acquire lots of information,
- 16:43and you're gonna follow them for survival.
- 16:45This could, with really the narrative
- 16:47about real-world evidence in oncology,
- 16:48has really been we can supplement
- 16:50those early phase decisions.
- 16:52But to do that,
- 16:53we really have to have a real-world tumor response.
- 16:55And right now we don't have it.
- 16:57This is a paper from Advanced Therapeutics
- 16:59that was published this year.
- 17:01We have the Flatiron group going back
- 17:04to the major immunotherapy trials
- 17:06that have been implemented in recent years.
- 17:08They're comparing their algorithm
- 17:10for real-world response rates
- 17:12with the trial confirmed response.
- 17:16So they're saying,
- 17:17for each patient, what did we say the response was
- 17:19based on our EMR data?
- 17:21What did the trial said the response was?
- 17:23And they're looking at sort of coordinates
- 17:25between those measures.
- 17:26And they're doing this by line of therapy.
- 17:28So maybe we'll get there,
- 17:30but right now the consensus is we're not there.
- 17:34So we presented this paper at ASCO,
- 17:38which is a big cancer meeting in the US last year,
- 17:41talking about,
- 17:42can we actually replace randomized controls
- 17:44with external real-world controls?
- 17:46And we actually built some tools that Genentech is using
- 17:51that actually calculate,
- 17:52that takes your assumptions about bias, heterogeneity,
- 17:56or other things that you might see in a trial
- 17:58and actually tells you how wrong you can go
- 18:01with a go-decision when you use an external control.
- 18:04And of course, I think maybe everybody knows this,
- 18:07that the reality is that if there's no bias, it's useful.
- 18:11If there is bias, things can go really wrong very quickly,
- 18:14depending on the direction of the bias.
- 18:16And that is really unknown.
- 18:18So we tried to think about this in a very systematic way,
- 18:23and I think it's challenging.
- 18:25I don't know that we can do this.
- 18:28So that leads to, you know, what this discussion was
- 18:31at UNC with the FDA, the EMA, and Genentech
- 18:34where we're talking
- 18:35about now, can we augment randomized control arms
- 18:39with data from real-world sources?
- 18:42So we don't get rid of the randomized control,.
- 18:46We keep the randomized control,
- 18:48but we supplement it with some external controls.
- 18:51How could we do that?
- 18:52And could we even acquire those
- 18:53before the trial gets initiated?
- 18:56Of course, it takes a long time for protocols
- 18:58to be reviewed and other things to happen.
- 19:00Well, this gets interesting to me
- 19:03because while I developed tools to do this a long time ago,
- 19:08which I called Multi-source Adaptive Designs.
- 19:11And this was done many years ago
- 19:15before we talked about real-world evidence.
- 19:17We were talking about historical controls at that time,
- 19:20but of course we can do interesting things
- 19:22with modeling here.
- 19:24We could take real-world controls,
- 19:26we could think about an interim analysis
- 19:29of a randomized trial,
- 19:30where we have randomized treated and randomized controls.
- 19:33We could do any sort of fancy model that you wanna fit,
- 19:37and we could assess how biased are these historical controls
- 19:40or real-world controls in relation to the control data
- 19:43that we're seeing in the actual randomized trial.
- 19:46On the basis of this model,
- 19:47we could actually adapt the allocation, right?
- 19:50If we don't see a lot of bias,
- 19:51so those patients, based on the eligibility of the trial,
- 19:55those patients from the community,
- 19:57they look a lot like the patients in the trial,
- 20:00then you have more information on the control side.
- 20:02You need to rebalance the rest of your allocation
- 20:04so that you can increase power.
- 20:06So this is the only designs
- 20:07where you can actually increase statistical power
- 20:10with a smaller trial.
- 20:12Because what we're trying to do,
- 20:13is we're trying to balance the overall information
- 20:15between the treatments, right?
- 20:17If you look at the outcome, adaptive randomized studies,
- 20:20they required larger trials
- 20:21because they're imbalancing.
- 20:23They're imbalancing based on outcomes.
- 20:25We're trying to balance based on bias.
- 20:27So we worked out this methodology
- 20:29and you know, ASCO and Flatiron
- 20:32are interested in using this.
- 20:34We have a paper that describes
- 20:36an open-source tool that we have.
- 20:37It's still on MD Anderson's website
- 20:39that I built when I was at MD Anderson with Nan Chen.
- 20:43Who is pictured is here.
- 20:44So Nan is now at Gilead.
- 20:47But if you interested in this, it's here.
- 20:49So I think based on in oncology setting,
- 20:54we need to focus on this area.
- 20:56We need to focus on hybrid controls,
- 20:58not replacing control arms, right?
- 21:00At least for most studies.
- 21:02Of course, in rare diseases or areas of pediatric cancer,
- 21:06or both, you need to do something else, right?
- 21:08And that's what the FDA is talking about
- 21:10when they talk about flexibility.
- 21:11But I'm talking about
- 21:12from kind of the standard drug development program
- 21:16in oncology right now.
- 21:18So I've talked about the issues,
- 21:21I've talked about the databases
- 21:24and sort of what's going on
- 21:25with real-world data in oncology.
- 21:27There's another group of sort of players in the space.
- 21:30And I would call them
- 21:31kind of the real-world evidence zealots.
- 21:35This guy, Dr. Butte from Stanford has,
- 21:40I think represents one of these people.
- 21:43So he is a strong advocate for using databases
- 21:47to replace clinical research.
- 21:52He has at least three TED Talks,
- 21:54and I was going through them yesterday.
- 21:59He has a very strong feeling that we just need
- 22:01to organize these databases,
- 22:03and we can answer any medical or scientific question
- 22:06that we want to answer.
- 22:07And in fact, he even says,
- 22:08the problem is there's not enough people asking questions.
- 22:12That's the real issue right now.
- 22:15So there's this other group of people
- 22:17that are you know really hyping up
- 22:21the fact that it's just a computing problem.
- 22:23we have the data,
- 22:23we can use algorithms to answer any question we want.
- 22:27This group of people seems to lack any recognition
- 22:31of the principles of experimental design.
- 22:35They don't seem to acknowledge them anywhere in the process.
- 22:40And Dr. Butte and his TED talks actually says
- 22:44that we don't need randomized controls after all,
- 22:46we just need to build databases.
- 22:49So we had these groups,
- 22:50so these are the kind of the players pushing this forward.
- 22:53So now I'm gonna transition here.
- 22:54I'm gonna talk about what's going on
- 22:56in precision oncology.
- 22:59Okay, so this is how you learned
- 23:01about drug development programs.
- 23:03You learned that we chose dose in phase one,
- 23:06if the dose was promising and we were able to discover
- 23:12what the MTD was, and we felt like it wasn't toxic
- 23:15and we had a good dose,
- 23:16we would go to a phase two trial.
- 23:18In oncology, we would look at tumor response.
- 23:20So again, reduction in tumor burden.
- 23:22Usually these would be uncontrolled.
- 23:24They would be about 50-100 patients.
- 23:26If we saw the drug had local activity on tumor burden,
- 23:30we would go to a phase three trial.
- 23:32The phase three trial would randomize
- 23:33to the existing standard of care.
- 23:35And would see if the treatment prolonged survival.
- 23:39This is what you learned about,
- 23:41but oncology has changed very rapidly.
- 23:44Regulatory policy has changed as well.
- 23:47So molecular biologists have some victories recently.
- 23:51They have really, you know, a lot of the biological models
- 23:53that were discovered a decade ago
- 23:57have been translated into therapeutics.
- 24:00So it used to be that we needed one
- 24:03or two well-controlled phase three trials
- 24:05before we got regulatory approval.
- 24:08It turns out that cancer biologists
- 24:11have identified very specific cancer subsets
- 24:15based on genetics and based on immunology.
- 24:18With those cancer subsets, we have seen very promising,
- 24:21very exciting results in phase two trials without controls.
- 24:26The FDA started to allow conditional approvals
- 24:29after phase two on the basis
- 24:31of those biomarker targeted treatments.
- 24:33Now we're in kind of stage three here.
- 24:36Now we have the awareness that many of the targets,
- 24:38many of the genetic targets,
- 24:39as well as the immune phenotypes that we're interested in,
- 24:44they actually exist across several different
- 24:47sort of traditionally distinct cancer patients.
- 24:50So patients with pancreatic cancer and lung cancer
- 24:54may be very different from a clinical perspective,
- 24:57but they might share a molecular feature
- 24:59that can be targeted by the same drug.
- 25:01And we're now in the space
- 25:03of histology-agnostic drug development,
- 25:06where we might be replacing
- 25:09traditional classification criteria
- 25:11based on molecular features.
- 25:13So we're basically finding new subtypes of cancers as we go.
- 25:19These subtypes are very small,
- 25:21and they're becoming smaller
- 25:22as we learn more about cancer biology.
- 25:25But a few of them had had very exceptional results.
- 25:28A few drugs targeting these event,
- 25:30have had very exceptional results,
- 25:31crossing many tumor types.
- 25:33And they have gotten accelerated approval for agnostic drugs
- 25:37and drugs that can be administered
- 25:38without regard to the tissue of origin.
- 25:41And this has happened in phase one.
- 25:44So the regulatory landscape has changed,
- 25:45the development landscape has changed.
- 25:50So I got to be a part of this review
- 25:53for Nature of Clinical Oncology,
- 25:57where we talked about these tissue-agnostic drugs.
- 26:00There's actually four drugs so far
- 26:02that have been approved by the FDA
- 26:04that could be administered based on a marker feature,
- 26:08not on the actual cancer tissue.
- 26:11So now look at the issues with this.
- 26:14There's four drugs, and there's three different biomarkers
- 26:17that have been approved for tissue-agnostic treatment.
- 26:21One of the biomarkers is the NTRK fusion,
- 26:23which we'll talk about little later.
- 26:25It's exceedingly rare.
- 26:27You can see that breast cancer,
- 26:28we're talking about less than 0.1%
- 26:31of the patients have an NTRK fusion, right?
- 26:34And CRC it's about 1% of patients.
- 26:37There's a few tumors where it's more common,
- 26:40but this becomes very challenging.
- 26:42It becomes very challenging to design a study
- 26:45where we can actually study patients with NTRK fusions.
- 26:48And then who are you gonna get in your study?
- 26:50You're going to get a mixture of many different tissues
- 26:53that were traditionally thought
- 26:54to be separate cancers.
- 26:57So with this transition to tissue-agnostic drug development,
- 27:03there's a statistical question that we have to answer,
- 27:05and that is who can be averaged?
- 27:07Which tissue types could be averaged statistically,
- 27:13when we assess the effectiveness of a biomarker targets
- 27:17and a therapeutic?
- 27:18And that's the question of statistical exchangeability.
- 27:22So we have developed patient models
- 27:24that actually characterize
- 27:28what subsets of tumors actually respond in a similar way
- 27:33to a targeted therapy.
- 27:34And this gives us statistical criteria
- 27:37for understanding what is agnostic and what is not.
- 27:40And I got the, you know, this is the first time,
- 27:43I got to collaborate with Dr. Kane
- 27:44on actually building out tools for this.
- 27:46So I can do the methods, but the tools or something else.
- 27:50So Michael got these incredible tools.
- 27:52And we have an open source package for fitting these models.
- 27:56Just to give you sort of motivation here.
- 27:58This is an actual trial
- 27:59that was evaluating a drug called Bendroflumeth
- 28:04in BRAF tumors, patients that have BRAF mutations.
- 28:08So there is BRAF mutations can occur
- 28:10in many different tumors.
- 28:11They initially developed this drug in Melanoma,
- 28:13but then they saw BRAF tumors,
- 28:15BRAF mutations exist in these other cancers.
- 28:17Histiocytosis, thyroid cancer,
- 28:20cholangiocarcinoma, for example.
- 28:22So they ran up, what's known as a Basket Trial,
- 28:24where they allowed these different tumor types
- 28:26in the same trial.
- 28:28So we show in this nature of these clinical oncology paper,
- 28:31how these exchangeability models work.
- 28:34We call them multi-source exchangeability models.
- 28:36Where we start with an assumption that these tumors
- 28:39are gonna act in the same way, right?
- 28:41So the drug target combination
- 28:43is going to be kind of equally efficacious
- 28:47among all the tumors.
- 28:48So they're exchangeable statistically.
- 28:50We can average them.
- 28:51As we start to get data from the trial,
- 28:55we can now start to assess the heterogeneity
- 28:58that we see across these tumors.
- 28:59And we can ask the question,
- 29:01is it really agnostic to the tumor type?
- 29:04Now, when it comes to vendor afatinib,
- 29:05we had three tumor types that did really well in this trial,
- 29:09histiocytosis, thyroid, and non-small cell lung cancer.
- 29:12Colorectal did not do well.
- 29:14So even though colorectal cancer patients
- 29:16had BRAF mutations,
- 29:17they did not respond to vendor afatinib.
- 29:20These tumors did respond.
- 29:22We don't know about cholangiocarcinoma.
- 29:24There wasn't enough information in that trial
- 29:26to really tell us.
- 29:27So they're kind of in the center here.
- 29:29So, you know, this is just to give you a flavor
- 29:31of what's going on in oncology right now,
- 29:33As we start to go towards precision medicine,
- 29:36that means that we have features across traditionally
- 29:39very different cancers.
- 29:40And we have to understand
- 29:41whether it's actually the feature that's driving,
- 29:44what we see in the response.
- 29:46Okay, so this is an issue
- 29:48that I don't think is that well understood outside
- 29:52of our sort of biostatistical and statistical communities.
- 29:55And that is how, just the extent
- 29:57to which prognostic heterogeneity plays a role
- 30:00in the precision oncology space, or any space
- 30:03where you're doing biomarker driven therapeutics.
- 30:06So what I'm showing you here is the cancer immunity cycle
- 30:09by Chen and Mellman.
- 30:11So this diagram sort of revolutionized
- 30:13how we think about how the immune system identifies
- 30:16and counteracts malignant cells.
- 30:21So cancer cells release antigens.
- 30:24They have to be detected by the immune system.
- 30:29If the immune system detects antigens,
- 30:32means your immune system is actually aware
- 30:33that you have cancer.
- 30:35They have to produce natural killer cells.
- 30:39So the T cells have to be produced in the lymph nodes.
- 30:41They have to infiltrate the tumor.
- 30:43They have to recognize which cells are malignant cells,
- 30:46and then they have to kill the malignant cells.
- 30:48This process is very complicated
- 30:51and there are biomarkers
- 30:52that can tell us about what's happening with the patient.
- 30:56What's happening with the patient's innate immune response
- 30:59to cancer.
- 31:01So the biomarkers that have been most developed recently
- 31:04are the PD-L1 biomarkers, which is this last step.
- 31:08So if a patient is expressing
- 31:10a lot of program death like in one,
- 31:12it means that the malignant cells
- 31:14are actually hiding from the T cells.
- 31:16So the patients might be producing lymphocytes.
- 31:19They might be getting to the tumor, but they can't attach.
- 31:22They can't identify which cells are malignant cells,
- 31:25malignant cells are hiding.
- 31:26So there're very interesting things that happen
- 31:28when you get to a biological perspective.
- 31:32The immune phenotypes based on these biomarkers.
- 31:34If we look at T-cell infiltration versus PD-L1 expression.
- 31:38Patients that are producing T cells
- 31:42and that have low PD-L1 expression.
- 31:44So that means T-cells are being produced,
- 31:46they're coming to the tumor
- 31:47and then they're effective when they get to the tumor.
- 31:50These patients have a different immune profile,
- 31:53than the opposite case
- 31:55where patients are not producing T cells.
- 31:57So it's like their immune system isn't aware
- 31:59that they have cancer.
- 32:00And then even if they did produce T cells,
- 32:03they're not effective once they get to the tumor.
- 32:05So there's various things happening in this phase.
- 32:10And so now I go back to Professor Butte
- 32:13and sort of what he's saying,
- 32:17So there's several articles that he's written
- 32:19that say things like this,
- 32:20precision medicine makes doctors nervous.
- 32:23And he says, the reason that makes doctors nervous
- 32:26is because they have to admit that what they were doing
- 32:29before was not precise.
- 32:32So we see these things
- 32:35and we see these kinds of narratives coming
- 32:38from the group that's really pushing that
- 32:40we just need to analyze these databases.
- 32:44So he's talking about retroactive crowdsourcing, right?
- 32:47A high school kid can do it.
- 32:49So if you've listened to his talks,
- 32:50he's always saying, a high school kid can do that.
- 32:52A high school kid could do this.
- 32:54I think a high school kid could apply a T test to a dataset.
- 32:58I don't disagree with that.
- 33:01But I have a 14 year old at home
- 33:03and he has trouble making his bed.
- 33:05So I think that there's a narrative out there
- 33:10that doesn't recognize things like this.
- 33:13So when I was at MD Anderson,
- 33:16we spent a lot of time thinking
- 33:17about these immune phenotypes.
- 33:19And I actually developed radiomics models,
- 33:21that characterized patterns
- 33:24that we saw in images in the tumor
- 33:26that reflected these immune phenotypes.
- 33:29And the reason we were doing that,
- 33:30is because these biomarkers were incredibly unreliable.
- 33:35What I'm showing you here is a scatter plot,
- 33:36that this came from the Garcia student's lab at MD Anderson,
- 33:39probably the best immune pathologists
- 33:43in the field right now.
- 33:46These are patients with non-small cell lung cancer.
- 33:49They all got treated with definitive surgery.
- 33:51So there was no chemotherapy.
- 33:53They came in, they could be treated with surgery.
- 33:56So we don't have sort of a confounding factor
- 33:59of chemotherapy here with these patients.
- 34:01We got their tissue microarray staining,
- 34:05and this was both malignant cells and immune cells,
- 34:08are PD-L1 positivity at biopsy.
- 34:10So the patients are coming in, they're getting a biopsy.
- 34:12The biopsy is taking a needle,
- 34:14sticking it in a few different locations.
- 34:16We use that tissue and we try to assess
- 34:18how much PD-L1 expression
- 34:19do they have and their lung cancer?
- 34:21Then they go in, they had surgery.
- 34:24We took their whole excise tumor.
- 34:27And we went back and we did whole section staining,
- 34:29of the excise tumor for PD-L1 expression.
- 34:32This is a scatterplot we got.
- 34:34So each point is the same patient.
- 34:37So this patient at biopsy,
- 34:39just this isn't the worst one,
- 34:40but this patient at biopsy was over 50%.
- 34:43After surgery, they're only at 15%.
- 34:47This patient is much worse.
- 34:49So what's going on here?
- 34:52Either the immune system is constantly changing
- 34:54and these biomarkers are not reproducible,
- 34:57in the sense that your state is changing,
- 35:00or when we stick that needle in
- 35:02and we take just a few points,
- 35:05we get a very different answer than when we do surgery.
- 35:08Of course, we have to use biopsy
- 35:09if we're gonna make a treatment selection.
- 35:11So this is problematic.
- 35:14So when I think about, you know, we just need databases,
- 35:17we don't have to understand the science
- 35:18and we can answer all these fundamental questions,
- 35:21I don't think it's true.
- 35:24You know, you have,
- 35:26There's issues like this with every biomarker.
- 35:29The biomarkers have to be reproducible.
- 35:30We have to understand them in a rigorous manner,
- 35:34if you're going to use scanning data.
- 35:37So, you know,
- 35:38so we've published this paper in scientific reports.
- 35:40It has been cited I think almost a hundred times
- 35:42in a few years.
- 35:43Where we actually developed a radiomics model
- 35:45for understanding the immune pathology.
- 35:49Now, why did we do that?
- 35:51We did that because we didn't think
- 35:52these biopsy assessments were reliable.
- 35:55So we thought that maybe the scans were more reliable.
- 35:57Maybe we could take the scans
- 35:58and we can understand the patterns in the scans.
- 36:01And you can see that patients
- 36:03with different immune phenotypes,
- 36:04but in terms of T-cell infiltration and PD-L1,
- 36:07they had very different expectations for survival.
- 36:09So this is not a treatment effect.
- 36:12This is just simply the impact
- 36:15of the fact that the patients have different immune systems.
- 36:17And those immune systems have differential effectiveness
- 36:22in fighting the tumor.
- 36:25So patients that have T-cells and low PD-L1 positivity,
- 36:28they're doing well.
- 36:30The opposite is true for patients
- 36:32that have high PD-L1 and low T cells.
- 36:35So we developed a radiomics model,
- 36:38which take the scans and actually assess these patterns.
- 36:41Of course, there's complications with that.
- 36:46If you're to scan any data in oncology,
- 36:48you're probably having contrast.
- 36:50You need to understand what the protocol
- 36:51for contrast was for that scan.
- 36:54Because you need to take the image
- 36:57when the contrast is in the tumor.
- 36:59So of course you can't just go blindly
- 37:01and grab a bunch of images from a database.
- 37:03So, I've talked a little bit about
- 37:06what's happening on precision oncology.
- 37:07Where we're developing biomarkers,
- 37:09we want to use to guide treatment,
- 37:10but it's very complicated.
- 37:12And I don't think doctors are scared
- 37:13because they're not precise,
- 37:14they're scared because we need to understand
- 37:16that these biomarkers
- 37:17and make sure they're reliable and reproducible.
- 37:20And that knowledge is important.
- 37:24Not only that, but because of all this complexity,
- 37:26drug development on oncology has changed a lot.
- 37:29And we no longer have this, phase one to phase two.
- 37:33This is what early phase drug trials look like now,
- 37:38especially for the big companies
- 37:39that have a lot of money to invest.
- 37:42They're taking multiple dose levels from dose expansion,
- 37:45they're running massive dose expansion cohorts.
- 37:48Those dose expansion cohorts,
- 37:50usually span multiple tumor types.
- 37:54And they might randomize across dose level,
- 37:56because we don't have
- 37:57these very clear monotonic relationships
- 38:00between dose and toxicity anymore.
- 38:02And selecting a dose isn't as simple as it used to be
- 38:05when we did cytotoxic drug development.
- 38:07So these non cytotoxic targeted therapies,
- 38:09it's hard to select a dose.
- 38:11These dose expansion cohorts can be hundreds of patients.
- 38:15They may not even stop for a phase two trial.
- 38:17They may go straight to phase two
- 38:19and expand on the expansion.
- 38:22Or they may skip phase two altogether
- 38:23because they've already acquired so much information
- 38:25in their phase one trial.
- 38:27So this is what we see happening now.
- 38:29Of course, the keynote trial evaluated in Pembrolizumab
- 38:32had eight expansion cohorts.
- 38:34There was over a thousand patients
- 38:35in this first in human phase one trial.
- 38:38This trial is what motivated
- 38:39that NCI Clinical Trial Design Task Force,
- 38:42that I got to be a part of,
- 38:43because this was a massive departure
- 38:46from what we saw typically in oncology
- 38:48and how IRBs would review these studies.
- 38:52More recently, Genentech drug (indistinct)
- 38:56had a phase one trial with nine expansion cohort.
- 38:59Looking at the dose, expansions alone,
- 39:01the bladder cancer cohort had 97 patients,
- 39:03and they randomized the three dose levels.
- 39:06So this is a new world.
- 39:0997 patients already in their dose expansion.
- 39:12So this is where Master Protocols come in.
- 39:14So we have innovations in design
- 39:17that are sort of targeting this
- 39:18and there's many, many methodology recommendations.
- 39:24The other thing that's happened in oncology
- 39:25is that phase three continues to be poor.
- 39:28So phase three trials continue
- 39:30to have a poor track record relative
- 39:31to other areas of medicine.
- 39:34You can see lots of articles that described this.
- 39:38Of course, Gan et al did a review
- 39:40of 235 published randomized controlled trials.
- 39:43Regulatory approval was, you know, less than 38%.
- 39:46And what's happening?
- 39:48While the investigators are not very good
- 39:50about making the assumptions for that phase three trial,
- 39:53we see a lot of phase three trials in oncology
- 39:55that have unrealistic expectations.
- 39:59Okay, so now I talked about precision oncology.
- 40:02I'm gonna go into some case studies
- 40:04that I think are interesting.
- 40:06And I want you ask the question,
- 40:09how could you have used real-world evidence
- 40:11to change what happens here?
- 40:14So this is coming at it
- 40:15from, these are the high profile trials
- 40:16that we have been running in the last few years in oncology.
- 40:20We want to know,
- 40:21how could we have used real-world evidence
- 40:23in these settings?
- 40:26So I'm gonna talk about Atezolizumab
- 40:28and bladder cancer.
- 40:30So Atezolizumab is another PD-1 inhibitor.
- 40:33So immunotherapy, similar to Pembrolizumab.
- 40:38So it was developed for many different areas.
- 40:40Again, we're talking about tissue-agnostic here.
- 40:42So it's targeting a feature of the immune system,
- 40:45that feature of the immune system can exist
- 40:47across many different tumor types.
- 40:49They evaluated nine in their phase one trial.
- 40:52So after the phase one trial,
- 40:53they ran a bunch of trials and different types of cancers
- 40:57and different lines of therapy.
- 40:59One of them was second-line bladder cancer.
- 41:02So these are patients with bladder cancer
- 41:03that have progressed on a prior therapy.
- 41:06So they already progressed on chemotherapy,
- 41:09now they're getting this immunotherapy.
- 41:11So they ran this study and the biomarker they're targeting
- 41:15is they're calling IC2/3.
- 41:18That is immune cell staining of PD-L1.
- 41:21And those immune cells have 5% or more expression.
- 41:25So 5% of the immune cells that they stained had,
- 41:30at least 5% had Programmed Death Ligand 1.
- 41:34That's their target.
- 41:36So, but they enrolled in this phase two trial,
- 41:38they enrolled all comers.
- 41:39It wasn't restricted to the target.
- 41:42They enrolled all comers.
- 41:43So the IC2/3 population is their target.
- 41:45That's where the mechanism is supposed to work.
- 41:48So among a hundred patients
- 41:49with that target they got a 26% response rate.
- 41:53You can see patients that don't have the target,
- 41:55there was 11 and there was eight.
- 41:57And if you look back at their paper,
- 41:59they told us that they expected 10%.
- 42:02So they said the null hypothesis was 10%
- 42:04for this population.
- 42:05We got 26%.
- 42:07This is very exciting, right?
- 42:10This is the survival curves that they present
- 42:12from their phase two trial.
- 42:13Again, this is uncontrolled.
- 42:15There's no chemotherapy arm here.
- 42:17This is just the treated arm, Atezolizumub
- 42:21by biomarker status.
- 42:23And when you look at this,
- 42:24you see this blue Kaplan-Meier curve,
- 42:25that's above everybody else.
- 42:27That Kaplan-Meier curve is the target feature.
- 42:29That's the IC2/3 population.
- 42:31So they're responding,
- 42:32their tumors are shrinking and they're living longer.
- 42:35It looks like this is very promising, right?
- 42:38On the basis of that, they got accelerated approval.
- 42:40And that was given in 2016.
- 42:42And the reason was increased levels of PD-L1 expression
- 42:45on immune cells are associated with increased response.
- 42:49Let's go to the phase three trial.
- 42:51So as a part of the conditional approval
- 42:54with accelerated approval,
- 42:55they have to run a randomized phase three trial
- 42:56and sort of replicate this result.
- 42:59So they designed this trial, IMvigor211,
- 43:02multi-center open-label phase three trial.
- 43:04They compared to three chemotherapies,
- 43:06which were standard chemotherapies used at the time.
- 43:08So there was a physician's choice.
- 43:11If the patient was randomized to chemotherapy,
- 43:12the physician would choose
- 43:14which among these three chemotherapies.
- 43:16So what happened?
- 43:18We had this blockbuster results in phase two,
- 43:21but there was no difference
- 43:22in overall survival in phase three.
- 43:24Not only was there not a difference in overall survival,
- 43:26the objective response rates were similar.
- 43:28So the tumor responses were similar.
- 43:30Moreover they enrolled 931 patients
- 43:33and only 234 actually had the target.
- 43:37So 24% of the trial was used for the primary analysis.
- 43:43When we look at the data, what happened?
- 43:4623% of the IC2/3 population responded.
- 43:49So that's close to 26%.
- 43:51It looks like that was replicated.
- 43:53When you look at the intention to treat populations,
- 43:55that's everybody here, regardless of biomarker,
- 43:57it's 13 and 13.
- 44:00So it was also lower without the target.
- 44:02But what's happening with chemotherapy with the target?
- 44:05It's 22%, right?
- 44:07So chemotherapy is doing great with this biomarker.
- 44:11So this biomarker profile
- 44:13is doing just as well as the targeted therapy,
- 44:17when the patients get the standard of care.
- 44:21Here's the survival curve.
- 44:23Okay, proportional hazards is probably violated.
- 44:26There is a heavy tail here for the Atezo group.
- 44:28Maybe there's, it looks like
- 44:29there's some long-term stable disease,
- 44:32people that are benefiting.
- 44:34But overall, this is not significant.
- 44:36And on the basis of this, actually this year,
- 44:39this drug was withdrawn from accelerated approval.
- 44:44So it got the accelerated approval,
- 44:46which was for very exciting drugs
- 44:48that need an accelerated pathway for regulatory.
- 44:52And then this phase three,
- 44:54on the basis of this phase three trial,
- 44:55they had to withdraw from that.
- 44:57So the question is,
- 45:00how do we use real-world evidence to change this?
- 45:03At the end, there were flaws in this design.
- 45:08They didn't understand the biomarker.
- 45:13They didn't understand the biomarker profile
- 45:14on the basis of the standard of care.
- 45:17So when I first got involved in sort of,
- 45:20well, over this past year, I've been thinking about
- 45:23how could we have used real-world evidence?
- 45:24Here's the case where, you know, there's,
- 45:27it's kind of a failure of the system here
- 45:30that we had this drug withdrawn from accelerated approval.
- 45:32And it's not the only one, by the way.
- 45:35Is there something in the historical data
- 45:38or the real-world data that we could have used
- 45:41that could have informed us to design a better trial,
- 45:45or could have told us something
- 45:47about the fact that this biomarker may be prognostic?
- 45:51Now it gets complicated
- 45:53because actually it's not prognostic for surgery.
- 45:56Patients that have surgery that have IC2/3 status,
- 45:59they're going to die sooner
- 46:02than patients that have IC1, IC0,
- 46:05So this marker seems to be a predictive marker
- 46:08for both chemotherapy and for Atezo.
- 46:11So, but we didn't know.
- 46:12I didn't know if that was true.
- 46:14So my postdoc and I went back and we did a meta-analysis.
- 46:17We went and we extracted all of the trials
- 46:20that had enrolled second-line bladder cancer patients
- 46:23in a prospective study
- 46:25that evaluated the three chemotherapies
- 46:27that were used in the control arm.
- 46:29So those are given here.
- 46:31So I think back to Dr. Butte saying, you know,
- 46:35the real problem in research
- 46:36is you don't have enough people asking questions.
- 46:39When we did this literature search,
- 46:43there were like 200 papers on second-line bladder cancer,
- 46:48Most of them were retrospective reviews and case studies,
- 46:51the overwhelming majority.
- 46:52There were only 11 that were actual perspective studies
- 46:57that we could use in this population.
- 46:59So there's a lot of people writing papers
- 47:01on retrospective databases, there's lots,
- 47:04but what we actually need are prospective studies.
- 47:07So we see here, we have these 11 trials.
- 47:10We're looking at the overall response rate
- 47:12from these 11 trials,
- 47:13we're doing a standard meta-analysis.
- 47:16You can see that Genentech said 10% was their null, right?
- 47:20And really the case for real-world evidence
- 47:22is you can do a better job specifying your null hypothesis.
- 47:25Your null hypothesis can be specified better
- 47:27because you know what to expect for control.
- 47:30So based on our meta-analysis of the objective response,
- 47:3310% is really good estimate.
- 47:35And 10% is like the hierarchal mean of this meta-analysis.
- 47:39So now we go to the chemotherapy arms
- 47:41that we saw in the Atezo trial.
- 47:43We see the IC0/1 population is right at 10%.
- 47:46But look at this,
- 47:48this IC2/3 population.
- 47:50Again, this is with chemotherapy.
- 47:52They're statistically significantly better
- 47:55than the hierarchal mean that we estimated
- 47:58from meta-analysis.
- 48:00So what does this mean?
- 48:02This means that this profile has not been studied before.
- 48:07These trials are mixtures of different immune phenotypes.
- 48:11So we don't know which mean phenotype they're studying.
- 48:14They have a different distribution.
- 48:17Maybe this one has more IC2/3 population
- 48:20because it's pulled over.
- 48:22But the reality is the information
- 48:25in these historical studies
- 48:27doesn't tell us about immune staining.
- 48:30So this is a biomarker that wasn't studied before.
- 48:34And certainly that's goNNA be the case
- 48:36in the community databases.
- 48:37Because there's only a few institutions
- 48:39that really can have the infrastructure
- 48:42to quickly stain these patients
- 48:45as these biomarkers are developing.
- 48:48So we extracted the Kaplan-Meier curves
- 48:52from these historical studies.
- 48:54And we did a meta-analysis of these Kaplan-Meier curves.
- 48:57And that's given here,
- 48:58we have a piece-wise exponential model and a Weibull model.
- 49:02When we put the Kaplan-Meier curves together
- 49:04with the survival.
- 49:06Oh, sorry, when you put the overall response
- 49:07with the survival data,
- 49:08we see this purple line is the chemotherapy arm
- 49:12with this targeted biomarker from the phase three study,
- 49:17that was implemented by Genentech.
- 49:18So responses is better, survival is significantly better
- 49:23than what our expectation was based on historical evidence.
- 49:26And we actually went back and did simulation studies
- 49:29where we fit piece-wise exponential and Weibull curves,
- 49:32till all of these Kaplan-Meier curves
- 49:33that we extracted from the web digitize the tool.
- 49:37When we actually simulated,
- 49:38was it the probability of success
- 49:40for the design implements?
- 49:42And we looked at that for the PDL-1 population,
- 49:45as well as the ITT population.
- 49:46We only give this trial 20% chance of success
- 49:49based on the extent to which chemo is interacting
- 49:52with PD-L1.
- 49:54If you like, if you wanna account for the heavy tail
- 49:56that we see in the model, and do piece of exponential,
- 49:58it goes up to 24%.
- 50:00So another case of a phase three trial
- 50:02there was, had unrealistic expectations, right?
- 50:06And it's a case
- 50:08where we didn't understand the biomarker profile.
- 50:11That biomarker profile had not been characterized
- 50:14in the historical evidence.
- 50:16It's not only Atezolizumab, this happened to Durvalumab.
- 50:20It happened in bladder cancer for Durvalumab again as well.
- 50:22Also a PD-1 inhibitor from AstraZeneca.
- 50:31So Precision Oncology is hard, right?
- 50:35It's hard.
- 50:35It's not, what I presented here,
- 50:39was not really about the lack of having information,
- 50:44it was a lack of having the biomarker target characterized
- 50:47in prior research studies.
- 50:50And without the understanding
- 50:51that profile could be predictive for the standard of care,
- 50:56we have these drugs withdrawing from accelerated approval.
- 51:00There's other issues when we look
- 51:01at tissue-agnostic development.
- 51:03So I worked with Bayer and MD Anderson last year
- 51:06to investigate and NTRK fusions.
- 51:09This is that rare biomarker profile
- 51:12that has led to two drugs getting tissue-agnostic approval,
- 51:17larotrectinib and entrectnib
- 51:20So Bayer bought larotrectinib from LAKSO
- 51:26and Roche bought entrectnib from Igniter.
- 51:29So they wanted to understand,
- 51:31and this was used actually in the Canadian approval process.
- 51:36The Canadian approval process is different.
- 51:38You have a higher level of threshold
- 51:39that you have to characterize
- 51:42for biomarker targeted therapies.
- 51:45And they wanted to know specifically,
- 51:48what is the evidence that NTRK is a prognostic marker?
- 51:51How do we know the drugs working
- 51:52and when it may just be the profile is favorable?
- 51:56And that's kind of exactly what happened with Ateza.
- 51:59So we thought that maybe we could interrogate this
- 52:05by matching.
- 52:07So we had 77 patients from MD Anderson
- 52:09that had NTRK fusions.
- 52:11Where MD Anderson did the staining,
- 52:13we knew they had NTRK fusions and we followed them.
- 52:15And some of these patients were on clinical trials.
- 52:19So we thought, okay, real-world evidence, right?
- 52:21We could match these patients to TCGA data.
- 52:25And we could use TCGA data kind of as a control.
- 52:29And we could compare them.
- 52:30We can kind of get a sense of what the expectation was
- 52:32based on TCGA data.
- 52:34TCGA doesn't have NTRK fusion as one of the mutations.
- 52:38But they have these indications that were enrolled.
- 52:41So among these 77 patients,
- 52:43we have like 14 different tumor types.
- 52:46So we did this study,
- 52:48we went to TCGA,
- 52:50here are the different tumor types
- 52:51that we had in this trial.
- 52:55So we're talking about breast cancer
- 52:57adenocarcinoma, cholangiocarcinoma, GBM.
- 53:02What I'm showing you here is we extracted the TCGA data
- 53:06from these different tumor types.
- 53:09We matched on stage.
- 53:10We matched on sort of performance status.
- 53:12We matched on gender or sex, I should say,
- 53:15we matched on all these factors that are relevant
- 53:19for understanding whether patient's expectation
- 53:21is for survival.
- 53:22And look at the tumor driven heterogeneity.
- 53:27Thyroid cancers is way up here.
- 53:29The patients with thyroid cancer that are matched
- 53:31to these patients at MD Anderson,
- 53:32they're living a really long time.
- 53:35Down here we have patients with GBM, Glioblastoma.
- 53:39This is pancreas.
- 53:41So even though these patients share
- 53:43a common biomarker profile,
- 53:46they have tissue types that are very different,
- 53:48and have very different expectations for survival.
- 53:51Putting this all together to try to understand
- 53:54whether NTRK was prognostic or not,
- 53:57was almost impossible to do.
- 54:01So, you know, conceptually, we have the idea,
- 54:05we have The Cancer Genome Atlas,
- 54:06we should be using it.
- 54:07We can use it to do these things.
- 54:09But when it comes down to actually doing it,
- 54:11it's a real challenge,
- 54:12and it may not provide the information that we need.
- 54:16Okay, so I have two conclusions, very simple ones.
- 54:20Okay, so real-world evidence in precision oncology,
- 54:24how do we use it?
- 54:25Can we use it?
- 54:26The reason we wanna use it,
- 54:27again is because it's very expensive to do,
- 54:32to run trials in oncology.
- 54:34We have these biomarker profiles.
- 54:35Patients have to be stained repeatedly.
- 54:37They have to get imaging,
- 54:39it's it's burdensome for the patient, and it's expensive.
- 54:43So we wanna make better decisions in early phase
- 54:47because we have all these failures in phase three.
- 54:49We want to do a better job
- 54:50of designing our phase three trials as well.
- 54:52So what we really wanna know
- 54:54is can we use real-world evidence to do a better job
- 54:56of setting our null.
- 54:58Where our expectation is.
- 55:00In that case, in that way we can run
- 55:02these uncontrolled trials in early phase
- 55:04and, you know, save all the patients
- 55:07to be treated on the potentially promising therapies.
- 55:10Because we'll have a better idea of what our expectation is
- 55:13and whether this is really promising or not.
- 55:15That is the promise that you hear
- 55:17about real-word evidence in this setting.
- 55:20So it's really about, can we define the null?
- 55:23So I think I showed you two examples here
- 55:25where we really couldn't.
- 55:28We tried to.
- 55:29Like the case is second-line bladder cancer,
- 55:32we went back to the randomized control trial evidence
- 55:35and the null hypothesis was exactly null hypothesis
- 55:38that Genentech used.
- 55:40It's just that, that profile was not steady before.
- 55:43So we couldn't do it there.
- 55:45When we went to the NTRK studies,
- 55:47the TCGA data didn't characterize NTRK,
- 55:50but NTRK is so rare that didn't bother us.
- 55:53So those patients are a mixture
- 55:54of different other mutations.
- 55:57We matched them based
- 55:58on the clinical prognostic characteristics,
- 56:00but the tumors are so different.
- 56:03The expectations are so different across the tumors.
- 56:06It's really hard to understand it from the TCGA data.
- 56:10In fact, this draws in the question,
- 56:14can you really say, if a patient has GBM
- 56:17and they also have thyroid cancer,
- 56:19but they share a mutation,
- 56:21can we really say something that mutation is the target?
- 56:26Can you really treat those cancers as one cancer type?
- 56:30Which is what the tissue-agnostic model says you can.
- 56:33Right, so the biology is that important.
- 56:36In some cases it has been,
- 56:37in the Pembrolizumab it is,
- 56:39like immunotherapy, the immune phenotypes really seem
- 56:42to transcend these cancer tissues,
- 56:46but for other genetic markers,
- 56:47it doesn't seem to be the case.
- 56:49So I guess my conclusion is retrospectively,
- 56:52we really can't.
- 56:53We can't use it right now.
- 56:56It doesn't seem like we can
- 56:58because we are now in the precision oncology setting.
- 57:01And yes, of course,
- 57:02if you're in rare disease setting
- 57:04or you're in a non something else, that's unique.
- 57:08You may have to, and do the best you can.
- 57:11But for the trials I showed you here,
- 57:13I don't see a solution here
- 57:14based on retrospective real-world evidence.
- 57:17I think you could do it prospectively.
- 57:20But I think if you're gonna do it prospectively,
- 57:23there has to be a commitment
- 57:25that right when you start the phase one trial,
- 57:28you need to start staining patients
- 57:30and following them for survival.
- 57:32We don't have a real-world tumor response right now,
- 57:33we can't use that.
- 57:35But you need to have a sort of prospective cohort study
- 57:41that enrolls patients from the community.
- 57:42You need to pay for them to get their assays.
- 57:45You need to understand that the assays may change
- 57:49or develop but to store some information.
- 57:51And then I think later on
- 57:53when you're coming to a decision
- 57:54about phase three or phase two,
- 57:55you go back to that prospective cohort.
- 57:58And you look for patterns based on the relationships
- 58:01between the biomarker and the standard of care.
- 58:04So I think you can do it prospectively.
- 58:05That's not what people wanna do though,
- 58:07they want to use these retrospective databases.
- 58:09So yeah, I guess that's the end of my talk.
- 58:15I didn't leave very much time for questions,
- 58:18but I'm happy to take a few if there's any.
- 59:26So it's somebody asking a question?
- 59:29I can't really hear.
- 59:31I'm sorry, I can't hear it at all, actually.
- 59:36(people chattering)
- 59:46<v Man>Professor Hobbes, can you hear us?</v>
- 59:48<v Brian Hobbes>Yeah, I can kind of hear you,</v>
- 59:50but there's a lot of noise.
- 59:53<v Man>We also have people trying to get in the room</v>
- 59:55so (indistinct)
- 59:57(students chattering)
- 01:00:08Thank you so much, Professor Hobbes.
- 01:00:10(students clapping)
- 01:00:11<v ->All right, thank you very much.</v>
- 01:00:19<v Man>Have a great time, thank you.</v>
- 01:00:21(people chattering)