YSPH Biostatistics Seminar: “Does Real-World Evidence have a Role in Precision Oncology?”

December 07, 2021

Information

Brian P. Hobbs, PhD, Associate Professor, Department of Population Health, University of Texas at Austin

December 7, 2021

ID7248

To CiteDCA Citation Guide

00:00(people chattering)
00:10<v Man>(indistinct) Biostatistics</v>
00:13at the University of Minnesota.
00:15And he's currently attending or an associate professor
00:19at the University of the Texas Dell Medical School.
00:23Dr. Hobbes is a library recognized as an expert
00:27in clinical oncology and research (indistinct).
00:31Among his many accomplishments,
00:35in 2017, Dr. Hobbes (indistinct)
00:40of The National Cancer Institute, clinical trial
00:44(indistinct) a national consensus recommendations
00:50or (indistinct).
00:55In 2019, Dr. Hobbes (indistinct).
01:08In 2021, Dr. Hobbes (indistinct)
01:47<v Brian Hobbes>Thank you, thank you very much,</v>
01:48and for that long and generous introduction.
01:54I'm excited to give this talk today.
01:55I wish I could visit in-person.
01:58I was fortunate to have that opportunity a few years ago,
02:00so thank you for inviting me back.
02:05Okay, so I got tired of giving talks
02:10that were very technical and very specific
02:13to a specific problem.
02:15Because, if you don't have an understanding of the problem,
02:18you don't have an understand that the biomarkers,
02:19or you don't work in a particular area of methodology,
02:23I think it becomes very, you know, what do you wanna say?
02:26I think people lose interest pretty quickly.
02:29And so I decided to start giving talks
02:31that had to do overview a subject
02:34that I think is very relevant in the field right now.
02:38So recently, I'm an external advisor on a grant
02:43that Genentech has got from the FDA
02:46for developing clinical trials that use real-world data.
02:50I've worked with Flatiron in the last few years,
02:52as well as the CancerLinQ.
02:54And I've been watching this field,
02:56sort of the discussions in this field
02:58about real-world evidence,
02:59and where does it fit-in?
03:00and specifically in the context of cancer drug development.
03:03So I decided to talk about that today.
03:06So yeah, I think this is what I'm gonna do.
03:10So does real-world evidence
03:11have a role in cancer drug development?
03:14So if you'll see,
03:15there's a question mark at the end of the statement.
03:18So I'm gonna talk about this,
03:20and I'm gonna give you the perspective that I have,
03:23which comes from a methodologist
03:25that really wants real-world evidence
03:27to have a role in cancer drug development,
03:28because, databases are growing.
03:33Data science becomes more relevant
03:34if those databases are useful.
03:37We all want to write algorithms
03:38and do, you know, causal inference on database.
03:41We want to unlock those databases with our intelligence,
03:44for drug development.
03:45Drug development is incredibly expensive.
03:48Patients need access to therapies
03:52that are gonna save their lives.
03:53We have refractory patients enrolling in clinical trials.
03:56There's probably not enough clinical trials.
03:58And we do have advances in biology
04:00that have manifest themselves in precision therapeutics.
04:05So we want all of this to work together.
04:08We want this to be true.
04:09On the other hand,
04:11I have designed hundreds of clinical trials
04:14and I continue to,
04:14most of my collaborations continue
04:16with MD Anderson in this space.
04:19I've worked with oncologists for over a decade now.
04:22I've worked with translational researchers in oncology,
04:25and I see the issues that are presented.
04:29I mean, maybe I should say the challenges,
04:31the challenges that we confront
04:33when we think about this space.
04:35So I'm gonna talk about this.
04:38And I think if I was the 30-year-old version of myself,
04:42Brian Hobbs, the 30-year-old,
04:44there would not be a question mark here.
04:46I would be saying we can use real-world evidence,
04:49and this is how.
04:50But now that I'm 40 years old, there's a question mark.
04:54And I think that, you know okay,
04:57so when you've seen other talks about this,
04:59I don't know if you're experiencing the same thing I have,
05:01but, I've seen several talks at seminars, conferences,
05:05where people are presenting very specific cases.
05:09Specific cases where they could use real-world evidence
05:12and it made sense,
05:13or it was the only thing that could be done in that context.
05:16So I've seen a lot of talks like that.
05:18I'm gonna take this from the other perspective,
05:20I'm gonna talk about what's going on in oncology right now.
05:22What are the most important developments happening
05:24in oncology?
05:25And then I'm gonna ask the question,
05:27can we use real-world evidence to help augment
05:31our trial designs and drug development in general?
05:35So to begin with what is real-world evidence?
05:37So there's different definitions of this.
05:40It tends to be a very broad definition.
05:45That's, you know, that different people use this.
05:49I've taken this diagram from the CancerLinQ,
05:51which is a nonprofit organization that works
05:54with the American Society of Clinical Oncology.
05:57They are massing and organizing a large database,
06:00I collaborate with Elizabeth Garrett-Mayer at CancerLinQ
06:03who's at ASCO, who's great.
06:05Who's have a PhD statistician working on this.
06:08So this diagram, you know,
06:10what we often think about as real-world evidence,
06:13we think about as the electronic medical health records
06:15that are in sort of community hospital systems, right?
06:19We think about data that's acquired from routine care
06:22or from claims that's not on patients
06:27that are in a clinical study.
06:29We tend to think about that as real-world evidence.
06:31And so CancerLinQ says Real-World Evidence has a capability,
06:35data tools, processes, organization, underpinning functions
06:38to drive business intelligence.
06:39So that's kind of, you know, very broad.
06:42They also tell us that there's other things
06:45that should count as real-world evidence
06:46beyond the EMR data.
06:48Okay, observational data
06:50as well as historical randomized controlled data.
06:52Okay, that makes sense.
06:54Pharmacy data, mortality registries, hospital visits,
06:57lab values, claim databases, social media,
07:02they put on this diagram as well.
07:05So you know maybe, right?
07:08But I think that we're at a place right now
07:11where people are excited about using
07:12these sources of information in research,
07:15but somebody really needs to develop a framework
07:17for each of these.
07:18There's not a single framework that says,
07:20this is how you use all of these
07:22in a clinical research program.
07:24If you're gonna use social media in clinical study
07:27for research purposes, you know,
07:28there needs to be a framework for how to do it,
07:30especially in the context of precision oncology.
07:33But so we have this,
07:34and we have groups that are working on these databases,
07:38they want to make this a realization.
07:40What are the regulators saying?
07:42Well, so real-world data and real-world evidence
07:44really got a boost from the 21st Century Cures Act
07:47signed into law in 2016.
07:49They advocated for the use of real-world evidence
07:51to support new indications for approved drugs.
07:54Of course, the US Government wants the innovations
07:57that we have in biology to translate into therapeutics
08:01for patients.
08:01And we have a very, you know,
08:04I think forward looking approach when it comes to that,
08:06if the drug is relatively safe
08:08and can demonstrate some efficacy
08:10it gets to the market, it gets to patients.
08:13So that there's a guidance document
08:15about the use of real-world evidence
08:16to support regulatory decision-making,
08:18which was initially for devices.
08:20There's another one for biologics in 2019,
08:23there's actually a website you can go to,
08:25which is the framework they discussed in 2018.
08:29If you go to that website, and this was done,
08:33they have quotes from Scott Gottlieb here.
08:36You can see that a little more of a definition,
08:39real-world data can be used to improve efficiency
08:41of clinical trials, even if it's not used
08:43for product effectiveness.
08:45So the FDA is still saying,
08:47we don't want to use real-world data as a control arm
08:49to replace a randomized control, for example,
08:53but we could use it to generate hypothesis, right?
08:56What is the expected event rate
08:57for this population that we're enrolling?
09:00How many events do we expected
09:01to have in a certain timeframe?
09:03How likely is it that we can roll that population.
09:06Trial feasibility and forming prior distributions
09:09in Bayesian models.
09:10So I liked that observation,
09:11but, you know, what is our expectation?
09:15Maybe we're not starting from nothing.
09:16And then prognostic indicators.
09:19Are there things we should stratify for
09:20or account for an analysis that could be imbalanced,
09:23especially when we don't randomize?
09:25So this is what regulators are saying,
09:28but they also say the standard
09:30for drug approval remains the same.
09:32And this is an important statement.
09:34The basis of approval remains the same.
09:36Substantial evidence that the drug will have the effect,
09:38and adequate well-controlled clinical investigations.
09:41So. and I was just at a meeting at UNC
09:44with Genentech and FDA and people from the EMA,
09:48and they're standing firm on this.
09:52While we're discussing
09:53how you could potentially augment a randomized-control
09:56with real-world controls,
09:58there's no sort of interest in replacing
10:02of randomized-control right now.
10:05Not unless there's absolutely no ethical way
10:08you could randomize.
10:11So they say that, you know,
10:12there's more flexibility when the disease is rare,
10:14and the patient population lacks a suitable control.
10:19So what about the CancerLinQ?
10:20So these slides are a little dated
10:22as of the last year, March of 2020,
10:24but they had at that time
10:26over two and a half million patients in their database.
10:30So they have worked on data codes
10:32and structuring outcomes, structuring CONMED data.
10:37They've done a lot with this database,
10:39and I used it at Cleveland Clinic.
10:43So this is growing as a resource.
10:46Also what happened is that Flatiron,
10:51which has over 2 million active patients in their database.
10:54Of course, this is an industry group
10:57that's partially owned by Roche.
11:00They have partnered with Foundation Medicine,
11:02and now there's an intersection of Flatiron patients
11:05that also have genetic testing from Foundation Medicine.
11:09And they're calling this the Clinical Genomic Database.
11:12And at the time that I took this slide,
11:14they had over 40,000 patients
11:15that had the real-world data matched to the molecular data.
11:21I think that's very interesting,
11:22and I think that's very important.
11:25One of the main issues with real-world evidence
11:28in the oncology setting
11:30is that we don't have a real-world tumor response.
11:34So for those of you that work in oncology,
11:36of course, you know that phase one, phase two trials
11:40are designed on the basis of endpoints
11:43that measure reductions in tumor burden.
11:46So for solid tumors, this is done through scans.
11:49So patients are scanned at baseline.
11:50They're scanned regularly at follow-up intervals
11:53after every visit or every cycle of therapy.
11:57Those scans go for an adjudication process,
12:00which is done by more than one person
12:02where they actually measure how much reduction
12:04in tumor burden happens after treatment.
12:06And then we look at that longitudinally,
12:08we take the best reduction
12:10or the most reduction that we saw,
12:13we consider did they have distant migration of disease?
12:16So for example, if they had a brain tumor,
12:18did they also come in with tumors in their liver?
12:22And then we come up, we have a four point ordinal scale,
12:25and it tells us whether the patient has a complete response,
12:29which means the tumor burden's gone, right?
12:31The lesions are gone,
12:32or the blast counts in their blood are gone,
12:35if they have leukemia.
12:36They had a partial response.
12:38That means there was a reduction in their tumor size,
12:40and it was a clinically meaningful reduction.
12:43They had stable disease,
12:44which means that there could have been a reduction,
12:46but it wasn't clinically meaningful,
12:48and it didn't really increase.
12:50And progressive disease, the tumor burden is much higher
12:53than it was at baseline.
12:55So this process is critical for understanding
12:59and making decisions in phase two trials,
13:02as well as now the phase one trials
13:03that we have in oncology, which are very large.
13:07This forms the basis for many go-decisions
13:09of whether you continue to develop a drug.
13:11Did it have a local effect on the tumor burden?
13:14It's very expensive to do this.
13:15It's very difficult to do this.
13:18So now we have to think
13:19about how can we get this information from an EMR?
13:23Certainly patients may have scans in an EMR
13:26that we could use,
13:28but there's several issues with that.
13:30So if we're going to use scans in a database
13:33to assess a patient's tumor burden,
13:36number one, those scans don't go for a central review.
13:40The process by which the community
13:42or the non-trial evaluation of those scans
13:45is very different than the clinical trial process.
13:49They don't really have an ordinal scale
13:51like this that they use.
13:54Certainly, I think you could distinguish progressive disease
13:58from complete response.
13:59I think it'd be very difficult
14:00to distinguish partial response from stable disease.
14:03So we have groups that are saying they can do this, right?
14:05They're going back to the clinical annotations
14:07and the writing algorithms that look
14:09at the clinical annotations that says,
14:11well, if the notes say the lesions are all gone,
14:14then they had a complete response, right?
14:16If there was an increase, overall increase,
14:20or there was new lesions, they had progressive disease.
14:23So if the annotations are good enough,
14:25I guess, you could get to progressive disease
14:27versus complete response.
14:30However, there are several issues with this.
14:33Everything in oncology is based on the line of therapy.
14:36Patients come in, they get a sequence of treatments.
14:39Usually, they progress and go to a second line of therapy.
14:43Or they progress again
14:44and they go to a third line of therapy.
14:46The expectations for tumor response as both survival
14:49are very different by line of therapy.
14:51So if you're gonna go into the EMR,
14:52you have to now make sure
14:55that the scans you're getting align with the line of therapy
14:58that you're enrolling in your clinical study.
15:01So most clinical studies in oncology require
15:04a specific line of therapy.
15:05So first-line therapy means patients
15:07that haven't been treated previously.
15:09Second-line therapy means patients
15:10that have progressed on a prior treatment,
15:12and now they're trying a subsequent treatment.
15:15So the expectations are very different for response by that.
15:18So you would have to know that this is the first,
15:20if you're using a first-line therapy study,
15:22you would have to know
15:23that this is this patient's first line of therapy
15:25and these scans correspond to that.
15:27Not only that, you'd have to make sure
15:29the scans reasonably aligned with the timeframe
15:31by which the clinical trial
15:32is actually going to acquire their energy.
15:37Beyond that, you'd have to, you know,
15:40there are several other issues with that, right?
15:43Patients may not be scanned in the community setting.
15:47And working with oncologists for a long time,
15:49I know that there's a certain point
15:52where if a patient fails a few lines of therapy,
15:55they may not wanna risk the patient getting nephrotoxicity
15:58from the contrast that are used in the scans.
16:01So if a patient doesn't have a lot of good treatment options
16:04or they're reasonably unhealthy,
16:06where there's concern about kidney or liver issues,
16:09they don't scan the patients in the community.
16:12So up till now, I think that the consensus has been,
16:16there is no real-world tumor response right now.
16:19We don't have that.
16:20And I think that's difficult because
16:23we want to use real-world data to sort of augment
16:26or supplement the areas
16:27where we don't have a lot of information, right?
16:30And that is the early phase studies, right?
16:33Once you go to phase three,
16:34you've kind of established that the drug may be promising
16:37and you're gonna run a seven-year trial.
16:39And over that seven years,
16:40you're gonna acquire lots of information,
16:43and you're gonna follow them for survival.
16:45This could, with really the narrative
16:47about real-world evidence in oncology,
16:48has really been we can supplement
16:50those early phase decisions.
16:52But to do that,
16:53we really have to have a real-world tumor response.
16:55And right now we don't have it.
16:57This is a paper from Advanced Therapeutics
16:59that was published this year.
17:01We have the Flatiron group going back
17:04to the major immunotherapy trials
17:06that have been implemented in recent years.
17:08They're comparing their algorithm
17:10for real-world response rates
17:12with the trial confirmed response.
17:16So they're saying,
17:17for each patient, what did we say the response was
17:19based on our EMR data?
17:21What did the trial said the response was?
17:23And they're looking at sort of coordinates
17:25between those measures.
17:26And they're doing this by line of therapy.
17:28So maybe we'll get there,
17:30but right now the consensus is we're not there.
17:34So we presented this paper at ASCO,
17:38which is a big cancer meeting in the US last year,
17:41talking about,
17:42can we actually replace randomized controls
17:44with external real-world controls?
17:46And we actually built some tools that Genentech is using
17:51that actually calculate,
17:52that takes your assumptions about bias, heterogeneity,
17:56or other things that you might see in a trial
17:58and actually tells you how wrong you can go
18:01with a go-decision when you use an external control.
18:04And of course, I think maybe everybody knows this,
18:07that the reality is that if there's no bias, it's useful.
18:11If there is bias, things can go really wrong very quickly,
18:14depending on the direction of the bias.
18:16And that is really unknown.
18:18So we tried to think about this in a very systematic way,
18:23and I think it's challenging.
18:25I don't know that we can do this.
18:28So that leads to, you know, what this discussion was
18:31at UNC with the FDA, the EMA, and Genentech
18:34where we're talking
18:35about now, can we augment randomized control arms
18:39with data from real-world sources?
18:42So we don't get rid of the randomized control,.
18:46We keep the randomized control,
18:48but we supplement it with some external controls.
18:51How could we do that?
18:52And could we even acquire those
18:53before the trial gets initiated?
18:56Of course, it takes a long time for protocols
18:58to be reviewed and other things to happen.
19:00Well, this gets interesting to me
19:03because while I developed tools to do this a long time ago,
19:08which I called Multi-source Adaptive Designs.
19:11And this was done many years ago
19:15before we talked about real-world evidence.
19:17We were talking about historical controls at that time,
19:20but of course we can do interesting things
19:22with modeling here.
19:24We could take real-world controls,
19:26we could think about an interim analysis
19:29of a randomized trial,
19:30where we have randomized treated and randomized controls.
19:33We could do any sort of fancy model that you wanna fit,
19:37and we could assess how biased are these historical controls
19:40or real-world controls in relation to the control data
19:43that we're seeing in the actual randomized trial.
19:46On the basis of this model,
19:47we could actually adapt the allocation, right?
19:50If we don't see a lot of bias,
19:51so those patients, based on the eligibility of the trial,
19:55those patients from the community,
19:57they look a lot like the patients in the trial,
20:00then you have more information on the control side.
20:02You need to rebalance the rest of your allocation
20:04so that you can increase power.
20:06So this is the only designs
20:07where you can actually increase statistical power
20:10with a smaller trial.
20:12Because what we're trying to do,
20:13is we're trying to balance the overall information
20:15between the treatments, right?
20:17If you look at the outcome, adaptive randomized studies,
20:20they required larger trials
20:21because they're imbalancing.
20:23They're imbalancing based on outcomes.
20:25We're trying to balance based on bias.
20:27So we worked out this methodology
20:29and you know, ASCO and Flatiron
20:32are interested in using this.
20:34We have a paper that describes
20:36an open-source tool that we have.
20:37It's still on MD Anderson's website
20:39that I built when I was at MD Anderson with Nan Chen.
20:43Who is pictured is here.
20:44So Nan is now at Gilead.
20:47But if you interested in this, it's here.
20:49So I think based on in oncology setting,
20:54we need to focus on this area.
20:56We need to focus on hybrid controls,
20:58not replacing control arms, right?
21:00At least for most studies.
21:02Of course, in rare diseases or areas of pediatric cancer,
21:06or both, you need to do something else, right?
21:08And that's what the FDA is talking about
21:10when they talk about flexibility.
21:11But I'm talking about
21:12from kind of the standard drug development program
21:16in oncology right now.
21:18So I've talked about the issues,
21:21I've talked about the databases
21:24and sort of what's going on
21:25with real-world data in oncology.
21:27There's another group of sort of players in the space.
21:30And I would call them
21:31kind of the real-world evidence zealots.
21:35This guy, Dr. Butte from Stanford has,
21:40I think represents one of these people.
21:43So he is a strong advocate for using databases
21:47to replace clinical research.
21:52He has at least three TED Talks,
21:54and I was going through them yesterday.
21:59He has a very strong feeling that we just need
22:01to organize these databases,
22:03and we can answer any medical or scientific question
22:06that we want to answer.
22:07And in fact, he even says,
22:08the problem is there's not enough people asking questions.
22:12That's the real issue right now.
22:15So there's this other group of people
22:17that are you know really hyping up
22:21the fact that it's just a computing problem.
22:23we have the data,
22:23we can use algorithms to answer any question we want.
22:27This group of people seems to lack any recognition
22:31of the principles of experimental design.
22:35They don't seem to acknowledge them anywhere in the process.
22:40And Dr. Butte and his TED talks actually says
22:44that we don't need randomized controls after all,
22:46we just need to build databases.
22:49So we had these groups,
22:50so these are the kind of the players pushing this forward.
22:53So now I'm gonna transition here.
22:54I'm gonna talk about what's going on
22:56in precision oncology.
22:59Okay, so this is how you learned
23:01about drug development programs.
23:03You learned that we chose dose in phase one,
23:06if the dose was promising and we were able to discover
23:12what the MTD was, and we felt like it wasn't toxic
23:15and we had a good dose,
23:16we would go to a phase two trial.
23:18In oncology, we would look at tumor response.
23:20So again, reduction in tumor burden.
23:22Usually these would be uncontrolled.
23:24They would be about 50-100 patients.
23:26If we saw the drug had local activity on tumor burden,
23:30we would go to a phase three trial.
23:32The phase three trial would randomize
23:33to the existing standard of care.
23:35And would see if the treatment prolonged survival.
23:39This is what you learned about,
23:41but oncology has changed very rapidly.
23:44Regulatory policy has changed as well.
23:47So molecular biologists have some victories recently.
23:51They have really, you know, a lot of the biological models
23:53that were discovered a decade ago
23:57have been translated into therapeutics.
24:00So it used to be that we needed one
24:03or two well-controlled phase three trials
24:05before we got regulatory approval.
24:08It turns out that cancer biologists
24:11have identified very specific cancer subsets
24:15based on genetics and based on immunology.
24:18With those cancer subsets, we have seen very promising,
24:21very exciting results in phase two trials without controls.
24:26The FDA started to allow conditional approvals
24:29after phase two on the basis
24:31of those biomarker targeted treatments.
24:33Now we're in kind of stage three here.
24:36Now we have the awareness that many of the targets,
24:38many of the genetic targets,
24:39as well as the immune phenotypes that we're interested in,
24:44they actually exist across several different
24:47sort of traditionally distinct cancer patients.
24:50So patients with pancreatic cancer and lung cancer
24:54may be very different from a clinical perspective,
24:57but they might share a molecular feature
24:59that can be targeted by the same drug.
25:01And we're now in the space
25:03of histology-agnostic drug development,
25:06where we might be replacing
25:09traditional classification criteria
25:11based on molecular features.
25:13So we're basically finding new subtypes of cancers as we go.
25:19These subtypes are very small,
25:21and they're becoming smaller
25:22as we learn more about cancer biology.
25:25But a few of them had had very exceptional results.
25:28A few drugs targeting these event,
25:30have had very exceptional results,
25:31crossing many tumor types.
25:33And they have gotten accelerated approval for agnostic drugs
25:37and drugs that can be administered
25:38without regard to the tissue of origin.
25:41And this has happened in phase one.
25:44So the regulatory landscape has changed,
25:45the development landscape has changed.
25:50So I got to be a part of this review
25:53for Nature of Clinical Oncology,
25:57where we talked about these tissue-agnostic drugs.
26:00There's actually four drugs so far
26:02that have been approved by the FDA
26:04that could be administered based on a marker feature,
26:08not on the actual cancer tissue.
26:11So now look at the issues with this.
26:14There's four drugs, and there's three different biomarkers
26:17that have been approved for tissue-agnostic treatment.
26:21One of the biomarkers is the NTRK fusion,
26:23which we'll talk about little later.
26:25It's exceedingly rare.
26:27You can see that breast cancer,
26:28we're talking about less than 0.1%
26:31of the patients have an NTRK fusion, right?
26:34And CRC it's about 1% of patients.
26:37There's a few tumors where it's more common,
26:40but this becomes very challenging.
26:42It becomes very challenging to design a study
26:45where we can actually study patients with NTRK fusions.
26:48And then who are you gonna get in your study?
26:50You're going to get a mixture of many different tissues
26:53that were traditionally thought
26:54to be separate cancers.
26:57So with this transition to tissue-agnostic drug development,
27:03there's a statistical question that we have to answer,
27:05and that is who can be averaged?
27:07Which tissue types could be averaged statistically,
27:13when we assess the effectiveness of a biomarker targets
27:17and a therapeutic?
27:18And that's the question of statistical exchangeability.
27:22So we have developed patient models
27:24that actually characterize
27:28what subsets of tumors actually respond in a similar way
27:33to a targeted therapy.
27:34And this gives us statistical criteria
27:37for understanding what is agnostic and what is not.
27:40And I got the, you know, this is the first time,
27:43I got to collaborate with Dr. Kane
27:44on actually building out tools for this.
27:46So I can do the methods, but the tools or something else.
27:50So Michael got these incredible tools.
27:52And we have an open source package for fitting these models.
27:56Just to give you sort of motivation here.
27:58This is an actual trial
27:59that was evaluating a drug called Bendroflumeth
28:04in BRAF tumors, patients that have BRAF mutations.
28:08So there is BRAF mutations can occur
28:10in many different tumors.
28:11They initially developed this drug in Melanoma,
28:13but then they saw BRAF tumors,
28:15BRAF mutations exist in these other cancers.
28:17Histiocytosis, thyroid cancer,
28:20cholangiocarcinoma, for example.
28:22So they ran up, what's known as a Basket Trial,
28:24where they allowed these different tumor types
28:26in the same trial.
28:28So we show in this nature of these clinical oncology paper,
28:31how these exchangeability models work.
28:34We call them multi-source exchangeability models.
28:36Where we start with an assumption that these tumors
28:39are gonna act in the same way, right?
28:41So the drug target combination
28:43is going to be kind of equally efficacious
28:47among all the tumors.
28:48So they're exchangeable statistically.
28:50We can average them.
28:51As we start to get data from the trial,
28:55we can now start to assess the heterogeneity
28:58that we see across these tumors.
28:59And we can ask the question,
29:01is it really agnostic to the tumor type?
29:04Now, when it comes to vendor afatinib,
29:05we had three tumor types that did really well in this trial,
29:09histiocytosis, thyroid, and non-small cell lung cancer.
29:12Colorectal did not do well.
29:14So even though colorectal cancer patients
29:16had BRAF mutations,
29:17they did not respond to vendor afatinib.
29:20These tumors did respond.
29:22We don't know about cholangiocarcinoma.
29:24There wasn't enough information in that trial
29:26to really tell us.
29:27So they're kind of in the center here.
29:29So, you know, this is just to give you a flavor
29:31of what's going on in oncology right now,
29:33As we start to go towards precision medicine,
29:36that means that we have features across traditionally
29:39very different cancers.
29:40And we have to understand
29:41whether it's actually the feature that's driving,
29:44what we see in the response.
29:46Okay, so this is an issue
29:48that I don't think is that well understood outside
29:52of our sort of biostatistical and statistical communities.
29:55And that is how, just the extent
29:57to which prognostic heterogeneity plays a role
30:00in the precision oncology space, or any space
30:03where you're doing biomarker driven therapeutics.
30:06So what I'm showing you here is the cancer immunity cycle
30:09by Chen and Mellman.
30:11So this diagram sort of revolutionized
30:13how we think about how the immune system identifies
30:16and counteracts malignant cells.
30:21So cancer cells release antigens.
30:24They have to be detected by the immune system.
30:29If the immune system detects antigens,
30:32means your immune system is actually aware
30:33that you have cancer.
30:35They have to produce natural killer cells.
30:39So the T cells have to be produced in the lymph nodes.
30:41They have to infiltrate the tumor.
30:43They have to recognize which cells are malignant cells,
30:46and then they have to kill the malignant cells.
30:48This process is very complicated
30:51and there are biomarkers
30:52that can tell us about what's happening with the patient.
30:56What's happening with the patient's innate immune response
30:59to cancer.
31:01So the biomarkers that have been most developed recently
31:04are the PD-L1 biomarkers, which is this last step.
31:08So if a patient is expressing
31:10a lot of program death like in one,
31:12it means that the malignant cells
31:14are actually hiding from the T cells.
31:16So the patients might be producing lymphocytes.
31:19They might be getting to the tumor, but they can't attach.
31:22They can't identify which cells are malignant cells,
31:25malignant cells are hiding.
31:26So there're very interesting things that happen
31:28when you get to a biological perspective.
31:32The immune phenotypes based on these biomarkers.
31:34If we look at T-cell infiltration versus PD-L1 expression.
31:38Patients that are producing T cells
31:42and that have low PD-L1 expression.
31:44So that means T-cells are being produced,
31:46they're coming to the tumor
31:47and then they're effective when they get to the tumor.
31:50These patients have a different immune profile,
31:53than the opposite case
31:55where patients are not producing T cells.
31:57So it's like their immune system isn't aware
31:59that they have cancer.
32:00And then even if they did produce T cells,
32:03they're not effective once they get to the tumor.
32:05So there's various things happening in this phase.
32:10And so now I go back to Professor Butte
32:13and sort of what he's saying,
32:17So there's several articles that he's written
32:19that say things like this,
32:20precision medicine makes doctors nervous.
32:23And he says, the reason that makes doctors nervous
32:26is because they have to admit that what they were doing
32:29before was not precise.
32:32So we see these things
32:35and we see these kinds of narratives coming
32:38from the group that's really pushing that
32:40we just need to analyze these databases.
32:44So he's talking about retroactive crowdsourcing, right?
32:47A high school kid can do it.
32:49So if you've listened to his talks,
32:50he's always saying, a high school kid can do that.
32:52A high school kid could do this.
32:54I think a high school kid could apply a T test to a dataset.
32:58I don't disagree with that.
33:01But I have a 14 year old at home
33:03and he has trouble making his bed.
33:05So I think that there's a narrative out there
33:10that doesn't recognize things like this.
33:13So when I was at MD Anderson,
33:16we spent a lot of time thinking
33:17about these immune phenotypes.
33:19And I actually developed radiomics models,
33:21that characterized patterns
33:24that we saw in images in the tumor
33:26that reflected these immune phenotypes.
33:29And the reason we were doing that,
33:30is because these biomarkers were incredibly unreliable.
33:35What I'm showing you here is a scatter plot,
33:36that this came from the Garcia student's lab at MD Anderson,
33:39probably the best immune pathologists
33:43in the field right now.
33:46These are patients with non-small cell lung cancer.
33:49They all got treated with definitive surgery.
33:51So there was no chemotherapy.
33:53They came in, they could be treated with surgery.
33:56So we don't have sort of a confounding factor
33:59of chemotherapy here with these patients.
34:01We got their tissue microarray staining,
34:05and this was both malignant cells and immune cells,
34:08are PD-L1 positivity at biopsy.
34:10So the patients are coming in, they're getting a biopsy.
34:12The biopsy is taking a needle,
34:14sticking it in a few different locations.
34:16We use that tissue and we try to assess
34:18how much PD-L1 expression
34:19do they have and their lung cancer?
34:21Then they go in, they had surgery.
34:24We took their whole excise tumor.
34:27And we went back and we did whole section staining,
34:29of the excise tumor for PD-L1 expression.
34:32This is a scatterplot we got.
34:34So each point is the same patient.
34:37So this patient at biopsy,
34:39just this isn't the worst one,
34:40but this patient at biopsy was over 50%.
34:43After surgery, they're only at 15%.
34:47This patient is much worse.
34:49So what's going on here?
34:52Either the immune system is constantly changing
34:54and these biomarkers are not reproducible,
34:57in the sense that your state is changing,
35:00or when we stick that needle in
35:02and we take just a few points,
35:05we get a very different answer than when we do surgery.
35:08Of course, we have to use biopsy
35:09if we're gonna make a treatment selection.
35:11So this is problematic.
35:14So when I think about, you know, we just need databases,
35:17we don't have to understand the science
35:18and we can answer all these fundamental questions,
35:21I don't think it's true.
35:24You know, you have,
35:26There's issues like this with every biomarker.
35:29The biomarkers have to be reproducible.
35:30We have to understand them in a rigorous manner,
35:34if you're going to use scanning data.
35:37So, you know,
35:38so we've published this paper in scientific reports.
35:40It has been cited I think almost a hundred times
35:42in a few years.
35:43Where we actually developed a radiomics model
35:45for understanding the immune pathology.
35:49Now, why did we do that?
35:51We did that because we didn't think
35:52these biopsy assessments were reliable.
35:55So we thought that maybe the scans were more reliable.
35:57Maybe we could take the scans
35:58and we can understand the patterns in the scans.
36:01And you can see that patients
36:03with different immune phenotypes,
36:04but in terms of T-cell infiltration and PD-L1,
36:07they had very different expectations for survival.
36:09So this is not a treatment effect.
36:12This is just simply the impact
36:15of the fact that the patients have different immune systems.
36:17And those immune systems have differential effectiveness
36:22in fighting the tumor.
36:25So patients that have T-cells and low PD-L1 positivity,
36:28they're doing well.
36:30The opposite is true for patients
36:32that have high PD-L1 and low T cells.
36:35So we developed a radiomics model,
36:38which take the scans and actually assess these patterns.
36:41Of course, there's complications with that.
36:46If you're to scan any data in oncology,
36:48you're probably having contrast.
36:50You need to understand what the protocol
36:51for contrast was for that scan.
36:54Because you need to take the image
36:57when the contrast is in the tumor.
36:59So of course you can't just go blindly
37:01and grab a bunch of images from a database.
37:03So, I've talked a little bit about
37:06what's happening on precision oncology.
37:07Where we're developing biomarkers,
37:09we want to use to guide treatment,
37:10but it's very complicated.
37:12And I don't think doctors are scared
37:13because they're not precise,
37:14they're scared because we need to understand
37:16that these biomarkers
37:17and make sure they're reliable and reproducible.
37:20And that knowledge is important.
37:24Not only that, but because of all this complexity,
37:26drug development on oncology has changed a lot.
37:29And we no longer have this, phase one to phase two.
37:33This is what early phase drug trials look like now,
37:38especially for the big companies
37:39that have a lot of money to invest.
37:42They're taking multiple dose levels from dose expansion,
37:45they're running massive dose expansion cohorts.
37:48Those dose expansion cohorts,
37:50usually span multiple tumor types.
37:54And they might randomize across dose level,
37:56because we don't have
37:57these very clear monotonic relationships
38:00between dose and toxicity anymore.
38:02And selecting a dose isn't as simple as it used to be
38:05when we did cytotoxic drug development.
38:07So these non cytotoxic targeted therapies,
38:09it's hard to select a dose.
38:11These dose expansion cohorts can be hundreds of patients.
38:15They may not even stop for a phase two trial.
38:17They may go straight to phase two
38:19and expand on the expansion.
38:22Or they may skip phase two altogether
38:23because they've already acquired so much information
38:25in their phase one trial.
38:27So this is what we see happening now.
38:29Of course, the keynote trial evaluated in Pembrolizumab
38:32had eight expansion cohorts.
38:34There was over a thousand patients
38:35in this first in human phase one trial.
38:38This trial is what motivated
38:39that NCI Clinical Trial Design Task Force,
38:42that I got to be a part of,
38:43because this was a massive departure
38:46from what we saw typically in oncology
38:48and how IRBs would review these studies.
38:52More recently, Genentech drug (indistinct)
38:56had a phase one trial with nine expansion cohort.
38:59Looking at the dose, expansions alone,
39:01the bladder cancer cohort had 97 patients,
39:03and they randomized the three dose levels.
39:06So this is a new world.
39:0997 patients already in their dose expansion.
39:12So this is where Master Protocols come in.
39:14So we have innovations in design
39:17that are sort of targeting this
39:18and there's many, many methodology recommendations.
39:24The other thing that's happened in oncology
39:25is that phase three continues to be poor.
39:28So phase three trials continue
39:30to have a poor track record relative
39:31to other areas of medicine.
39:34You can see lots of articles that described this.
39:38Of course, Gan et al did a review
39:40of 235 published randomized controlled trials.
39:43Regulatory approval was, you know, less than 38%.
39:46And what's happening?
39:48While the investigators are not very good
39:50about making the assumptions for that phase three trial,
39:53we see a lot of phase three trials in oncology
39:55that have unrealistic expectations.
39:59Okay, so now I talked about precision oncology.
40:02I'm gonna go into some case studies
40:04that I think are interesting.
40:06And I want you ask the question,
40:09how could you have used real-world evidence
40:11to change what happens here?
40:14So this is coming at it
40:15from, these are the high profile trials
40:16that we have been running in the last few years in oncology.
40:20We want to know,
40:21how could we have used real-world evidence
40:23in these settings?
40:26So I'm gonna talk about Atezolizumab
40:28and bladder cancer.
40:30So Atezolizumab is another PD-1 inhibitor.
40:33So immunotherapy, similar to Pembrolizumab.
40:38So it was developed for many different areas.
40:40Again, we're talking about tissue-agnostic here.
40:42So it's targeting a feature of the immune system,
40:45that feature of the immune system can exist
40:47across many different tumor types.
40:49They evaluated nine in their phase one trial.
40:52So after the phase one trial,
40:53they ran a bunch of trials and different types of cancers
40:57and different lines of therapy.
40:59One of them was second-line bladder cancer.
41:02So these are patients with bladder cancer
41:03that have progressed on a prior therapy.
41:06So they already progressed on chemotherapy,
41:09now they're getting this immunotherapy.
41:11So they ran this study and the biomarker they're targeting
41:15is they're calling IC2/3.
41:18That is immune cell staining of PD-L1.
41:21And those immune cells have 5% or more expression.
41:25So 5% of the immune cells that they stained had,
41:30at least 5% had Programmed Death Ligand 1.
41:34That's their target.
41:36So, but they enrolled in this phase two trial,
41:38they enrolled all comers.
41:39It wasn't restricted to the target.
41:42They enrolled all comers.
41:43So the IC2/3 population is their target.
41:45That's where the mechanism is supposed to work.
41:48So among a hundred patients
41:49with that target they got a 26% response rate.
41:53You can see patients that don't have the target,
41:55there was 11 and there was eight.
41:57And if you look back at their paper,
41:59they told us that they expected 10%.
42:02So they said the null hypothesis was 10%
42:04for this population.
42:05We got 26%.
42:07This is very exciting, right?
42:10This is the survival curves that they present
42:12from their phase two trial.
42:13Again, this is uncontrolled.
42:15There's no chemotherapy arm here.
42:17This is just the treated arm, Atezolizumub
42:21by biomarker status.
42:23And when you look at this,
42:24you see this blue Kaplan-Meier curve,
42:25that's above everybody else.
42:27That Kaplan-Meier curve is the target feature.
42:29That's the IC2/3 population.
42:31So they're responding,
42:32their tumors are shrinking and they're living longer.
42:35It looks like this is very promising, right?
42:38On the basis of that, they got accelerated approval.
42:40And that was given in 2016.
42:42And the reason was increased levels of PD-L1 expression
42:45on immune cells are associated with increased response.
42:49Let's go to the phase three trial.
42:51So as a part of the conditional approval
42:54with accelerated approval,
42:55they have to run a randomized phase three trial
42:56and sort of replicate this result.
42:59So they designed this trial, IMvigor211,
43:02multi-center open-label phase three trial.
43:04They compared to three chemotherapies,
43:06which were standard chemotherapies used at the time.
43:08So there was a physician's choice.
43:11If the patient was randomized to chemotherapy,
43:12the physician would choose
43:14which among these three chemotherapies.
43:16So what happened?
43:18We had this blockbuster results in phase two,
43:21but there was no difference
43:22in overall survival in phase three.
43:24Not only was there not a difference in overall survival,
43:26the objective response rates were similar.
43:28So the tumor responses were similar.
43:30Moreover they enrolled 931 patients
43:33and only 234 actually had the target.
43:37So 24% of the trial was used for the primary analysis.
43:43When we look at the data, what happened?
43:4623% of the IC2/3 population responded.
43:49So that's close to 26%.
43:51It looks like that was replicated.
43:53When you look at the intention to treat populations,
43:55that's everybody here, regardless of biomarker,
43:57it's 13 and 13.
44:00So it was also lower without the target.
44:02But what's happening with chemotherapy with the target?
44:05It's 22%, right?
44:07So chemotherapy is doing great with this biomarker.
44:11So this biomarker profile
44:13is doing just as well as the targeted therapy,
44:17when the patients get the standard of care.
44:21Here's the survival curve.
44:23Okay, proportional hazards is probably violated.
44:26There is a heavy tail here for the Atezo group.
44:28Maybe there's, it looks like
44:29there's some long-term stable disease,
44:32people that are benefiting.
44:34But overall, this is not significant.
44:36And on the basis of this, actually this year,
44:39this drug was withdrawn from accelerated approval.
44:44So it got the accelerated approval,
44:46which was for very exciting drugs
44:48that need an accelerated pathway for regulatory.
44:52And then this phase three,
44:54on the basis of this phase three trial,
44:55they had to withdraw from that.
44:57So the question is,
45:00how do we use real-world evidence to change this?
45:03At the end, there were flaws in this design.
45:08They didn't understand the biomarker.
45:13They didn't understand the biomarker profile
45:14on the basis of the standard of care.
45:17So when I first got involved in sort of,
45:20well, over this past year, I've been thinking about
45:23how could we have used real-world evidence?
45:24Here's the case where, you know, there's,
45:27it's kind of a failure of the system here
45:30that we had this drug withdrawn from accelerated approval.
45:32And it's not the only one, by the way.
45:35Is there something in the historical data
45:38or the real-world data that we could have used
45:41that could have informed us to design a better trial,
45:45or could have told us something
45:47about the fact that this biomarker may be prognostic?
45:51Now it gets complicated
45:53because actually it's not prognostic for surgery.
45:56Patients that have surgery that have IC2/3 status,
45:59they're going to die sooner
46:02than patients that have IC1, IC0,
46:05So this marker seems to be a predictive marker
46:08for both chemotherapy and for Atezo.
46:11So, but we didn't know.
46:12I didn't know if that was true.
46:14So my postdoc and I went back and we did a meta-analysis.
46:17We went and we extracted all of the trials
46:20that had enrolled second-line bladder cancer patients
46:23in a prospective study
46:25that evaluated the three chemotherapies
46:27that were used in the control arm.
46:29So those are given here.
46:31So I think back to Dr. Butte saying, you know,
46:35the real problem in research
46:36is you don't have enough people asking questions.
46:39When we did this literature search,
46:43there were like 200 papers on second-line bladder cancer,
46:48Most of them were retrospective reviews and case studies,
46:51the overwhelming majority.
46:52There were only 11 that were actual perspective studies
46:57that we could use in this population.
46:59So there's a lot of people writing papers
47:01on retrospective databases, there's lots,
47:04but what we actually need are prospective studies.
47:07So we see here, we have these 11 trials.
47:10We're looking at the overall response rate
47:12from these 11 trials,
47:13we're doing a standard meta-analysis.
47:16You can see that Genentech said 10% was their null, right?
47:20And really the case for real-world evidence
47:22is you can do a better job specifying your null hypothesis.
47:25Your null hypothesis can be specified better
47:27because you know what to expect for control.
47:30So based on our meta-analysis of the objective response,
47:3310% is really good estimate.
47:35And 10% is like the hierarchal mean of this meta-analysis.
47:39So now we go to the chemotherapy arms
47:41that we saw in the Atezo trial.
47:43We see the IC0/1 population is right at 10%.
47:46But look at this,
47:48this IC2/3 population.
47:50Again, this is with chemotherapy.
47:52They're statistically significantly better
47:55than the hierarchal mean that we estimated
47:58from meta-analysis.
48:00So what does this mean?
48:02This means that this profile has not been studied before.
48:07These trials are mixtures of different immune phenotypes.
48:11So we don't know which mean phenotype they're studying.
48:14They have a different distribution.
48:17Maybe this one has more IC2/3 population
48:20because it's pulled over.
48:22But the reality is the information
48:25in these historical studies
48:27doesn't tell us about immune staining.
48:30So this is a biomarker that wasn't studied before.
48:34And certainly that's goNNA be the case
48:36in the community databases.
48:37Because there's only a few institutions
48:39that really can have the infrastructure
48:42to quickly stain these patients
48:45as these biomarkers are developing.
48:48So we extracted the Kaplan-Meier curves
48:52from these historical studies.
48:54And we did a meta-analysis of these Kaplan-Meier curves.
48:57And that's given here,
48:58we have a piece-wise exponential model and a Weibull model.
49:02When we put the Kaplan-Meier curves together
49:04with the survival.
49:06Oh, sorry, when you put the overall response
49:07with the survival data,
49:08we see this purple line is the chemotherapy arm
49:12with this targeted biomarker from the phase three study,
49:17that was implemented by Genentech.
49:18So responses is better, survival is significantly better
49:23than what our expectation was based on historical evidence.
49:26And we actually went back and did simulation studies
49:29where we fit piece-wise exponential and Weibull curves,
49:32till all of these Kaplan-Meier curves
49:33that we extracted from the web digitize the tool.
49:37When we actually simulated,
49:38was it the probability of success
49:40for the design implements?
49:42And we looked at that for the PDL-1 population,
49:45as well as the ITT population.
49:46We only give this trial 20% chance of success
49:49based on the extent to which chemo is interacting
49:52with PD-L1.
49:54If you like, if you wanna account for the heavy tail
49:56that we see in the model, and do piece of exponential,
49:58it goes up to 24%.
50:00So another case of a phase three trial
50:02there was, had unrealistic expectations, right?
50:06And it's a case
50:08where we didn't understand the biomarker profile.
50:11That biomarker profile had not been characterized
50:14in the historical evidence.
50:16It's not only Atezolizumab, this happened to Durvalumab.
50:20It happened in bladder cancer for Durvalumab again as well.
50:22Also a PD-1 inhibitor from AstraZeneca.
50:31So Precision Oncology is hard, right?
50:35It's hard.
50:35It's not, what I presented here,
50:39was not really about the lack of having information,
50:44it was a lack of having the biomarker target characterized
50:47in prior research studies.
50:50And without the understanding
50:51that profile could be predictive for the standard of care,
50:56we have these drugs withdrawing from accelerated approval.
51:00There's other issues when we look
51:01at tissue-agnostic development.
51:03So I worked with Bayer and MD Anderson last year
51:06to investigate and NTRK fusions.
51:09This is that rare biomarker profile
51:12that has led to two drugs getting tissue-agnostic approval,
51:17larotrectinib and entrectnib
51:20So Bayer bought larotrectinib from LAKSO
51:26and Roche bought entrectnib from Igniter.
51:29So they wanted to understand,
51:31and this was used actually in the Canadian approval process.
51:36The Canadian approval process is different.
51:38You have a higher level of threshold
51:39that you have to characterize
51:42for biomarker targeted therapies.
51:45And they wanted to know specifically,
51:48what is the evidence that NTRK is a prognostic marker?
51:51How do we know the drugs working
51:52and when it may just be the profile is favorable?
51:56And that's kind of exactly what happened with Ateza.
51:59So we thought that maybe we could interrogate this
52:05by matching.
52:07So we had 77 patients from MD Anderson
52:09that had NTRK fusions.
52:11Where MD Anderson did the staining,
52:13we knew they had NTRK fusions and we followed them.
52:15And some of these patients were on clinical trials.
52:19So we thought, okay, real-world evidence, right?
52:21We could match these patients to TCGA data.
52:25And we could use TCGA data kind of as a control.
52:29And we could compare them.
52:30We can kind of get a sense of what the expectation was
52:32based on TCGA data.
52:34TCGA doesn't have NTRK fusion as one of the mutations.
52:38But they have these indications that were enrolled.
52:41So among these 77 patients,
52:43we have like 14 different tumor types.
52:46So we did this study,
52:48we went to TCGA,
52:50here are the different tumor types
52:51that we had in this trial.
52:55So we're talking about breast cancer
52:57adenocarcinoma, cholangiocarcinoma, GBM.
53:02What I'm showing you here is we extracted the TCGA data
53:06from these different tumor types.
53:09We matched on stage.
53:10We matched on sort of performance status.
53:12We matched on gender or sex, I should say,
53:15we matched on all these factors that are relevant
53:19for understanding whether patient's expectation
53:21is for survival.
53:22And look at the tumor driven heterogeneity.
53:27Thyroid cancers is way up here.
53:29The patients with thyroid cancer that are matched
53:31to these patients at MD Anderson,
53:32they're living a really long time.
53:35Down here we have patients with GBM, Glioblastoma.
53:39This is pancreas.
53:41So even though these patients share
53:43a common biomarker profile,
53:46they have tissue types that are very different,
53:48and have very different expectations for survival.
53:51Putting this all together to try to understand
53:54whether NTRK was prognostic or not,
53:57was almost impossible to do.
54:01So, you know, conceptually, we have the idea,
54:05we have The Cancer Genome Atlas,
54:06we should be using it.
54:07We can use it to do these things.
54:09But when it comes down to actually doing it,
54:11it's a real challenge,
54:12and it may not provide the information that we need.
54:16Okay, so I have two conclusions, very simple ones.
54:20Okay, so real-world evidence in precision oncology,
54:24how do we use it?
54:25Can we use it?
54:26The reason we wanna use it,
54:27again is because it's very expensive to do,
54:32to run trials in oncology.
54:34We have these biomarker profiles.
54:35Patients have to be stained repeatedly.
54:37They have to get imaging,
54:39it's it's burdensome for the patient, and it's expensive.
54:43So we wanna make better decisions in early phase
54:47because we have all these failures in phase three.
54:49We want to do a better job
54:50of designing our phase three trials as well.
54:52So what we really wanna know
54:54is can we use real-world evidence to do a better job
54:56of setting our null.
54:58Where our expectation is.
55:00In that case, in that way we can run
55:02these uncontrolled trials in early phase
55:04and, you know, save all the patients
55:07to be treated on the potentially promising therapies.
55:10Because we'll have a better idea of what our expectation is
55:13and whether this is really promising or not.
55:15That is the promise that you hear
55:17about real-word evidence in this setting.
55:20So it's really about, can we define the null?
55:23So I think I showed you two examples here
55:25where we really couldn't.
55:28We tried to.
55:29Like the case is second-line bladder cancer,
55:32we went back to the randomized control trial evidence
55:35and the null hypothesis was exactly null hypothesis
55:38that Genentech used.
55:40It's just that, that profile was not steady before.
55:43So we couldn't do it there.
55:45When we went to the NTRK studies,
55:47the TCGA data didn't characterize NTRK,
55:50but NTRK is so rare that didn't bother us.
55:53So those patients are a mixture
55:54of different other mutations.
55:57We matched them based
55:58on the clinical prognostic characteristics,
56:00but the tumors are so different.
56:03The expectations are so different across the tumors.
56:06It's really hard to understand it from the TCGA data.
56:10In fact, this draws in the question,
56:14can you really say, if a patient has GBM
56:17and they also have thyroid cancer,
56:19but they share a mutation,
56:21can we really say something that mutation is the target?
56:26Can you really treat those cancers as one cancer type?
56:30Which is what the tissue-agnostic model says you can.
56:33Right, so the biology is that important.
56:36In some cases it has been,
56:37in the Pembrolizumab it is,
56:39like immunotherapy, the immune phenotypes really seem
56:42to transcend these cancer tissues,
56:46but for other genetic markers,
56:47it doesn't seem to be the case.
56:49So I guess my conclusion is retrospectively,
56:52we really can't.
56:53We can't use it right now.
56:56It doesn't seem like we can
56:58because we are now in the precision oncology setting.
57:01And yes, of course,
57:02if you're in rare disease setting
57:04or you're in a non something else, that's unique.
57:08You may have to, and do the best you can.
57:11But for the trials I showed you here,
57:13I don't see a solution here
57:14based on retrospective real-world evidence.
57:17I think you could do it prospectively.
57:20But I think if you're gonna do it prospectively,
57:23there has to be a commitment
57:25that right when you start the phase one trial,
57:28you need to start staining patients
57:30and following them for survival.
57:32We don't have a real-world tumor response right now,
57:33we can't use that.
57:35But you need to have a sort of prospective cohort study
57:41that enrolls patients from the community.
57:42You need to pay for them to get their assays.
57:45You need to understand that the assays may change
57:49or develop but to store some information.
57:51And then I think later on
57:53when you're coming to a decision
57:54about phase three or phase two,
57:55you go back to that prospective cohort.
57:58And you look for patterns based on the relationships
58:01between the biomarker and the standard of care.
58:04So I think you can do it prospectively.
58:05That's not what people wanna do though,
58:07they want to use these retrospective databases.
58:09So yeah, I guess that's the end of my talk.
58:15I didn't leave very much time for questions,
58:18but I'm happy to take a few if there's any.
58:24(indistinct)
59:26So it's somebody asking a question?
59:29I can't really hear.
59:31I'm sorry, I can't hear it at all, actually.
59:36(people chattering)
59:46<v Man>Professor Hobbes, can you hear us?</v>
59:48<v Brian Hobbes>Yeah, I can kind of hear you,</v>
59:50but there's a lot of noise.
59:53<v Man>We also have people trying to get in the room</v>
59:55so (indistinct)
59:57(students chattering)
01:00:08Thank you so much, Professor Hobbes.
01:00:10(students clapping)
01:00:11<v ->All right, thank you very much.</v>
01:00:19<v Man>Have a great time, thank you.</v>
01:00:21(people chattering)