# YSPH Biostatistics Seminar: “The Predictive Individual Effect for Survival Data: A Patient-Oriented Summary Measure for Treatment Benefit”

November 14, 2022## Information

Speaker: Satrajit Roychoudhury, PhD, Pfizer

November 8, 2022

ID9090

To CiteDCA Citation Guide

- 00:00(presenter faintly speaking continues)
- 00:05<v Presenter>Dr. Roychoudhury,</v>
- 00:07(presenter faintly speaking continues)
- 00:26research institute (indistinct).
- 00:30He had 15 years of extensive experience.
- 00:34(presenter faintly speaking continues)
- 00:43Model based projects (indistinct).
- 00:47He served as the (indistinct) co-chair
- 00:51(faintly speaking) workshop.
- 00:55(presenter faintly speaking)
- 00:56And he's serving as co-chair for DIA, FDA,
- 00:59biostatistics (indistinct).
- 01:04Dr. Roychoudhury was exacted to be a panel
- 01:07of the American (indistinct) Association.
- 01:10(presenter faintly speaking)
- 01:11the country (indistinct),
- 01:15international (indistinct) society in 2019.
- 01:20So let's welcome Dr. Roychoudhury.
- 01:23Now I'm yours.
- 01:25<v ->[Dr. Roychoudhury] Thank you.</v>
- 01:28Thanks a lot Dr. (indistinct) for the nice introduction.
- 01:30Can you all hear me well?
- 01:34And thank you for the opportunity to present here
- 01:38and having a chance to interact with all of you.
- 01:41So today, (faintly speaking) I'm gonna talk
- 01:45about a problem that is many
- 01:48of the recent drug development are facing,
- 01:52and we try to talk about better interpretation
- 01:55of clinical trial data,
- 01:57especially bringing the different perspective
- 01:59as some of you know and some of you don't.
- 02:02FDA actually specifically started
- 02:05a patient oriented drug development program,
- 02:08which kind of how to make the data more understandable
- 02:12to the patient, more reachable to the patient.
- 02:14So this was kind of on that theme mostly.
- 02:18Before I begin, just I wanted to mention
- 02:21the standard disclaimer, this my own view,
- 02:25and not necessarily reflect the view of the Pfizer.
- 02:30So I think many of you have work
- 02:33on survival analysis as a coursework
- 02:36or maybe analyzing trial data, research perspective.
- 02:41So often, in critical trial setting,
- 02:44we call that analysis over time to event data.
- 02:47Because we look into the data that up to time,
- 02:51so we analyze the time up to a certain event
- 02:54or sensor the data looked at.
- 02:57And the standard way of analyzing such data,
- 02:59these are the, more or less, a standard tool.
- 03:01I'm sure you all have done or are going to do
- 03:04in your courseworks, that looking into Kaplan marker
- 03:07which looks into the survival function,
- 03:10the effect over time.
- 03:12Log-rank test, which basically tests the difference
- 03:16between the two curve, that if the treatment is better
- 03:19than control or control is worse.
- 03:22And then last, to summarizing treatment of it, right?
- 03:26At the end of the day, we need to know
- 03:28how good is the treatment.
- 03:29And one way to say that was basically hazard ratio
- 03:32or something we call Cox regression.
- 03:37Now low-rank p val-
- 03:38These are very standard reporting techniques.
- 03:41Any time to event data, if you pick up any medical journals,
- 03:45are typically analyzed using three (indistinct).
- 03:48But the question was, and based on some examples done,
- 03:52we'll dive into the details as we go along.
- 03:55the question really came up,
- 03:56are these really good metrics to analyze the data?
- 04:03So just to give a quick introduction,
- 04:05I'm sure all of you know it very well.
- 04:06But just to understand the fundamental assumptions,
- 04:10what the Cox regression means.
- 04:13So Cox regression and hazard ratio are the very,
- 04:17very popular method.
- 04:18And it started, basically introduced
- 04:20by Dr. DR Cox in 1972, one of the brilliant statistician.
- 04:26And it's closely related to mathematically
- 04:29with the log-rank test,
- 04:30which actually kind of increases its popularity.
- 04:33There mathematically, the score function,
- 04:35the score test for Cox regression is equal value
- 04:40to the log-rank test under two sample case.
- 04:44But it has an assumption also.
- 04:46It assumes that the treatment effect basically is constant
- 04:50over time, so it emerges at the beginning of the trial
- 04:53and it remain kind of a constant over time.
- 04:58Which is a problem when this is not true,
- 05:01because not all drugs work the same way.
- 05:03Sometimes, maybe some patient get benefited
- 05:05after getting treated for longer time.
- 05:10On such case, such a measure has a ratio,
- 05:13start to lack its interpretation.
- 05:16And also there is some problem
- 05:18regarding causal inference perspective.
- 05:22Let's look into two examples, two, three examples,
- 05:25real life examples, and try to understand
- 05:27where the problem was.
- 05:28So first, let's start with more simpler
- 05:31when treatment effect emerges at the beginning
- 05:34and remained at that (indistinct) over the trial.
- 05:38So the trial on...
- 05:39And they're all real trials,
- 05:41and I added the references in case you want
- 05:44to look into later.
- 05:45The one on the left is basically a trial
- 05:48for non-small cell lung cancer where gemcitabine
- 05:51and gemcitabine plus erlotinib combination has been
- 05:55looked into, which is showed the blue curve.
- 05:58There is experimental combination drug,
- 06:01which shows superiority
- 06:03over the standard of care, gemcitabine.
- 06:06The one on the right is basically
- 06:09on refractory multiple myeloma disease,
- 06:12which is a very fatal disease.
- 06:15And they're basically triple combo,
- 06:17so kRd is a triple combination drug,
- 06:19where Rd is basically a double combination drug.
- 06:22I'm not going into the details of that.
- 06:24Looking into basically kRd,
- 06:27was the experimental drug of this showing the superiority?
- 06:30In both cases, the proportionality has assumption,
- 06:33looks valid, because the effects started and it continued.
- 06:37But that's not all always the case.
- 06:39Here are the very few, very recent examples,
- 06:42especially if you are looking into the newspapers as well.
- 06:46And I think two years or three years back,
- 06:49the person from MD Anderson actually got the Nobel Prize
- 06:52for looking into the immunotherapy,
- 06:54one of the fundamental research from that area.
- 06:57So when it does this particular type of therapy,
- 07:01basically putting people immune system,
- 07:05basically and they kind of train them
- 07:08to fight against their cancer.
- 07:11And CheckMate 141, which is squamous cell carcinoma
- 07:15on head and neck,
- 07:16and nivolumab is one of the immunotherapies
- 07:19of the first (indistinct) cluster for their client.
- 07:22When they looked into the actual treatment effect,
- 07:26started to emerge pretty late.
- 07:27So that means it take the time for the immune system
- 07:31to actually work.
- 07:33And then the question is,
- 07:35the treatment effect is no more constant, right?
- 07:38Immunity is at three month.
- 07:40It's even more problematic coming into the example
- 07:43in IM211 trial, which is urothelial carcinoma
- 07:47and well atezolizumab, a compound,
- 07:50and Roche was actually looking into against chemotherapy.
- 07:54The compound was detrimental at the beginning
- 07:58and slightly at least compared to the standard of care,
- 08:01the chemotherapy that how the people are treated.
- 08:04And then it showed benefit, definitely.
- 08:08But there is something more interesting in the second ex-
- 08:10I will concentrate and each of them are interesting,
- 08:13each of them have a story of its own.
- 08:15But I just focus on this IM211 trial
- 08:19to bring in the patient perspective a little bit.
- 08:22So if you look into this trial, of course the trial
- 08:25doesn't have a significant amount, right?
- 08:28The log-rank test, which tested the superiority
- 08:31of the curve basically said it's nonsignificant.
- 08:35But if you actually wanna look
- 08:38into this survival curve a little bit more details,
- 08:42one can see the survival effect is really emerging
- 08:46and once it gets into the 18 month,
- 08:49it started to be pretty significant.
- 08:52You come in carcinoma.
- 08:53So this is kind of an 8% survival benefit is for this class
- 08:57of patient is quite a meaningful benefit on that context.
- 09:01So the question is, are we evaluating this drug correctly
- 09:06by these two standard metrics like Cox, hazard ratio,
- 09:09which doesn't look good here, 0.9, close to one.
- 09:12So it has a ratio less than one
- 09:14because we are comparing treatment versus control.
- 09:17Less than one means treatment is better
- 09:19and bigger than one means treatment is worse.
- 09:22So basically doesn't really reflect these angles.
- 09:25And also, if you look into the hazard function plots
- 09:29or hazard ratio plots,
- 09:30you can quickly see they're not constant over time.
- 09:33They're actually emerging or varying there.
- 09:41I mean, there was been a working group,
- 09:44there has been a number of workshop here,
- 09:46and even FBA and other regulators got interested.
- 09:50There has been a lot of discussions started in 2016
- 09:53when nivolumab, the first class
- 09:56of immunotherapy compound came in in a development phase.
- 10:01And there has been a lot of research.
- 10:02People were looking into different tests, okay,
- 10:05log-rank is not good, because it is most powerful test
- 10:09under proportionality hazard assumption.
- 10:11We don't have it, especially for situation like this,
- 10:14CheckMate or IM211 trial;
- 10:17or we should do some other test.
- 10:19This is not powerful, some better test.
- 10:21So there has been a zoo of tests being came in.
- 10:24You may have heard about some of them weighted,
- 10:26so instead of log-rank test, weighted log-rank test
- 10:29where we selectively weight that separate areas
- 10:33of the Kaplan-Meier curve.
- 10:35Then some a little bit more robust, like MaxCombo,
- 10:38weighted Kaplan-Meier test, restricted mean survival time,
- 10:42and there are many, many.
- 10:43I mean, and they try to look into
- 10:45how to handle different potential.
- 10:47Because one thing we need to understand,
- 10:49proportional hazard is a very specific character.
- 10:54When we say proportional hazard
- 10:55that these two hazard plots are (indistinct).
- 10:59But once we set non-proportional hazard,
- 11:01there could be many, many possibilities.
- 11:04Drugs can be crossed, drugs can separate the link.
- 11:08Drugs can first separate and then emerge back.
- 11:11So it's a no more unique (indistinct).
- 11:13So we need a set of methodology that's sort of robust
- 11:16across this different class of alternatives.
- 11:21But the problem is even having a P value,
- 11:25suppose once we call about, okay, there's a good test
- 11:28and we got a good P value for such a curve.
- 11:31But one can say, oh how can a P value is meaningful?
- 11:35The beginning of the curve, the patients are getting worse.
- 11:39Is a P value really meaningful in this context?
- 11:43So rejecting the null is really a less informative.
- 11:47You need to know more information
- 11:50to understand these kind parts of compounds better.
- 11:54And that's also been looked into.
- 11:56There are multiple, multiple things has been looked into
- 12:00with simulation, with datas, lot of meta analysis
- 12:06with different data looking into the percentile, right?
- 12:08I mean one part of we're looking,
- 12:09why don't we look into the ratio of the median
- 12:13of the compound?
- 12:14Or we look into the (indistinct) time or percentile, right?
- 12:17Things are separating at percentile.
- 12:19Maybe we can look into percentile
- 12:21over the time ratios of those.
- 12:25Then milestone survival and coming into a moment,
- 12:28which is a sort of a very meaningful
- 12:31to a patient's restricted means survival.
- 12:34Basically you average over the area under the curve,
- 12:39which it has a ratio, because we are doing with a test,
- 12:42a dual estimator of that is weighted hazard ratio,
- 12:45piecewise hazard ratio.
- 12:46Cox model, now, the initial Cox model doesn't have
- 12:51a time component in it (indistinct).
- 12:53Only the time component is in the baseline hazard.
- 12:55We can introduce a time component into that
- 12:58to make sure the treatment effect is sort of time dependent.
- 13:03And then there are other things like net benefit.
- 13:08I won't dive every one of details,
- 13:10but I just wanted to mention that hazard,
- 13:12so weighted hazard ratio means hazard ratio,
- 13:14we were doing a Cox regression.
- 13:16We were looking into basically the regression coefficient
- 13:20corresponding to the treatment using
- 13:21a partial likelihood method.
- 13:23And now, the whole idea was similar to like testing
- 13:26where we are weighting different part
- 13:28of Kaplan-Meier differently, we weight different part
- 13:31of the partial likelihood differently.
- 13:33Or there are other type of weightings as well,
- 13:36like average weights and others.
- 13:38So basically, the whole idea was not
- 13:39to treat each event similar,
- 13:41but differently based on their interest.
- 13:44But that's the (indistinct) of course,
- 13:48treatment emerges at the end.
- 13:50We may be interested more towards the end,
- 13:52but it adds a subjective choice, right?
- 13:55And it's not very easy for non-clinicians
- 14:00to understand on that aspect.
- 14:06So now then, of course in 2005, people also talk quite a bit
- 14:12about piecewise hazard ratios
- 14:15and now still piecewise hazard is still very important.
- 14:18Like you divide the whole time axis into different intervals
- 14:23and you basically calculate local hazard ratios.
- 14:28Which is very meaning 'cause you can look
- 14:30into the first part when (indistinct) not separated.
- 14:32The hazard ratio's close to one,
- 14:34then the hazard ratio emerges.
- 14:36And there's a natural extension
- 14:37of that was basically using a regression using a time factor
- 14:41into the core value.
- 14:41But really, the power and the performance really depends
- 14:47on the function that you choose, which is, again,
- 14:51is difficult to interpret in a practical sense
- 14:54that if results really value on such a choice.
- 14:57But one thing was kinda after all,
- 15:00among all this discussion,
- 15:02one thing was when we talked to non-statisticians,
- 15:04specifically clinician, one thing was very clear,
- 15:07one measure we found.
- 15:09The measure is improvement in survival.
- 15:12That means they're very clear about certain metric
- 15:15that, okay, what is the survival gain at five year
- 15:18or eight year?
- 15:19Those metrics seems to be very intuitive,
- 15:21very clear to to non-statistician patient
- 15:25and all other stakeholders.
- 15:28But the only problem is at the beginning of the file,
- 15:31you just don't know when the curve is gonna be separate.
- 15:35You don't know that.
- 15:36So (indistinct) finds such point,
- 15:38can be very dangerous for you.
- 15:44The last measure I mentioned was this,
- 15:46been over and over discussed,
- 15:47which is called residual, meaning lifetime,
- 15:50basically (indistinct) lifetime,
- 15:53residual means survival time.
- 15:56Which has been over and over discussed recently
- 15:58in clinical literature, as well as in statistical,
- 16:00restricted mean people.
- 16:02When people looking into restricted mean,
- 16:03that means they're looking for supposed up to a time Tau.
- 16:08You cut the curve, you compare the area under the curve,
- 16:12which is in this block.
- 16:13And you see if you can look into the difference,
- 16:16you can look into the ratio,
- 16:17and try to see if the area under the curve is better
- 16:21on that context.
- 16:22But remember, the problem is here,
- 16:24the comparison also very much depends on the choice
- 16:26of the cutoff, where you choose it, basically.
- 16:31The Tau and risk of the censoring pattern
- 16:33of the Kaplan-Meier plays a very big role
- 16:35in such a such compass,
- 16:39especially in a setting for such metrics.
- 16:43Especially, it's very problematic
- 16:46when in a cancer setting like metastatic,
- 16:49the one I showed you earlier,
- 16:50because there is a, the disease is very fit on it,
- 16:54happens very quickly.
- 16:55Patient gets multiple therapies.
- 16:57So censoring patterns are much more aggressive,
- 16:59whereas there are other disease setting
- 17:01where RMST seems to be very meaningful.
- 17:04Because you get lot of follow up much more uniformly.
- 17:10Now the problem was, okay, all these measures differently.
- 17:13We apply, we get different results, we get, okay,
- 17:16one is good right here, right?
- 17:18We talked about IM211 at beginning.
- 17:20We saw survival benefit.
- 17:22We see that the survival benefit coming
- 17:24into the 12 month zone, but RMST was still showing, okay,
- 17:30there is no confirmation of such an effect.
- 17:34So there has been a interesting paper
- 17:36I came across last year.
- 17:38It was a patient voice survey in UK.
- 17:41They basically did a survey on patients
- 17:45as well as the health practitioners
- 17:49who are participating in the TACT trial.
- 17:53Taxotere is a adjuvant chemotherapy drug.
- 17:56It's a very big trial.
- 17:57And they surveyed that what patients really want,
- 18:01how they want the trial results are for.
- 18:04I mean, because at the end of the day,
- 18:07the doctors tried to discuss these results
- 18:09with the patient before they choose as a therapy.
- 18:11And then, it's not surprising that actually,
- 18:16it's coming in that the patients are really want
- 18:19to understand these results in a very simple term.
- 18:22I mean, it's not like,
- 18:23oh, your drug may not be decreased event rate,
- 18:27but it expected survival going to get really bigger.
- 18:31But that's not the question a patient is asking, right?
- 18:34I mean, it's not type of mindset a patient has.
- 18:37Then of course, the patients are one who have the results
- 18:41as soon as possible.
- 18:42It's a very interesting article I suggest to read.
- 18:44It's a very fun reading article
- 18:46to look into the patient voice, how patient wanted...
- 18:51Because most of the time patients are not being
- 18:53always communicated about trial results
- 18:56and how they wanted.
- 18:58It came out to be very nicely in this,
- 19:00especially most of them actually want still comfortable
- 19:05to have the results from their nurse or doctors.
- 19:09Because they're more comfortable to discuss with them
- 19:11rather than having a big seminar or formal paper.
- 19:16But really, let's think about the patient perspective,
- 19:21what question they really ask.
- 19:22The question they really ask are these, right?
- 19:25I mean, if we put ourself onto that shoes,
- 19:29the question is does this drug really work?
- 19:32What are my chances that I will do better
- 19:36in terms of survival or in terms of painful quality
- 19:40of my life if on the new drug compared to no treatments?
- 19:46So these are the much more simple question.
- 19:48Unfortunately, many of the well known methods are unable
- 19:53to address because I mean, you can indirectly address
- 19:57those question or you may have the study shows you have
- 20:02a benefit of pharma.
- 20:04But that really doesn't answer the question
- 20:07from that patient perspective.
- 20:10Which actually motivates us a little bit,
- 20:12that can we dig in?
- 20:14Can we try to see if we can use
- 20:17our modern statistical analytic methods
- 20:20to address some of that question,
- 20:21especially from peer's perspective,
- 20:24from a kind of practitioner's perspective.
- 20:30So one of the thing that is more easier for anybody
- 20:34to understand the visual graphics, right?
- 20:36Kaplan-Meier plot is something which is more easy
- 20:39to comprehend compared to many other metrics.
- 20:42So that was kind of how our motivation (indistinct) became.
- 20:45And then the question is really,
- 20:47it's not like what trial shows.
- 20:49Now the question, if a new patient is entering the trial,
- 20:52can we predict their benefits based
- 20:57on the available data we have?
- 21:01And this could be an additional metrics.
- 21:02Of course we are not saying these metrics is gonna change
- 21:06the trial or the trial practice,
- 21:08but this is definitely another metric
- 21:10which can help patient much better
- 21:12for making their decisions.
- 21:16So what we introduced is basically this quantity
- 21:20which is called individual effect, Y minus X.
- 21:24What is Y?
- 21:25So Y is a survival time at the treatment arm.
- 21:30I'm just making simpler, instead of progression survival,
- 21:33let's have the conversation on survival,
- 21:35because it's just easier to understand.
- 21:38So Y is basically if a patient receives treatment
- 21:45and control, Y means their survival time in treatment
- 21:49and X is the survival time in control.
- 21:53But in a real trial, there was never been one patient
- 21:56who received both treatment and control.
- 21:59So this quantity is actually counterfactual.
- 22:03We cannot have that data.
- 22:05So we somehow have to predict this.
- 22:09So in order to do that,
- 22:11we first need to understand the marginal distribution
- 22:14of Y and X.
- 22:15And then also understand the need
- 22:18to take account the correlation in Y and X.
- 22:23Let's dive in what we need to do in this step,
- 22:25and I'll go into details of the statistics
- 22:27in next of my few slides, what technically it means.
- 22:31Basically, we are trying to find,
- 22:35so what we are trying to do,
- 22:36we are trying to find this predictive patient level effect
- 22:42of a drug.
- 22:44So what we need to have for that?
- 22:46We need to have a marginal distribution
- 22:49of survival for both,
- 22:52if a patient is independently goes to treatment
- 22:56or in control, basically.
- 22:57We need the marginal distribution first.
- 23:00Then we somehow need to calculate the difference
- 23:04by considering the association between them.
- 23:07Because Y and X are coming from a same patient.
- 23:13So first thing was interest,
- 23:15first thing was kind of how to do the marginal distribution
- 23:19of Y and X.
- 23:20That part is easier.
- 23:22That part is technically maybe intrigued, but easier.
- 23:27So suppose any trial data we have,
- 23:29we can have multiple trial data.
- 23:31We look into the treatment, and we look into the control.
- 23:35I just, for simplicity,
- 23:36suppose we have one one trial in our hand.
- 23:40And if there are multiple,
- 23:41we can definitely add more layers to that
- 23:43and adding trial specific event.
- 23:45So we can fit a...
- 23:47Piecewise exponential models are more,
- 23:50kind of a flexible model.
- 23:51So of course, one can use (indistinct),
- 23:53other parametric family.
- 23:54The reason we are moving into parametric
- 23:56because we wanted to extrapolate.
- 23:58We wanted to see the survival in the future.
- 24:01So that's why we chose basically
- 24:03piecewise exponential graph where each of the within,
- 24:08so the whole time axis is divided into different time span,
- 24:12different time points,
- 24:15and then we assume the hazard response within that.
- 24:18But just not, we don't assume any hazard ratio.
- 24:21We are (indistinct) treatment effect responsive.
- 24:23We are fitting this piecewise exponential separately
- 24:26for treatment and control, basically.
- 24:29So there is no proportional hazard option.
- 24:32And then here, we assume that alpha and beta (indistinct)
- 24:35and using the, basically, the gamma prior
- 24:38for each of the interval specific hazard.
- 24:41And we assume non-informative practice.
- 24:44Most often we only have the trial data,
- 24:45not much information,
- 24:47but if they have more information from early trials,
- 24:49you can use informatic (faintly speaking).
- 24:51But one of the major challenge always
- 24:54for piecewise exponential is
- 24:55how you choose cut points there.
- 24:59There's like the cut choice of cut points,
- 25:01people often do eyeballing, right?
- 25:03They look into the plot, eyeball the times,
- 25:05but those are mostly subjected.
- 25:07That means one prediction to another prediction,
- 25:10it can vary there, which is problematic.
- 25:13We need a a little bit more uniformal way
- 25:17of selecting these cut offs.
- 25:19The second more easier one can think, okay,
- 25:22I use the person notes there.
- 25:24But here the problem is when we fixed the personnel,
- 25:27maybe from one plot to another,
- 25:30there may be intervals which doesn't have much event.
- 25:33So (indistinct) may not be basically calculated
- 25:37in a right way.
- 25:40So not stairwise.
- 25:41There is no event within that interval.
- 25:45So what we did, we basically looked into
- 25:47a optimal cutoff points searching algorithm.
- 25:50So we basically first divided the cut,
- 25:54the axis based on the person health.
- 25:56So you have 10 intervals to...
- 25:58So basically, at most, 10 intervals.
- 26:00So then we consider all possible models.
- 26:03So one component, two component,
- 26:05to the power 10 component model.
- 26:08Of course, if some of intervals doesn't have very few event,
- 26:11we basically merge them
- 26:12so that you have a reasonable estimate
- 26:15for (indistinct) basically.
- 26:17And we chose the best model based on the DIC
- 26:20among those two to the power, N number of models.
- 26:22Of course, there's not always 10
- 26:23beause some of the intervals may be empty, so we plot them.
- 26:28And one can actually do a k-fold cross validation as well,
- 26:32which is we also looked into kind of a giving more
- 26:35or less similar result.
- 26:36But if you have a long term data,
- 26:38one can also do a k-fold validation
- 26:40in order to choose the hard points.
- 26:44Now the second part is prediction, right?
- 26:47So prediction, so what you got from the model now
- 26:51is that distribution of (indistinct).
- 26:53That's parameter.
- 26:54But when you go to prediction,
- 26:56we are talking about sampling space now.
- 26:58So we are talking about a new patient's survival time.
- 27:02So that needs to take account uncertainty of the new sample.
- 27:07That's the beauty of the Bayesian distribution,
- 27:09the posterior predictive distribution
- 27:11automatically does that.
- 27:12And the setting, the reason we took this setting,
- 27:16because the predictive posterior parameter distribution
- 27:19is again, a piecewise (indisitinct) distribution,
- 27:22which is basically closed form,
- 27:24that first hour computation quite a bit.
- 27:27And then we basically use, as I say,
- 27:30it's can easily done with this Beyesian computation.
- 27:33But if you want to do more complex model,
- 27:36we'll one, need to move into a little bit more
- 27:38MCMC algorithm or kind of writing and way of sampling it.
- 27:45Now the third aspect, which was actually the more,
- 27:48most interesting aspect.
- 27:50Now we got the marginal distribution.
- 27:52We predicted the marginal survival times.
- 27:56Now the question is, it's the same patient, right,
- 27:59we are talking.
- 28:00So they're correlated, X and Y.
- 28:02How we most are correlation structure, which is meaningful?
- 28:07Now that thing was actually the idea was came
- 28:11from a very old paper by Lehmann and Doksum in 1974,
- 28:16which we were looking for a scale shifting distribution
- 28:20I'm going into in a moment.
- 28:21Which actually brings up a very important property.
- 28:24And that property is very important
- 28:27because this methodology, it depends on that property.
- 28:30And we can talk about in the setting
- 28:33where you think this property is not true.
- 28:36So the property is basically a rank preserving property.
- 28:39What that means, that means if a two population,
- 28:42so if one fail earlier in control,
- 28:49they will fail also earlier in treatment.
- 28:51That's a rank preserving property.
- 28:53That's a very important property for this.
- 28:56But one can also question that, okay,
- 28:58for targeted therapy, that may not be true.
- 29:00Somebody with a biomarker that order may change, right?
- 29:03So we need to do appropriate adjustments
- 29:06to make that assumption.
- 29:07I'm going into that.
- 29:08That was one of the referee comments, by the way.
- 29:12But before going into that,
- 29:13this paper was really fascinating.
- 29:15I mean, it was very simple paper in actually 1974
- 29:20and also of statistics, very simply written paper.
- 29:24And what they were saying that they were looking
- 29:28into distance between two normal curves there.
- 29:33And the property they introduced was scale shifting,
- 29:36a shift function, which basically makes that,
- 29:41so your X and Y, what you just mentioned,
- 29:43we said X plus delta X,
- 29:45which is one can interpret as gain also,
- 29:49and Y kind of have a same distribution.
- 29:54That was the main property.
- 29:56But it's very, basically, that means you project this curve
- 30:00to other curve.
- 30:01There is a path to project all.
- 30:05And the solution of delta X is,
- 30:07I mean, what can easily cost start that it's not unique.
- 30:10But they actually, this is how they constructed it
- 30:14using the shift function.
- 30:17But what did that means for our case?
- 30:19Why we need that, right?
- 30:20So this means that, so if we,
- 30:23if I know marginally, the (indistinct) tells of X,
- 30:26I can project the same content into the Y distribution,
- 30:30that basically we'll coordinate ourselves.
- 30:32So that means if we build, if the ordering of X remain,
- 30:37ordering in Y will remain too.
- 30:38That is what the rank preservation property's coming in.
- 30:43I'm sorry, I'm going to revisit the...
- 30:45I mean, like I said, sometimes simple papers give you
- 30:48very nice ideas as I find.
- 30:51So now the job is simple, right?
- 30:52Now we have marginal distribution in hand.
- 30:56We get the predictive distribution,
- 30:57how to predict the marginal, and now we know
- 31:00how to link them using the quantile function
- 31:03from one to another, how to project.
- 31:05So basically what we did,
- 31:06we simulated from this posterior distribution
- 31:10and then we simulated this uniform numbers.
- 31:13And then what we did in order to bring X and Y to related,
- 31:17we basically from same quantile,
- 31:20we obtained the X for given US,
- 31:23we obtained the quantile XS and YS for each case.
- 31:27So they're related by that way.
- 31:29We project it from X to Y
- 31:31and then that gives us a pair XS YS for them
- 31:35and we can make the distribution.
- 31:41But of course, this is questionable, right?
- 31:44I mean, we are saying the order remains,
- 31:46which is not always true.
- 31:48Suppose this is a very classic example
- 31:52from the nivolumab trial.
- 31:55So it's attacking the PD1 inhibitor.
- 31:58So people with PD1 expressed, it's supposed to work better.
- 32:03So if you pick two subject,
- 32:05one is PD1 expressed and PD1 expressed not,
- 32:08if in control, PD1 expressed one actually works worse
- 32:12than the PD1 non-expressed treatment that can reverse.
- 32:15Because PD1 is the line, the target of the truck.
- 32:19So on such a case,
- 32:21our solution is basically divide the groups
- 32:24into the homogenous biomarker class.
- 32:26And then, the (indistinct) still works.
- 32:28And that can be easily done adding it as a regression
- 32:32into the model.
- 32:35So then finally, we summarize this
- 32:42with a survival gain and out the loss of tests,
- 32:45I mean basically using the posterior, sorry,
- 32:48the predictive distribution of Y minus X.
- 32:52And then also, we summarized the median
- 32:56and the 95% prediction intervals.
- 32:59And also to us, the Kaplan-Meier plot is still important
- 33:03because that's where the whole story begin.
- 33:06That's the data which generates our marginal distribution
- 33:09where we actually...
- 33:10So we should compare that side by side on this.
- 33:16Let's see how this method works then.
- 33:18I mean, so we started, does it improves anything
- 33:20after all doing fancy things,
- 33:23make things complicated, making lot of mathematical names.
- 33:27Did you improve anything?
- 33:28So did you gain more insight into this?
- 33:32So let's go back to this trial
- 33:34where begin the urothelial cancer trial of atezolizumab,
- 33:37which is basically the low-rank test says
- 33:41it was basically...
- 33:43The survival curves are not separated.
- 33:46Basically significance that doesn't reach.
- 33:50The Cox has a ratio, upper bound is about one,
- 33:54even the median, because it's very interesting.
- 33:56Because the curve actually is separated after median.
- 33:59So even somebody looking into the median,
- 34:01the treatment is worse than that for the standard of care.
- 34:05So the question is really, how can we...
- 34:08But when we look into the survival differences,
- 34:11we see a significant survival gain by the patient
- 34:14who remain on the therapy for a longer time.
- 34:16But the question is, can we somehow communicate
- 34:24that better with this individual event?
- 34:29So the one on the left hand,
- 34:32one on the left side is basically the probability plot.
- 34:37So it is basically the first column.
- 34:40We looked into the gain of survival bigger than zero month,
- 34:45one month, two month, six month.
- 34:48And the plot on the right side is basically median
- 34:51and then predicted interval.
- 34:53So as you can see, there is an initial setback.
- 35:00There is a significant probability
- 35:03that atezolizumab can actually improve the survival
- 35:07in the problem.
- 35:08At least you have,
- 35:09if you're looking to this plot,
- 35:11the improvement of three month or higher,
- 35:14which is urothelial cancer is pretty good,
- 35:16you have 30 to 40% probability,
- 35:18which needs to be considered for these patient.
- 35:20Because they don't have,
- 35:21they only have chemotherapy as a treatment for that.
- 35:25The most interesting thing comes actually
- 35:28on the right hand side.
- 35:30If you look into that, the patient as we talked about,
- 35:34the benefit actually emerges as the patient went long term
- 35:38into the therapy.
- 35:41It start pretty much, much (faintly speaking) to others,
- 35:45compared to others.
- 35:49Let's look into another.
- 35:50So okay, non-proportional hazard,
- 35:52this may be useful.
- 35:53But can we still use that for proportional hazard?
- 35:56Do they have any value?
- 35:57Actually, do they add anything?
- 35:59'Cause they're log-rank has a ratio are more popular, right?
- 36:02Our argument is no,
- 36:04but maybe this measure can help you there too.
- 36:06So I go back to this lung cancer PA3 example
- 36:09of gemcitabine and gemcitabine plus erlotinib,
- 36:13which is statistically significant
- 36:14with a very marginal hazard ratio.
- 36:17But statistically significant
- 36:20and the media advantage was also not very good
- 36:24by only .3 months of advantage.
- 36:27The question is really,
- 36:29does the survival effect is meaningful in that way?
- 36:34Always.
- 36:37So again, we started to plot these two.
- 36:42So the one here is basically the plot
- 36:47for the survival gain, X minus Y, once again, control.
- 36:51Same patient treated in treatment
- 36:53versus same patient treated in control.
- 36:55What is the survival gain?
- 36:57And the one on the right is basically the median
- 37:01and the corresponding interval.
- 37:03As you can see here,
- 37:05especially the patient who discontinued
- 37:09or there are some questionable benefit on that question.
- 37:12So just having a proportional hazard
- 37:14and giving a hazard ratio may not be giving the full point
- 37:19that we are looking for.
- 37:20There may be some more to it,
- 37:22which can be further investigated.
- 37:25When it's prescribing a patient,
- 37:28maybe there are certain characteristics
- 37:30why the patients may be continuing early
- 37:34and some kind of (indistinct) and some kind of special...
- 37:37So those patient may not be the benefit is as good,
- 37:40as prominent as the patients who are long term treated.
- 37:46So we also investigate CM.
- 37:48I just don't want to show all the plot just not
- 37:51to bore anymore, but messages are very similar.
- 37:53We also look into CM141.
- 37:57We also look into that ASPIRE trial
- 37:59as I explained you earlier about multiple myeloma
- 38:03and the CheckMate141 is basically squamous cell carcinoma
- 38:08in head and neck.
- 38:10And we also looked into the CheckMate057 trial
- 38:14where there's a significant subgroup
- 38:17that means the one group with PDL-1 expressed
- 38:23has a significant survival.
- 38:25But PDL-1 non-expressed has no survival benefit
- 38:33with the (indistinct).
- 38:36So we looked into all these examples
- 38:38into the export data and our codes are also relevant
- 38:42in public if you want to play with that.
- 38:44Basically the message we got,
- 38:46so the one on here is that this table
- 38:49toward the mean survival gain,
- 38:52median survival gain, the predictor interval.
- 38:55And what is the chance that probability
- 38:57that Y, which means your time,
- 39:02if a subject is receiving treatment
- 39:05and X if a subject is receiving control,
- 39:08what's the time Y is bigger than X,
- 39:11is basically we started to see some interesting feature.
- 39:15It gives us much more quantification of benefit compared
- 39:19to our P value or a hazard ratio in that way.
- 39:22Of course there are uncertainties in all of them
- 39:25as well as the clear cases
- 39:26where we saw statistical significance.
- 39:29We see some uncertain situation, that patient may
- 39:33or may not be benefit in some cases.
- 39:37So the quick conclusion with the increasing complexity
- 39:42of the drug and the different complex pathways
- 39:45we are talking, it is very important.
- 39:47And different patterns of treatment effect,
- 39:51it's very important to have a simpler language
- 39:55with the patient.
- 39:56I mean, because we're all telling what they're answering
- 39:59their question at least directly.
- 40:01We find predicted individual effects sort of a step towards
- 40:05that, which answers the patient's question more clearly,
- 40:11kind of a clinically relevant.
- 40:12And also it step forward,
- 40:14and as well as it supports this 21st century cure act
- 40:19of patient centricity.
- 40:21Especially find very useful in where your standard is
- 40:26of proportional hazard fails.
- 40:31I just wanted to quickly thank my collaborators
- 40:34whom I collaborated with and here is outpatient
- 40:39along with the software is available
- 40:41in case you want to try out.
- 40:43And the references that I mentioned
- 40:45including actually the...
- 40:47I just wanted to always mention,
- 40:50if you are reading the famous Lehmann book,
- 40:54it's actually there as well.
- 40:55I just find out last night it's there as well.
- 40:59And not only (faintly speaking) statistics people.
- 41:03And I just want to thank you for your attention.
- 41:06Thank you.
- 41:18<v Presenter>Does anybody have any questions?</v>
- 41:22<v Attendee>Sure.</v>
- 41:23Is there a notion of why you get that kind of culling effect
- 41:27with the treatment groups you were showing
- 41:30in the survival curves that the,
- 41:31basically the treatment arm looked worse at first.
- 41:33<v ->[Dr. Roychoudhury] Yes, I think for a (indistinct),</v>
- 41:38I think there was a certain group biomarker.
- 41:42It's again, a heterogeneity of treatment effect.
- 41:45Certain biomarkers did, I mean,
- 41:46most of the non-proportional hazard are the same story.
- 41:49I mean, the certain groups didn't function well
- 41:51at the beginning until they basically received the treatment
- 41:54follow up as well.
- 41:55They actually worse, they were quite a bit.
- 41:59<v Presenter>Okay, so do you think it's a treatment effect,</v>
- 42:01not a property of the population that was in it?
- 42:04<v ->[Dr. Roychoudhury] I think it's, I mean,</v>
- 42:06it's more of a safety of words I guess I believe.
- 42:09But of course, it's a road we don't know all the details
- 42:13inside of it.
- 42:15But the compound, which is the results which is available
- 42:17in that paper, it seems like there is an effect
- 42:20where it's really detrimental, the treatment effect.
- 42:23(faintly speaking)
- 42:37<v Presenter>Any other questions?</v>
- 42:43All right, anything from Zoom land?
- 42:56<v Attendee>Yeah, I have a question.</v>
- 42:59So it's interesting to model non-proportional patterns,
- 43:03but I think maybe another interesting question
- 43:07is to why there was non-proportional pattern
- 43:10(faintly speaking), right?
- 43:11So maybe (faintly speaking).
- 43:15So say for example if we owe something
- 43:18like a random voice or like classification,
- 43:21(indistinct) then we'll be able
- 43:23to see each subgroup benefits from the treatment
- 43:27or like why this (faintly speaking)?
- 43:32(indistinct) comment something else,
- 43:34like the modeling, the causes or figuring out
- 43:38why there's non-proportional patterns.
- 43:40<v ->[Dr. Roychoudhury] Sure, I think what you just said,</v>
- 43:43that was basically the method, some of five star,
- 43:47some of the method that more people develop.
- 43:50(indistinct) and their group did.
- 43:52They basically looked into a elastic net
- 43:57to find out the sets were basically you have,
- 44:01which is heterogeneous.
- 44:03They divided the group,
- 44:05this heterogeneous clusters, basically, into that.
- 44:08And then tried to interpret treatment of problems.
- 44:11I mean, the major problem is sometimes
- 44:15those groupings are very hard to interpret.
- 44:18Because it's so much data driven, right?
- 44:21And secondly, specify such a method as an analysis
- 44:25and this is a great method to exploration.
- 44:27I fully agree.
- 44:28But if you think about a drug
- 44:29and kind of a reporting of a drug,
- 44:32that could be a very risky method to do.
- 44:34But definitely, they are thinking down on that avenue.
- 44:37The only reason I think the five star was
- 44:39a very interesting idea, the only problem really came
- 44:43in is the estimation of treatment effect
- 44:45at the end of the day.
- 44:46Because now, you have a selection, right?
- 44:49Now you have to have the selection probability incorporated
- 44:54into the treatment effect.
- 44:56Some clusters are so small when you put the adjustment
- 44:59to the selection probability,
- 45:01it's not very intuitive to non-status station anymore.
- 45:05But it's been done.
- 45:07I mean, there are an example,
- 45:08I think in (indistinct) medicine,
- 45:10if you search by five star, you can see that.
- 45:13I think that the major got hit by the interpretation
- 45:16of the treatment effect.
- 45:17But you know what they did?
- 45:19They actually fit a parametric...
- 45:22First, they did three things.
- 45:24They looked into each set,
- 45:26because those population are homogenous.
- 45:28So they fit the Cox regression model there.
- 45:31And also they looked into a parametric regression model.
- 45:35But the only problem is,
- 45:37as soon as you adjust for your selection probabilities,
- 45:42if you have a huge effect, right?
- 45:44The selection probability somehow do a tool on that,
- 45:50which is clinicians don't find very intuitive, that case.
- 45:54Because at the end of our regular...
- 45:56I mean, how do you put such a thing on a drug level?
- 45:59That is a problem.
- 46:01But I think such a thing should be done for our...
- 46:04If we have already face data, we should explore this.
- 46:07I really think that should be the case.
- 46:24<v Presenter>Okay, so we don't have any questions.</v>
- 46:27Let's thanks again.
- 46:29<v Attendee>Yeah, I have one, just one quick question.</v>
- 46:32When you're predicting your why,
- 46:35why not augment that data with publicly available data
- 46:40based on features like comorbidity, age,
- 46:43some of the known predictors in terms of survival rates?
- 46:49<v ->[Dr. Roychoudhury] Absolutely, absolutely.</v>
- 46:52Sorry, I skipped that.
- 46:54But that's a great question.
- 46:57Actually, if we see that, I just wanted to go...
- 46:59We actually said if you have such a thing,
- 47:01we just convert this into a regression.
- 47:04So you can actually,
- 47:05instead of having just a unstructured model here,
- 47:08we can actually plug in all the rigorous (indistinct).
- 47:10But of course then, you don't have (indistinct)
- 47:13in your family anymore.
- 47:14It'll be conditionally extensive.
- 47:16But that's a very easy extension of this.
- 47:18Yes, absolutely.
- 47:20Absolutely.
- 47:27We actually did that for, sorry.
- 47:31We actually did that for this example
- 47:33where there's heterogeneous effect by the biomarker.
- 47:37You basically fill in the regression,
- 47:40and that's how we operate.
- 47:42Sorry, I interrupted.
- 47:43Somebody was...
- 47:45<v Attendee>Great, thank you.</v>
- 47:47<v Attendee>Oh, I have another question.</v>
- 47:50So when you tell the patients
- 47:51about their predictive treatment effects,
- 47:54if that affects I'd say how patients attach
- 48:02to the assigned treatment,
- 48:07that some way actually affects how you estimate your,
- 48:14for example, the first step for the estimates.
- 48:18And does that actually affect the way how you...
- 48:24Because I think estimation
- 48:26and also the following steps are actually the different,
- 48:30for example, the purpose of clinical trial
- 48:33will be estimation.
- 48:35But the purpose of, like say telling patients
- 48:38the individual treatment effects,
- 48:40a predicted individual treatment effects will be
- 48:43like a different purpose.
- 48:44It's not really the estimation for the clinical trial.
- 48:47Those two can become comfort when you, for example,
- 48:52using a patient algorithm to update your,
- 48:54like say marginal distribution.
- 48:58And then the next step is telling like say patients
- 49:02about prediction.
- 49:04And for new patients coming in,
- 49:06you do the marginal distribution estimation again.
- 49:13And does that actually pose a little bit of a problem
- 49:17when prediction actually change people's mind
- 49:20about their, like say how they attach
- 49:24to the assigned treatment?
- 49:25<v ->[Dr. Roychoudhury] Yeah, I think yeah.</v>
- 49:28That's a very valid question.
- 49:30I think that's the main reason we needed
- 49:33this algorithm, right?
- 49:34And we did not stop just marginally.
- 49:37But if you look into the new data that's coming in,
- 49:41of course we need to be careful
- 49:42because that's not coming from a clinical trial data.
- 49:47And this one is a more simplified
- 49:49because the third step was calculated
- 49:52from clinical trial data,
- 49:53which is a randomized study.
- 49:55But if you add more observational data to it,
- 49:57if I understand how they're touched
- 49:58and we continue to update that,
- 50:01it just need to be more careful
- 50:03about using two different quality of data in that way.
- 50:07Does that answer your question?
- 50:11<v Attendee>Sure, thank you.</v>
- 50:15<v Presenter>Okay, so we are running out of time,</v>
- 50:19so let's thanks Dr. Roychoudhury again.
- 50:22Wonderful talk.
- 50:24<v ->[Dr. Roychoudhury] Thank you.</v>
- 50:28<v Presenter>Please make sure you sign</v>
- 50:31the admit sheet.
- 50:33(attendees chattering continues)
- 50:53(attendees chattering continues)