YSPH Biostatistics Seminar: “Biostatistician Roles in the Pharmaceutical Industry”

October 05, 2023

Information

Glen Laird, PhD, Biostatistics Senior Director, Vertex Pharmaceuticals, Inc.

October 3, 2023

ID10815

To CiteDCA Citation Guide

00:01<v ->For this time.</v>
00:03So we're
00:04(presenter muttering indistinctly)
00:05<v ->All right, so hey, everybody, welcome.</v>
00:08Today's my privilege to introduce Dr. Glen Laird.
00:11Dr. Laird earned his PhD in statistics
00:13from Florida State University in 2000,
00:16then worked as a survey statistician
00:17for RTI International before joining
00:20the pharmaceutical industry
00:22where he worked at Novartis,
00:24Bristol Myers Squibb and Sanofi.
00:27And so now, he's at Vertex Pharmaceuticals.
00:31And so let's welcome Dr. Laird.
00:37<v ->I hope everybody can hear me also online.</v>
00:41I hope we can have a good discussion today.
00:43Have a lot to talk about.
00:44Feel free to interrupt me at any time with questions.
00:48There's really nothing overtly technical here,
00:51so I wanna be very accessible to everyone.
00:54I'd like to hear your feedback go along.
00:58So I'm gonna be talking today
01:00about industry-sponsored clinical trials,
01:02that is pharmaceutical industry sponsored trials.
01:08So disclaimer, I work for Vertex,
01:10but any opinions are mine, not theirs.
01:14So for a clinical trial,
01:18you're gonna have a clinical trial team, right?
01:20At Vertex, we call it a study execution team.
01:22Other companies call it something different,
01:24but it's the same kinda thing.
01:25It's a group of people who are responsible for running,
01:28conducting, executing the trial.
01:31It's gonna vary by the study,
01:32but usually, this is gonna include a clinician of course,
01:36who's gonna make the key clinical decisions about the study.
01:39An operations person's gonna do a lot of coordinating
01:42with the site, a lot of communication with the site
01:46actually conducting the study.
01:48Also shepherding documents through reviewing,
01:50things like that.
01:52Clinical pharmacology, they deal with pharmacokinetics,
01:55which is how the body processes the drug,
01:59metabolism, that sort of thing.
02:02Safety,
02:03at some point, FDA let it be known that they wanted you
02:07to have a person explicitly responsible for safety
02:12on your study team, so.
02:14Because then, I think there was kind of a mindset
02:16that if you had the same person
02:17trying to look at safety and efficacy,
02:21that they would probably end up spending most of their time
02:23looking at efficacy.
02:25Safety might not get the attention it deserves,
02:27so you have to have a person explicitly for safety.
02:30Clinical biomarkers,
02:31we often like to look at a lot of different biomarkers.
02:34Data management deals with the actual database itself,
02:38setting it up
02:40and the sort of execution around locking it
02:43and all that sort of thing.
02:44And cleaning the data.
02:48The statistical programmer is responsible for a lot
02:52of the actual execution of the various plans, right?
02:56So, and, of course, the statistician,
02:58which I'm gonna talk a little bit more about.
03:01The statistician and the programmer
03:02really work kinda hand in hand for a lot of things, right?
03:05There's a lot of things where the statistician
03:07is planning things, specifying things,
03:09and the programmer is the one writing the code
03:12to actually execute it.
03:18FYI, so what I just talked about was a study level team.
03:22There's also a project level team.
03:24So by project, I mean a drug or a therapy, right?
03:28So there'd be some more senior people.
03:30So there would be like a project level statistician
03:32and a team at the project level
03:34with a lot of these same similar functions plus some others.
03:38Legal, for example, comes to mind.
03:39That project team kinda guides
03:42the overall development of the drug.
03:45But today, I'm gonna focus more on the study,
03:51what the statistician and that team is doing.
03:54A lot of you may know this,
03:56but there's four sort of commonly recognized phases
04:00in drug development.
04:02Phase one is mostly about safety.
04:04You're trying to find the right dose of the drug.
04:07Phase two is kind of an initial assessment of efficacy,
04:11whether you think the drug works.
04:13Main purpose of that is to convince yourself
04:16whether you want to do phase three,
04:18which is the pivotal study,
04:21the main bulk of your evidence that you claim to submit,
04:25to say, "Here's our evidence that this drug works."
04:29Right, that study is often the biggest
04:33and it's generally randomized, right?
04:37And then there's phase four,
04:38which would be anything that's post
04:40(Glen muttering indistinctly)
04:41right, and those kinda studies
04:43can depend on the market conditions for your drug
04:46after it's gotten on the market.
04:50I'm gonna focus the most on the phase two,
04:52three type studies
04:54'cause that is sort of the most classic
04:57clinical trial experience.
04:59And it's perhaps the part where the statistician
05:02and the programmer are really the most key to being
05:06and their involvement.
05:08That is the scientific rigor of actually demonstrating
05:12this drug works.
05:17And as I noted the bottom there,
05:20the great majority of drugs that start in phase one
05:22end up dying somewhere along the way unfortunately.
05:26You can look up various numbers,
05:27but it's a pretty small percentage and actually end up
05:30making it to the market, unfortunately,
05:32from direct to start in phase one.
05:36Oh, okay.
05:37All right, so now we're at the survey here.
05:40All right, so all right, then.
05:42So then. <v ->Yep.</v>
05:43<v ->This is my survey question, hope everybody.</v>
05:47<v ->Oh, and then just hit present.</v>
05:48<v ->And then I need to do this,</v>
05:52so, okay.
05:55So I'm wondering what you think.
06:00So when in the life of a study do you think is the most work
06:05for the statistician?
06:08So if you can't see there, so option A,
06:12these plots are qualitative, it's conceptual, right?
06:16So the x-axis is time,
06:18the y-axis is the amount of work, right?
06:20So option A would be level, you know?
06:23It's basically the same amount of work over the whole course
06:26of the study from when you first start conceiving the study
06:28until you rep the study before, right?
06:31Option B is going up and up and up,
06:33getting busier and busier and busier
06:35the longer the study goes on.
06:37C is the opposite. Start very busy, gets less and less busy.
06:41D is Gaussian looking, right?
06:46There's a bulge of work in the middle.
06:48And E is kind of the opposite of that.
06:50A lot of work at the beginning and the end,
06:52maybe a bit of a lump.
06:56So I know people know how to fill this out or.
06:59<v ->Yep, text.</v>
07:00<v ->Whatever.</v> <v ->Get our your phones,</v>
07:02which you don't hear often.
07:03(Glen laughing)
07:05<v ->Yeah.</v>
07:14People online, I hope, are voting too.
07:20When do you think the most work is?
07:29Most people answered D.
07:35Maybe I don't know how to tell how many people,
07:37I hope it's more than. <v ->Yeah. (laughs)</v>
07:39<v ->I hope it's more than like six people that are voting.</v>
07:43I feel good when you see some prime numbers
07:45and stuff in there, it makes you feel like,
07:47"Okay, 10 must be big enough
07:49that you're getting something."
07:50But, okay, so it looks like most people say D,
07:55fair number of people say E,
07:57now, it's not a lot for the other choices.
08:01Like so do I just go back?
08:03And how do I go back? <v ->Yeah, you just go back</v>
08:04to that.
08:05<v ->Do I just hit escape?</v> <v ->Escape. Yeah, you can.</v>
08:12<v ->Present mode.</v>
08:16<v ->It's not working?</v>
08:19<v ->That's it?</v> <v ->Yeah.</v>
08:20<v ->I think we're out of present mode though.</v>
08:24<v ->Yeah.</v> <v ->There we go.</v>
08:26<v ->So in my opinion, I think most people</v>
08:28would agree with this.
08:29I would say the answer is actually E,
08:32the opposite of what most of you picked.
08:35And the reason for that is there's a lot of stuff
08:37the statistician has to do at the beginning of the study
08:39in terms of planning,
08:41specifying what kinda study are we gonna do,
08:44how are we gonna plan all kinds of stuff.
08:45I'll talk some more detail in just a minute.
08:48And then there's a lot of work reporting
08:49at the end of the study
08:51executing everything you said you were gonna do, right?
08:54And it's not uncommon that in the middle
08:56maybe there's a bit of a low
08:57where you're mostly kinda waiting for patients to enroll
09:00and everything is maybe blinded even.
09:03So you don't have it available, right?
09:06So what does the life of a study look like
09:10and what is the statistician doing during this study?
09:15So I'm gonna give you an outline.
09:17Again, it's just main steps.
09:19Don't take anything here too literally,
09:21this is just kind of my ballparking of things,
09:24way things tend to go at most companies,
09:28but companies in general are more alike than different.
09:30A lot of this process is actually quite standard.
09:33They just have little different flavors, you know,
09:35different tweaking of the timelines and such.
09:37But the general idea should be pretty consistent.
09:41This isn't covering special studies, targeted study.
09:45I'm talking about a sort of a classic, you know,
09:47phase three type study here.
09:49<v ->You wanna move that window?</v>
09:52<v ->Yes.</v>
09:53<v ->I'm sorry.</v>
09:55<v ->Thank you 'cause I've got that.</v>
09:57<v ->I know it's hard to figure out.</v>
09:57<v ->Stuff I want to, yeah.</v>
09:59So the first thing you notice here
10:01is that there's tons of acronyms, right?
10:03That's part and parcel in the industry.
10:06There's a lot of things here.
10:07But that right, I'll go through 'em.
10:08So the first thing here starts with protocol concept, right?
10:11So the protocol concept is basically a document
10:14that just gives you kinda the bare bones
10:15of what do you plan to do in this study?
10:18What's the disease?
10:19What kinda patients do you plan to enroll?
10:21What are you gonna measure on those patients?
10:23When are you gonna measure it?
10:26A little bit about how you're gonna analyze it.
10:28And, of course, the sample size, right?
10:30Which the statistician has to calculate
10:34how many patients you're gonna study, right?
10:36That gets reviewed by various functions,
10:38including, of course, biostats.
10:40And also gets reviewed by a PRC,
10:43which is a protocol review committee.
10:47And oh, they got blocked out a bit there.
10:49So FSFV is first subject, first visit.
10:53If you're studying patients,
10:54you often say first patient first visit.
10:56So those are really the same thing,
10:58just depending on whether you're actually studying patients
11:01that have the disease
11:02or just healthy volunteers for example.
11:06And so this gets reviewed by the protocol review committee.
11:10Again, that's one of those things
11:12that every company's gonna have
11:14one or more protocol review committees,
11:16but they're all gonna be,
11:17and they're gonna have a little different flavor,
11:19but it's gonna be pretty similar.
11:21So if it's approved by the PRC,
11:23then you come back maybe two, three months later, say,
11:27with a full protocol,
11:29which should be very similar to the protocol concept.
11:33You're just filling in more details
11:34of how are you gonna measure these endpoints, for example.
11:38You know, details on inclusion and exclusion criteria
11:41for exactly who gets in the study, some things like that.
11:44Still doesn't have all the statistical
11:48details in it, right?
11:49Has some high-level summaries
11:51of what kind of analysis you plan to do.
11:53But it's not table shells,
11:55it's not the real statistical rigor details.
11:59So let's say that gets approved by the PRC.
12:01Now I move on to case report forms or CRFs.
12:05Those are the actual forms
12:07where the site enters the data, right?
12:09So principle here is the sites enter the data,
12:13the sites change the data, we don't touch the data, right?
12:17We just talk to them
12:18about how they're supposed to do that, right?
12:20We don't touch it, we just query them and say,
12:22"Hey, do you need to change this data?"
12:25And then it's up to them to change it.
12:27So it's important, this is a process
12:30not driven by biostatistics, right?
12:32Operations and data management, run it,
12:35but it's important for the statistician to be there
12:38and the programmer to review it and look, right?
12:43Because if you don't have a good case report form,
12:45you're not gonna get the data you need, right?
12:47You're gonna be in a bind at the end of the study
12:50when it turns out the form didn't collect
12:52what you wanted to report.
12:55Similar to that, there's edit checks.
12:57So edit checks is something to respond to the site
13:01whenever they enter something that is questionable, right?
13:04So the site enters that the patient was 200 years old,
13:07that's gotta be some kinda typo.
13:08It's gonna immediately spit up something saying,
13:11"Hey, double check that number, right?"
13:14So edit checks are important in terms of getting good data
13:17in the system in the first place, right?
13:21PD specifications. So PD stands for protocol deviation.
13:26In the real world,
13:27things don't always go according to the protocol, right?
13:30There's often missed assessments,
13:33assessments that weren't done at the right time.
13:36Patients that were enrolled that actually weren't supposed
13:39to be enrolled according to the infusion criteria,
13:42various things in the real world may go wrong.
13:44And so the statistician plays a key part in specifying
13:47what you're going to do about those, right?
13:50So this is still at the beginning, right?
13:51This is before you've enrolled anybody.
13:53You're planning, okay, we can foresee that this may happen.
13:58What are we gonna do about it?
13:59You might have a,
14:00you might say, if patients are not enrolled,
14:03if patients are enrolled
14:04who don't have the treatment history we intended,
14:07then, for example, you might say,
14:08"We're not gonna include that.
14:09We're not going to include that patient
14:11in this particular analysis."
14:13Might be one thing you would pre-specify
14:15about how are you gonna handle
14:16that protocol deviation, right?
14:20The randomization request.
14:21Every company's gonna have a form
14:23the status session fills out to say,
14:24"Please do the randomization in this way."
14:27We almost always do some form
14:28of stratify block randomization, right?
14:31So anybody who maybe doesn't know, right?
14:34A block is a small sample size where you know
14:38the randomization's gonna work out even, right?
14:40So if your block size is four,
14:42you're guaranteed that two of those four
14:44are gonna be treatment,
14:45two of those four are gonna be controlled, right?
14:47It's just a matter of which two bits the order.
14:52So that helps enforce some balance, right?
14:54And then we're gonna have stratification factors.
14:57Those are often a common topic for discussion
15:01and exactly what are we gonna stratify
15:04for the randomization, right?
15:06So a statistician's very important in making sure
15:10and figuring out how that randomization is gonna be done
15:13and filling out the form properly, right?
15:17The data monitoring plan,
15:18that's more driven by data management,
15:21but the statistician needs to look at it.
15:24So the monitoring plan is like
15:26how are we gonna look at the data in an ongoing way
15:28during the study, right?
15:30So if say sites are not understanding the protocol,
15:34they're enrolling the wrong kind of patients,
15:36you wanna catch it as soon as possible, right?
15:39So you're looking at baseline data, blinded data,
15:42and trying to see if there's problems that could affect
15:45the scientific validity of the study.
15:51Okay, then just,
15:59there we go.
15:59All right, so during the study,
16:01I have the red box here around finalize the SAP,
16:05which is the statistical analysis plan.
16:09This is the single document that the statistician
16:11is most responsible for.
16:13And that's the document, statistician authors it,
16:17facilitates the review of that document.
16:20This is the document where you do put all those details,
16:24all the statistical nitty gritty details
16:26about how are you gonna handle missing data?
16:28How exactly are you gonna define the baseline?
16:31What are you gonna do?
16:32What covariance are you gonna put in your model?
16:34All these kind of details about exactly how you plan
16:38to do the analysis, right?
16:40And this also gets reviewed and approved
16:43by all the usual review machinery in the company, right?
16:48So notice about the timing.
16:51So if you have a unblinded study,
16:55you need to do this before you enroll anybody, right?
16:58Before first patient, first visit.
17:02If you have a blinded study,
17:04it can be done somewhat later
17:07after you've started enrolling patients.
17:09You still need to do it in time to allow programming
17:13to do the programming and validate stuff and all that.
17:15You can't do it at the last minute,
17:17but you don't have to do it before you enroll a patient.
17:20Does anybody have any idea why that would matter?
17:25Whether it's a blinded study or not on the timings?
17:32Somebody who doesn't have sandwich in their mouth, perhaps.
17:34(attendant chuckling)
17:36<v ->You have the big rooms.</v>
17:43<v ->I highlight this because this is another</v>
17:46very important principle of these studies,
17:48which is pre-specification, right?
17:52Things that you say and do after the data are known,
17:56after you know who's and what treatment group
17:59are considered post hoc, right?
18:02And they're going to be viewed,
18:04I'm not sure if suspiciously is quite the right word,
18:06but are gonna be viewed with additional skepticism, right?
18:10So before that, you start enrolling patients,
18:14or before the study's unblinded,
18:15you can still claim that you're pre-specifying things.
18:18Hey, when I said we were gonna do the analysis this way,
18:21I didn't know that this patient was in treatment
18:23and that patient was controlled, right?
18:25So you could still claim to be even handed
18:29when you do the plan.
18:30Then I'd say maybe comes the lull
18:33I was talking about, right?
18:36Maybe in the middle, yes, you're executing the data,
18:38monitoring stuff that you said you were gonna plan.
18:41That's not really heavily driven by stats.
18:46There's always gonna be team meetings.
18:48It varies, they might be say monthly.
18:50A lot of that is kind of study status things on enrollment
18:54and discussions about whether we need to do an amendment
18:57to the study.
18:58Again, not really driven by stats, right?
19:00So maybe there's a bit of a lull there.
19:02Then as you're starting to get closer
19:03towards clinical database lock,
19:07which is what CDBL is, right?
19:09Now, you want to do dry runs.
19:11So by now, programming has done the programs
19:15you want to execute those programs on some version
19:20of the data in order to see whether there's, you know,
19:24issues at the tables look fine.
19:25So a lot of times when people do the blinded study
19:29is you're gonna use dummy codes,
19:31you just make up false treatment assignments
19:35and you stick that in and then you run the table
19:37and you just kinda see where they're fine.
19:41Then you have to identify the protocol deviations.
19:45This is the part where,
19:46remember earlier you were planning
19:48how you're gonna handle the deviations.
19:50Now you can get close to database lock,
19:52you have to execute that.
19:54You have to apply it to the actual data and say,
19:56"Hey, just looking at the baseline data,
19:58I still don't know who's treatment, who's control."
20:00I'm gonna say that patient is not in that analysis
20:04because of the rule I said before
20:06and I'm doing this now before I know, right?
20:09So you have to do that kind of applying it
20:11and you have to sign off on that saying,
20:13"Here's the official call of who had what deviation."
20:18Then at the database lock, there's all the reporting stuff.
20:21You fill out a form to unblind the data.
20:25Usually very quickly, within a week or two,
20:27you have to deliver the key results.
20:30Vertex would call it the key reports memo or KRM,
20:33other companies call it something similar.
20:35But basically, within a week or two,
20:37management's gonna wanna know kinda the bottom line, right?
20:40And was the p-value less than 0.05?
20:43Was there some kinda major safety situation
20:45we oughta be aware of?
20:47That kinda thing, right?
20:48After that come the full list of tables,
20:52listings and figures.
20:53And then you have to finalize
20:54the clinical study report for CSR.
20:58And for that, you would have to author, you know,
21:01the statistical section of CSR.
21:04So that is kind of an overview of this is what a clinical,
21:08you know, study sort of looks like to a statistician
21:10and how you're doing, right?
21:17I'll note that it does vary some by the phase, right?
21:20To me, phase one, it's more exploratory.
21:24It's often unblinded.
21:26There's more kinda going on during the study
21:27'cause you don't really understand the drug yet, right?
21:30So there's amendments maybe more common,
21:33I think of it a little bit
21:34more like drug babysitting, you know?
21:37You're kinda like, "Okay, what's gonna happen today,
21:40you know, with each new dose that's going on?"
21:42So there's kinda more work to do during the study.
21:46People don't worry as much about the planning
21:49'cause everybody knows it's exploratory, right?
21:52Phase three is kinda the opposite.
21:53Everything I just said before, lots of planning, you know?
21:58Lots of trying to pre-specify things.
22:00Even things that are maybe somewhat unlikely,
22:04you know, very rigorous, right?
22:06It's 'cause it's often a very big study,
22:08it's very expensive, it's very costly,
22:11and a number of ways, if it fails, it's gonna be, you know,
22:14it could be pretty bad for the company
22:17depending on the situation, right?
22:18But the point is you have to carefully consider,
22:21you know, the details.
22:23There's more attention, more review by both management
22:26and, of course, health authorities like FDA.
22:31So for phase four,
22:33there's a group often called Global Medical Affairs.
22:38There's another group called Health Economics
22:40that often deal with these kind of studies.
22:43You can often look at longer-term safety and efficacy.
22:47They may address reimbursement.
22:50So reimbursement is kind of a bigger deal in Europe
22:53because they have single-payer systems.
22:57And so just because you get a drug approved by the EMA,
23:01which is kinda their version of FDA,
23:03that means you can sell it,
23:04but that doesn't mean
23:05the governments have to pay for it, right?
23:07You have to make a separate case to them to say,
23:10"Hey, not only does this drug work,
23:13it's actually worth what we want you to pay for, right?"
23:17There's a negotiation there.
23:19There gonna be a lot of publications involved in this.
23:22I don't know if you've heard the term real-world evidence
23:24or real-world data,
23:25but this is being used more and more in phase four.
23:29Once the drug is on the market in the real world,
23:33there's data related to that.
23:35There's insurance claims,
23:38there's electronic health records,
23:40things that weren't around back when I started, right?
23:44That can help you understand what's going on
23:47in the real world with your drug.
23:48And these are often very big datasets,
23:50but they can also be kind of messy in a lot of ways.
23:53Sometimes, there's a specific group for real-world evidence,
23:58but sometimes, that group is closely aligned biostats.
24:02Vertex has a group called
24:04(Glen muttering indistinctly)
24:05statistics,
24:06which is statisticians who are kind of particularly
24:09knowledgeable about dealing with these kind of data.
24:13People sometimes ask,
24:14"Well, what kinda statistics do you use?"
24:16Not really a good answer to that.
24:18It varies a lot by the disease you're using,
24:21by endpoint, I mean, variable,
24:23the outcome that you're measuring there.
24:26So it depends on the challenges of the setting.
24:31Like maybe sample size is a big issue,
24:34others may be missing data as a big problem.
24:37I used to work in oncology before I worked at Vertex.
24:40They use a lot of time to event endpoints,
24:42like time until the disease progresses.
24:44So they do a lot of survival analyses, right?
24:48Vertex, we don't do oncology anymore,
24:50so we have some time to event endpoints,
24:53but not that much.
24:54So the point is it just kinda depends
24:56on what you're studying.
24:58But, you know, companies understand that, you know,
25:01people aren't gonna necessarily walk in the door
25:03happening to be specialists in the exact kinda statistics
25:07that we're using right now.
25:09So, as an example of it depends on the setting.
25:14Vertex does a good bit in rare diseases.
25:15So I thought I'd just highlight a couple things
25:17about rare diseases.
25:19I'm not gonna go through all of these,
25:20but just in general,
25:22kind of the understanding of the disease and rare diseases
25:25can be limited.
25:27There haven't been a lot of studies
25:29conducted on this before.
25:30There's often not a lot of good prior information.
25:33Identifying patients can be difficult.
25:37You don't often get enough small sample sizes
25:39because there's not a lot of patients out there.
25:42A lot of these diseases are congenital, right?
25:47They're genetic, you're born with 'em.
25:49So a lot of the patients, I've read more than 1/2,
25:51are actually children.
25:52So, you know, that creates a whole nother aspect
25:55to the study if you're trying to study this in a child.
26:00A lot of use of innovative study designs, adaptive designs,
26:03things like that.
26:05Maybe I'll talk a little bit more about that,
26:08and a lot of use with biomarkers and modeling simulation.
26:13If you wanna know more about these sorts of things,
26:15I'll give you a shameless plug for a book.
26:19I'm actually not one of the editors of this book.
26:21These people are my coworkers in our department at Vertex.
26:28I contributed to some of the chapters.
26:30But I think it's a nice book
26:32in that parts of it are technical,
26:36a lot of it isn't,
26:37but it is written by quantitative people
26:39kind of with a quantitative focus on,
26:43or, you know, kind of through a quantitative lens
26:45on what one does and are disease drug development.
26:48So that's my plug.
26:52Little bit of organizational notes about how companies work.
26:56A lot of companies are organized by therapeutic area
27:00and or phase of development.
27:02Some companies have an early phase group
27:05that sort of all they do is phase one studies
27:08and they kinda crank out
27:08these fairly standardized phase one studies.
27:13Vertex is not that way actually,
27:15we just go by different therapeutic areas
27:17and have the same people who do the phase one study
27:21do the phase two, phase three studies.
27:25In general, a lot of companies
27:27are more alike than different.
27:29We have a similar regulatory framework, right?
27:31So like I said, FDA says,
27:33"We want you to do things this way."
27:35So everybody does things that way, right?
27:36We have a lot of the same employees.
27:38So, again, there's different flavors of things, right?
27:42Like the protocol review committee,
27:44they're all gonna have one.
27:45But some companies might have different
27:46protocol review committees for different types of studies,
27:50or maybe it's a little bit different
27:52how they set it up or, you know,
27:54but it's largely the same thing.
27:57For biostats,
27:59my advice would be to inquire with any sort of company
28:04you're thinking about working for or with.
28:07I would inquire about the methods group.
28:11Why do you think I say that?
28:18I'm the methods person,
28:19it's not because the methods group
28:21is the most important group, right?
28:26Why do you think I would say
28:29understand the methods group
28:30at whatever company you might be?
28:35It's actually related to what I just said.
28:37<v ->So maybe to stay on top of the latest trends</v>
28:41in the methods,
28:42make sure that you guys have time devoted for that.
28:46Stay on top of that.
28:49<v ->A very noble answer and kind of right.</v>
28:51(attendant laughing)
28:53I mean kind of
28:55in the sense that how you do that's gonna,
28:59you want to do that, but how you do that is gonna vary.
29:03I just said companies are more similar than different,
29:05but your methods group is an exception to that.
29:09It's actually not standard
29:10and it varies a lot by the company, right?
29:13So I used to work at Novartis, as we told you.
29:17Novartis has pretty much kind of an internal department
29:22of methods that it's almost like a mini academic institution
29:25within the company
29:26that they crank out academic-style papers.
29:31Pretty large group, quite technical in their focus, right?
29:34On the other extreme,
29:35I also used to work for BMS
29:38back when they had a site in Wallingford,
29:40they had no methods group whatsoever.
29:43You wanna do methods? It's your job.
29:45Do it on nights and weekends, whatever, right?
29:47So that's why I mean you're kinda right in the sense that
29:49if you want to do that, you need to understand like,
29:51"Well, am I gonna be working with something
29:53like the Novartis group
29:54or am I doing this all by myself, right?"
29:56So you might ask a board about me.
29:58At Vertex, I'm neither of those
30:00kind of triangulated to that.
30:02I don't have a group. I'm a one man group.
30:07And so I view myself as kind of a facilitator
30:12or a focus kind of person.
30:14So if people are interested in doing methods,
30:17I work with that person.
30:18I'm co-authoring some papers.
30:20I try to keep tabs on things that are going externally,
30:23that kind of thing.
30:24I try to help focus resources and utilize people
30:29who have interest and availability at that time
30:33maybe 'cause they're in that role, you know,
30:37as possible to look at topics
30:41that I can sense are of interest, right?
30:44But my bigger point is it's gonna depend
30:46quite a bit by the company.
30:48FDA's not gonna specify how you use a methods group.
30:52Really quickly, people often ask me,
30:54"Well, what's kinda the difference
30:56between people that are successful and not?"
30:58These are pretty high level, but in general,
31:00communication is important, right?
31:03Being able to make a point concisely, clearly,
31:07being able to communicate with non-statisticians,
31:10being able to give a presentation even in front
31:13of fairly large group of people
31:14and understand and explain your arguments
31:18for why you're doing what you are.
31:21Time management, like I said,
31:25there's a lot going on at a trial, you might be assigned to,
31:28you know, two, three, four, five trials, right?
31:30And they're all at a different point
31:32in that live curve, right?
31:33And so you need to be able to figure out
31:36how you're gonna manage your time
31:37across all those things, right?
31:39So, you know, you're here in school,
31:43maybe you have a job outside, you know,
31:45whatever, at the library, you know?
31:48People here don't care what's going at the library.
31:50Library doesn't care what you're doing here, right?
31:53So you might have five different studies
31:55and you may have to figure out, well,
31:58I need to do this on this study now,
32:00not because the team's telling me they have to,
32:02but because I know next month,
32:03I'm gonna have to do something else in another study.
32:06Right, so you have to kinda like juggle
32:08those different time commitments
32:09and that's something your manager would hopefully be able
32:12to help you with.
32:14But there's some skill in trying to figure that out.
32:18And just being generally proactive and visible.
32:21You want to,
32:25you wanna be seen.
32:26You know, you can give presentations, staff meetings,
32:28there's working groups.
32:29I'm involved with that kinda thing,
32:30which is kind of like a team approach to research, right?
32:34We see a topic that's of interest
32:36and we kinda divvy people up and okay,
32:38well, you can do the simulation,
32:39you go look at the literature.
32:41You know, something to get your name out there
32:45that people can remember you.
32:48But being the methods guy,
32:49I thought I should comment at least a little bit
32:51on some things I see going on in research right now,
32:56what my thoughts on are.
32:57There's a lot going on now with borrowing data
33:01and using real-world data, right?
33:03So people want to do a clinical trial.
33:07It might only be a single-arm study
33:10or it might be randomized,
33:11but they wanna try to use historical data
33:14or real-world data that are out there,
33:15sorta combine the two in a way that borrows strength
33:19and gives you a stronger conclusion.
33:26There's a lot coming out with that now,
33:29there's Bayesian approaches.
33:30I don't know if many of you are familiar
33:32with propensity score, I don't have time to go into it now,
33:35but propensity score is basically an approach for trying
33:38to connect historical data to your clinical trial data
33:43and maybe match patients up in ways
33:46that are similar as possible.
33:48Right, you often know a lot of things the baselines
33:51that are prognostic for the patient, right?
33:54So you try to make it where you're as close
33:57to an apples to apples comparison as possible.
34:00There's a lot of details about exactly how you do that
34:03that I think people can still figure out better
34:06and learn more.
34:08A lot of work with adaptive designs.
34:09For example, you might combine a phase two dose selection
34:13with the phase three efficacy part.
34:15So there's a lot of people looking at that
34:19because you can gain a lot of efficiency by not having to do
34:23a separate phase two study and sort of start all over
34:27with a separate phase three study, right?
34:31My opinion, adaptive designs is that
34:36if you sort of know what you need to do
34:38that is you know your population,
34:40you know what you wanna measure in those people,
34:42you have a decent idea of what your treatment effect may be,
34:46you know, then just do the phase three study
34:47you think you oughta to do, right?
34:49If you're kind of at the other extreme,
34:50you really don't know the answer to much
34:52of any of that stuff,
34:53then you should probably do two separate studies, right?
34:56Just do the phase two study that's not pivotal.
34:59Learn what the heck is going on
35:01and then do the phase three study.
35:03If you're in the middle, which is you kinda mostly know
35:06what you're doing,
35:07but there's this one nagging question,
35:08I don't know if I wanna do the high dose or the low dose,
35:11or I don't know whether the patients need to be, you know,
35:14have this biomarker or maybe a, you know,
35:17I can do it on everybody, you know?
35:19What population?
35:20You have that one nagging question,
35:21that's where an adaptive design can often be helpful, right?
35:25That way, you can build a design around getting information
35:30about that key piece
35:32and going straight into phase three.
35:36A couple things I think
35:37are maybe a little bit under-researched,
35:39could be looked at more.
35:41I think a single-arm design that can change
35:44to a randomized design, stage two,
35:46is something I would like to see a better treatment of
35:50because what I was talking about before
35:52with the real-world data,
35:53you're trying to compare it, right?
35:55That works best in the extreme cases, right?
35:58So if the real-world data say this is what happens
36:02to an untreated patient, right?
36:04You tend to see this sort of result.
36:06If you do a single-arm study in your experimental therapy
36:10and it looks the same, then you have a good answer.
36:13The answer is your drug's not that good
36:15and, you know, and you've done it efficiently, right?
36:18Single-arm study is smaller, right?
36:19If the results are great, much better,
36:23then you've also have a good answer, right?
36:25Even if there's some bias in the real-world data,
36:27the results are so big,
36:30it's gotta be something good with the drug
36:32going on there, right?
36:33It's that middle case that's kind of awkward, right?
36:36Well, it's better, but it's maybe even p is less than 0.05,
36:40but there might be bias in that historical data
36:42and dang, I wish I'd done a randomized study
36:45sometimes what you might think, right?
36:47So then I think it'd be interesting,
36:49you do state choose the randomized study,
36:51you combine the two phases, right?
36:53And then you come up with one result for the whole study.
36:57And lastly, I'll mention,
36:59I think there's more actually to do
37:00with good old stratification.
37:04We've had a couple situations where we were unsure
37:07how to stratify in a study.
37:09We actually had a group go back, look at the literature,
37:12the literature actually a little bit more thin,
37:16vague and conservative than I thought it was.
37:19If you really want to understand, hey, from my study,
37:22I've got 150 patients, these are the factors.
37:26It not actually specific as you might think.
37:29And you can get into things like whether
37:31the stratification factors are correlated
37:33with each other, right?
37:36And continuous factors you might wanna stratify on
37:39is another kinda area people could go.
37:41So I think there's still more to do there.
37:44I say it's important for small studies, right?
37:46So if you're doing a big study,
37:48the law of large numbers is gonna probably cover,
37:50you could probably stratify nothing
37:52and it'll be probably okay, right?
37:54But studies are getting smaller and smaller,
37:56people are in more and more focused groups.
37:59A small study,
38:01if I can say something a little bit controversial,
38:04small randomized studies I think are a bit dangerous, right?
38:08People love this notion that a randomized study's unbiased,
38:11but that's in the long term.
38:15I only get one chance to do my study.
38:17There's only 30 or 40 patients in it
38:20that might not be big enough to guarantee
38:21that everything's gonna work out even.
38:23So that could be a little bit dangerous.
38:25If you're gonna do it,
38:26you might wanna think about stratification carefully.
38:28Probably already talked to you.
38:30I wanted to leave at least eight minutes.
38:31<v ->Okay, you've got plenty of time,</v>
38:33you've got like 10 minutes.
38:34<v ->I think I was told like or by 12:50 or whatever.</v>
38:38<v ->Yeah, we have to be done by 12:50, yeah.</v>
38:40By 12:50. <v ->Right, so.</v>
38:42<v ->Question.</v> <v ->12:40, so we got like 10.</v>
38:45<v ->Anyone in the room or on.</v>
38:46<v ->Yes.</v>
38:48<v ->So, okay, I feel like drug development,</v>
38:49and in particular FDA, are pretty conservative
38:52with how they like designed their trials,
38:53especially with like phase two and phase three trials.
38:55So again,
38:56obviously, you've talking about like some of these
38:58more interesting like, you know, ideas like adaptive trials.
39:01And let's say like you're in a company that like has,
39:03I'm not sure Vertex has done a kind of adaptive trial,
39:07not that I'm aware of.
39:08But like if let's say
39:10you thought it's a good idea for a certain drug,
39:11for a certain program, like how would you go about
39:14like making the case that an adaptive trial is better?
39:17Like obviously, like this is assuming
39:19you have like a theory behind it
39:20that it is, for some reason, better.
39:23<v ->Yeah, that's a very good question.</v>
39:27We do have an adaptive study actually,
39:29the one like I had mentioned there
39:30with two different doses, do a phase two,
39:33and then we're gonna pick a dose and dose into phase three.
39:37There's a series of meetings.
39:39I didn't have time to talk about it,
39:42but there's like type A, type B, type C meetings
39:45you have with FDA along the way.
39:47There's another type of meeting,
39:48one of them is called the end of phase two meeting.
39:51So you do have meetings at FDA
39:52where you can propose things and say,
39:54"Hey, we think we oughta do it this way."
39:58As you may have briefly seen on the slide
40:01about rare diseases,
40:03the regulatory framework on rare diseases is less certain,
40:08which is both good and bad.
40:09I mean, right, it can be bad in the sense
40:13that you're not really sure what you're allowed to do.
40:16But it's also good in the sense
40:17that it's more possible for you to argue things like,
40:20"Hey, there's not that many, you know,
40:24say kids with Duchenne muscular dystrophy, you know?"
40:27It's not that big a population.
40:28These kids have a serious disease.
40:31We need some flexibility in our design
40:33to show that our drug is working, you know?
40:35So it's a little bit easier in rare diseases.
40:39So you could either use
40:40those type A, B, C meetings with them
40:42and, of course, you're gonna send them
40:44your protocol and stuff
40:46to sort of make your case in a meeting.
40:48They also have a program called
40:50the Complex Innovative Design Program,
40:53which is actually run by their stats people
40:56where you can set up extra meetings
40:59to review things like simulations, right?
41:02So their biggest concern is maintaining type one error,
41:07right? <v ->So I mean like,</v>
41:08so I worked in drug development for the past six years
41:10and like interacting with FDA and like FDA minutes and such,
41:14like I've seen like them like say one thing
41:17and then like the next meeting say,
41:18"Actually, we change our minds."
41:20Or they give like vague answers.
41:22And so you like internally have to kinda figure out
41:24like what you're gonna do.
41:25So like in those situations,
41:26like where okay, like FDA like might be okay,
41:28we're not actually sure, like I guess
41:30like how do you build like the,
41:32and then obviously, the tendency then
41:33is to like just go back into just do
41:36like just what you traditionally done,
41:37but like if you like are really advocating
41:38for something like this.
41:40<v ->Yeah, there's a balance there.</v>
41:41It's not uncommon
41:42to be like not completely sure what FDA does.
41:45I mean if you schedule one of these meetings with 'em,
41:48yeah, they will give you a response.
41:50It might be in person, it might be written,
41:53it might not be everything you would want to see.
41:56You might still have questions after seeing it.
41:58So it depends.
41:59Sometimes they're pretty clear,
42:01no, we don't like this or whatever.
42:03Other times, you're kinda still
42:04kinda scratching your head a bit.
42:07A lot of times, they say something is a review issue,
42:10which means, well, you know,
42:12if you get the data, we'll look at it
42:14and see then, you know?
42:17So that's kinda the best you can do.
42:19It's difficult to get certainty.
42:21There's definitely a lot of planning
42:23around communication with FDA.
42:26What do we wanna say?
42:27I think of it a little bit
42:28as kinda like going to the oracle in ancient Greece, right?
42:32It's sort of like, you know,
42:34you have to plan and hope that, you know,
42:36they're gonna tell you.
42:38You can interpret what sort of prophetic thing
42:41they're going to tell you.
42:44Sorry, I don't have a better answer for you than that.
42:47Oh, but what I was saying earlier was there is something
42:49called the Complex Innovative Design Program
42:52where you can set up,
42:55if they accept you, you get like two extra meetings
42:58where you can review things like simulations.
43:00So if you wanna do something complicated, they'll often say,
43:03"Well, we wanna make sure type one error is controlled."
43:07And if the answer to that question is,
43:09"Well, we got a bunch of simulations to show you
43:12that it controls type one error,"
43:14then you might wanna do something like that
43:16to kinda dig through the details of,
43:19well, how did you set up your simulations and all that.
43:23Other questions?
43:27I feel like I've been ignoring everybody over here.
43:30<v ->Got a question over here.</v>
43:31<v ->Oh, yes.</v>
43:32<v Student>Thank you for the presentation.</v>
43:35The question is,
43:36is it possible to revise your SAP
43:38after the trial started?
43:40If the answer is yes, is there any restriction on it?
43:46<v ->So again, back to the blinded versus unblinded, right?</v>
43:50If it's an unblinded study, you can,
43:54but it's gonna be viewed suspiciously,
43:57for lack of a better word, right?
43:58It's gonna be viewed as a post hoc change.
44:01Why are you changing this?
44:03You suspected that something,
44:05if it's a blinded study, yes, you can.
44:08You can amend your SAP.
44:11That's not terribly uncommon.
44:13For example, you might,
44:14during the course of the study, still blinded,
44:17you might learn new information,
44:19new published data may come out.
44:21You might learn something about the baseline data
44:24on your study, you know, the distribution
44:26or something like that.
44:28So as a result, you may wanna pivot what your SAP is
44:32and if it's still blinded,
44:34generally speaking, you could still do that
44:37and it'd be used pre-specified.
44:39<v Student>Thank you.</v>
44:42<v ->Yes.</v>
44:44<v Learner>I'm very sure that there should be</v>
44:46many variables to consider when it comes to this study.
44:48And in case of these small sample size studies,
44:52I'm pretty sure that a stratification
44:54might really be inefficient
44:57to contain all these variables at one place.
45:01And I'm very curious,
45:02how do you actually like manage when it comes
45:05to the small sample size
45:06studies? <v ->Yeah.</v>
45:07Yeah, also good question.
45:08Again, I think this is a good area for more research.
45:11We had a group look at some simulations.
45:15Here's my qualitative assessment of what we found.
45:18One, I think in general, people worry a bit too much
45:21about what you're saying.
45:23As long as like the marginals work out pretty well,
45:27then you're actually probably still okay
45:30as far as stratification goes.
45:34I think there's a bigger danger of bad luck imbalance.
45:39I don't wanna speculate too much,
45:41but there was a competitor that had a study come out,
45:44rare disease, small study,
45:46just by bad luck, they had some imbalance
45:49in one other strata.
45:50And maybe it could be the reason why the study,
45:55statistically speaking, failed.
45:59And so, yeah, here's my sports analogy, okay?
46:04So small studies are kind of like a football game
46:08where you're losing at the end of the game.
46:11You wanna throw the ball 'cause you need to score, right?
46:14The defense is going to be playing for that.
46:17They're gonna make it harder for you to do that,
46:18but you need to do it anyhow, right?
46:21That's kinda like the way stratification is.
46:22Yes, it's harder to do it in a small study,
46:25but you need to think about it and try to do it anyhow.
46:28'Cause if you just throw your hands up and say,
46:29"Eh, whatever," then you might have what happened to you,
46:33what happened to this competitor.
46:36And so we actually wrote a program so you could simulate
46:39and say, "Hey, from my study, I've got X patients,
46:42these are the stratification factors.
46:45What's gonna happen to my type one and type two error?"
46:49But you are right that in principle,
46:51you can't overdo it.
46:53I just think the point where you overdo it
46:54is further out than most people think.
47:02<v ->Two more minutes.</v>
47:04Any other questions?
47:08Or online?
47:11<v ->Sorry if I've ignored people online.</v>
47:13<v ->We have-</v>
47:14<v Student>I have a question.</v>
47:15<v ->I don't know how many people we have online.</v>
47:17<v ->Let me just move to see if there's the chat.</v>
47:19<v ->Do I?</v>
47:21<v ->To pop up.</v>
47:22<v ->Oh, it would pop up? Okay.</v>
47:25That's doesn't look like we have any chat.
47:27<v Student>Can I ask a question?</v>
47:29So you mentioned time management
47:31as an important skill obviously.
47:35Can you tell us about sort of what is
47:39the work cycle of a biostatistician?
47:42So are they working on many studies at one time?
47:46Are they getting a lot of experience doing phase one
47:50or what's the volume of which they're working on and how?
47:56<v ->Yeah, it's, as you expect it, you know, it depends.</v>
47:59I mean what sort of study someone has assigned to you
48:04is a little bit random.
48:05I mean what they need somebody to do.
48:08It's not uncommon for people to be assigned to say
48:11two to five studies depending on how big they are,
48:16how short you are on people, et cetera, you know?
48:20And so you have to try and manage that kind of work.
48:23I was just talking about across those, you know,
48:25say two to five studies.
48:28You also spend, I'd say roughly 10 to 20% of your time
48:34doing non-project stuff.
48:37Things I mentioned like the working groups,
48:40maybe some independent research,
48:42maybe other kinda service to the department.
48:44I mean, you know, obviously,
48:45I spend time interviewing people, stuff like that.
48:49So that's kind of the breakdown of what people are doing.
48:53<v Student>And are they working in teams</v>
48:55as statisticians or?
48:56<v ->Yeah, so you would have, you know, again,</v>
48:59you have a project level, right?
49:01So you would have a project statistician,
49:02somebody who's somewhat more senior,
49:06who manages the whole project.
49:08And then under that person, you might have whatever,
49:11you know, two, three, four,
49:12depends how big the project is,
49:14statisticians who manage individual studies, right?
49:17So you might have, you know, I don't know,
49:1910 studies in the project, right?
49:21And you might have three statisticians
49:24who each have three each or something like that
49:27reporting to that project statistician
49:29who's kinda doing the overall work on the drug.
49:36<v ->All right.</v>
49:37So thanks so much.
49:41In the interest of time,
49:42we're going to go ahead and stop here.
49:44But let's thank our speaker again.
49:50<v ->Great insight into the industry</v>
49:54and have a wonderful day.
49:57<v ->Sign in sheet.</v>
49:57<v ->Oh yeah.</v>
49:58We have a sign in sheet.
50:00(attendants chattering indistinctly)
50:02Thank you.
50:03(attendants chattering indistinctly)
50:10So we got a couple of 'em up here.
50:15You still need to sign in, please do.
50:17<v ->The thing is that</v>
50:18(student muttering indistinctly)
50:19well, technically, have like four, five.
50:22(students chattering indistinctly)