Impact of COVID-19 Pandemic on Oncology Clinical Trials - Simulation Results for Time Off-Treatment (Section 3.4) and Missed Visits (Section 3.3) on Efficacy Assessment

Jiabu Ye, Binbing Yu, Zsolt Szijgyarto and Helen Mann
for the COVID-19 Oncology Biometrics Working Group
AstraZeneca Oncology Biometrics

July 10, 2021

Simulated Clinical trial and Questions

A Phase III randomized controlled trial of sample size n with r:1 randomization ratio. Primary endpoint is time to event (OS or PFS). Assume the median survival time is m years for the control with an hypothesised treatment (statistically measured by a hazard ratio and associated p value).
Design characteristics considered in the simulation

Analysis	Endpoint	SOC.median	Target.HR	Analysis.Stage	HR.cv	Number.events	Number.of.patients	Followup.Time
1	PFS	26	0.69	Interim	0.736	233	530	45
2	PFS	26	0.69	Final	0.796	311	530	59

Question. Effect of missing visits during pandemic on interim and final analyses
- Section 3.3: Effect of missing-2 visit rule if the missing probability during pandemic is higher. If we apply the missing 2 visits censoring rule on interuppted tumour assessments, how will the power and hypothesised HR change for PFS/EFS/DFS.
- Section 3.2: The time off-treatment/missing on-study procedure may be associated with OS/PFS. We set the multiplicative factor of HR due to COVID-19 is 1.0 or 1.1 for the control group and 1.4 (1/0.72) for the treatment group. This creates differential COVID-19 impact on the treatment and control groups.

Simulation for Section 3.3. Impact of missing visits on HR, Type 1 error and Power

The missingness indicator for a clinical visit follows a Bernoulli distribution during the COVID-19 pandemic (a,b)

Missing type: 1=missing by patient, 2=missing by visit
Probability of missing: 0.25 (moderate), 0.5 (high)
Time that pandemic occurs before DOC: 6 and 24 months before the FA PFS time (59m)
Pandemic duration: 3, 6 and 9 months

Supplementary Table 5. Impact of 2-missed visits censoring rule on bias of HR estimates and power/Type 1 error of the interim PFS (patient-level missing)

The COVID start time (months after FPR) = total study time (59m) - Time from COVID to DCO (data cutoff)
Columns under H1: HR=0.69 are the power reported in Table 5.
Columns under H0: HR=1.0 are not reported

Design characteristics
Endpoint	SOC.median	Target.HR	Analysis.Stage	HR.cv	Number.events	Number.of.patients	Followup.Time
PFS	26	0.69	Interim	0.736	233	530	45

			H1: HR=0.69					H0: HR=1.0
Use missing-2-visit rule?				Yes		No			Yes		No
Time COVID to DCO	Duration	P(Miss)	P(2 missing)	% Bias	Power	%Bias	Power	P(2 missing)	% Bias	Alpha	%Bias	Alpha
0	0	0.00	0.00	0.76	67.06	0.76	67.06	0.00	0.76	1.98	0.76	1.98
24	3	0.25	0.01	0.76	66.46	0.76	67.08	0.01	0.78	2.06	0.76	1.96
24	3	0.50	0.03	0.76	66.34	0.76	67.06	0.02	0.78	2.10	0.76	1.96
24	6	0.25	0.16	0.67	64.88	0.77	66.94	0.14	0.81	1.88	0.76	1.96
24	6	0.50	0.31	0.54	62.86	0.78	66.98	0.29	0.88	1.94	0.75	2.08
24	9	0.25	0.16	0.81	64.50	0.77	67.00	0.15	0.80	2.02	0.77	1.98
24	9	0.50	0.32	0.78	61.68	0.78	67.04	0.30	0.85	2.04	0.76	2.02

Supplementary Tabels 6 and 7. Impact of 2-missed visits censoring rule on bias of HR estimates and power/Type 1 error of the final PFS (patient-level missing)

-The COVID start time (months after FPR) = total study time (59m) - Time from COVID to DCO (data cutoff)

Columns under H1: HR=0.69 are the % bias and power reported in Table 6.
Columns under H0: HR=1.0 are the % bias and Type 1 error reported in Table 7

Design characteristics
Endpoint	SOC.median	Target.HR	Analysis.Stage	HR.cv	Number.events	Number.of.patients	Followup.Time
PFS	26	0.69	Final	0.796	311	530	59

			H1: HR=0.69					H0: HR=1.0
Use missing-2-visit rule?				Yes		No			Yes		No
Time COVID to DCO	Duration	P(Miss)	P(2 missing)	% Bias	Power	%Bias	Power	P(2 missing)	% Bias	Alpha	%Bias	Alpha
0	0	0.00	0.00	0.50	90.30	0.50	90.30	0.00	0.50	5.10	0.50	5.10
24	3	0.25	0.01	0.50	89.86	0.51	90.30	0.01	0.51	5.08	0.50	5.08
24	3	0.50	0.03	0.48	89.82	0.52	90.16	0.02	0.52	5.10	0.49	5.02
24	6	0.25	0.16	0.39	87.18	0.60	90.14	0.14	0.53	5.18	0.51	5.00
24	6	0.50	0.31	0.23	83.68	0.71	89.60	0.29	0.64	5.34	0.49	5.38
24	9	0.25	0.16	0.23	86.82	0.62	90.08	0.15	0.53	5.30	0.50	5.28
24	9	0.50	0.32	0.01	83.94	0.72	89.96	0.30	0.66	5.24	0.48	4.92
6	3	0.25	0.00	0.50	90.30	0.50	90.30	0.00	0.50	5.10	0.50	5.10
6	3	0.50	0.00	0.50	90.30	0.50	90.30	0.00	0.50	5.10	0.50	5.10
6	6	0.25	0.05	0.56	90.04	0.50	90.26	0.05	0.50	5.16	0.50	5.10
6	6	0.50	0.11	0.62	89.96	0.50	90.28	0.09	0.51	5.50	0.50	5.06

Figure 2A. Plot of # events, # of having 2 missing and # events being impacted by missing 2 visit rule by total study duration (COVID-19 pandemics starts 24 months before the target DCO (59m) and has duration of 9 months, patient-leve missing probably 50% during pandemic)

Figure 2B. Plot of patients being impacted: a sensitivity analysis by changing patient-level missing probability from 0.5 to 0.25

Figure 2C. Plot of patients being impacted: a sensitivity analysis by changing Pandemic duration 9m to 3m

Figure 2D. Plot of patients being impacted: a sensitivity analysis by changing patient-level missing to visit-level missing

Simulation for Section 3.2. Impact of excess/reduced hazard due to time off-treatment during COVID-19 on HR and power

Both treatment and control groups have the same excess/reduced hazard during COVID19 pandemic
Treatment group has more excess hazard than the control during COVID19, i.e., a mulitiplicative factor of HR due to pandemic is 1.0 and 1.1 among the control and a multiplicative factor of HR is 1.1 or 1/0.72 in the treatment group, respectively. For example, when the multiplicative factor is 1.0 in the control and 1/0.72 in the treatment group, this means the patients in the treatment group has the same mortality rate as those in the control group during the COVID-19 period.
We assume that 10% or 30% of the patients in the control and treatment groups will be impacted by COVID-19
Measure of impact (see results in the tables hereafter):
- Delta: Number of additional number of events in the treatment group vs. control due to COVID-19 if 100% patients are impacted.
- Bias of the HR estimate
- Power/Type 1 error of the log-rank test

Supplementary Table 1. Impact of time off-treatment on PFS HR estimates and Type 1 error under the null hypothesis for PFS

In Table 1, The COVID start time (months after FPR) = total study time (59m) - Time from COVID to DCO (data cutoff)

Design characteristics
Endpoint	SOC.median	Target.HR	Analysis.Stage	HR.cv	Number.events	Number.of.patients	Followup.Time
PFS	26	1	Final	0.796	311	530	59

		Excess HR of patients during pandemic among control and treatment groups (HR0,HR1)
		10% Impacted						30% Impacted
		(1.0,1.4)			(1.1,1.4)			(1.0,1.4)			(1.1,1.4)
Time COVID-DCO	COVID duration	Delta	% Bias	Alpha	Delta	% Bias	Alpha	Delta	% Bias	Alpha	Delta	% Bias	Alpha
0	0	0.0	0.5	5.1	0.0	0.5	5.1	0.0	0.5	5.1	0.0	0.5	5.1
6	3	2.9	0.9	5.3	2.1	1.0	4.8	2.9	1.3	5.1	2.1	0.8	4.4
6	6	5.6	1.0	5.1	4.2	0.5	4.8	5.6	1.5	5.1	4.2	1.4	5.5
24	3	2.9	0.9	4.8	2.1	0.8	4.8	2.9	1.8	5.3	2.1	0.9	5.4
24	6	5.6	1.3	5.2	4.2	1.2	5.2	5.6	2.1	5.1	4.2	1.7	5.0
24	9	8.3	1.6	5.2	6.1	1.3	5.2	8.3	3.1	6.0	6.1	2.6	4.9

Supplementary Table 2. Impact of time off-treatment on PFS HR estimates and power under the alternative hypothesis for PFS .

In Table 2, The COVID start time (months after FPR) = total study time (59m) - Time from COVID to DCO (data cutoff)

Design characteristics
Endpoint	SOC.median	Target.HR	Analysis.Stage	HR.cv	Number.events	Number.of.patients	Followup.Time
PFS	26	0.69	Final	0.796	311	530	59

		Excess HR of patients during pandemic among control and treatment groups (HR0,HR1)
		10% Impacted						30% Impacted
		(1.0,1.4)			(1.1,1.4)			(1.0,1.4)			(1.1,1.4)
Time COVID-DCO	COVID duration	Delta	% Bias	Power	Delta	% Bias	Power	Delta	% Bias	Power	Delta	% Bias	Power
0	0	0.0	0.5	90.5	0.0	0.5	90.5	0.0	0.5	90.5	0.0	0.5	90.5
6	3	2.7	0.9	89.7	2.0	0.7	89.6	2.7	1.1	89.4	2.0	1.2	89.3
6	6	5.4	1.1	89.2	3.9	0.6	90.2	5.4	1.8	87.9	3.9	1.6	88.9
24	3	2.7	0.9	89.4	2.0	1.1	89.0	2.7	1.4	89.4	2.0	1.2	89.1
24	6	5.4	1.2	89.0	3.9	0.8	90.3	5.4	2.5	87.1	3.9	1.9	88.2
24	9	8.0	1.6	88.9	5.8	1.2	89.4	8.0	3.2	85.7	5.8	2.5	87.7

Supplementary Table 3. Impact of time off-treatment on OS HR estimates and Type 1 error under the null hypothesis for OS

In Table 3, The COVID start time (months after FPR) = total study time (74m) - Time from COVID to DCO (data cutoff)

Design characteristics
Endpoint	SOC.median	Target.HR	Analysis.Stage	HR.cv	Number.events	Number.of.patients	Followup.Time
OS	38.6	1	Final	0.791	300	530	74

		Excess HR of patients during pandemic among control and treatment groups (HR0,HR1)
		10% Impacted						30% Impacted
		(1.0,1.4)			(1.1,1.4)			(1.0,1.4)			(1.1,1.4)
Time COVID-DCO	COVID duration	Delta	% Bias	Alpha	Delta	% Bias	Alpha	Delta	% Bias	Alpha	Delta	% Bias	Alpha
0	0	0.0	0.2	4.8	0.0	0.2	4.8	0.0	0.2	4.8	0.0	0.2	4.8
6	3	0.0	0.3	5.4	0.0	0.6	5.2	0.0	0.6	4.7	0.0	0.5	4.6
6	6	0.0	0.5	5.1	0.0	0.4	4.6	0.0	0.7	4.7	0.0	0.5	5.5
21	3	2.1	1.1	5.3	1.5	0.7	5.2	2.1	1.2	5.0	1.5	1.0	4.8
21	6	4.1	0.9	5.2	3.0	0.7	4.6	4.1	1.6	5.1	3.0	1.4	5.4
39	3	2.1	1.1	5.2	1.5	0.7	4.5	2.1	1.4	5.3	1.5	1.2	5.4
39	6	4.1	1.0	5.0	3.0	0.9	5.1	4.1	2.3	4.5	3.0	1.7	5.0
39	9	6.1	1.1	4.6	4.5	1.1	4.8	6.1	2.8	5.7	4.5	2.4	5.0

Supplementary Table 4. Impact of time off-treatment on OS HR estimates and power under the alternative hypothesis for OS .

In Table 4, The COVID start time (months after FPR) = total study time (74m) - Time from COVID to DCO (data cutoff)

Design characteristics
Endpoint	SOC.median	Target.HR	Analysis.Stage	HR.cv	Number.events	Number.of.patients	Followup.Time
OS	38.6	0.72	Final	0.791	300	530	74

		Excess HR of patients during pandemic among control and treatment groups (HR0,HR1)
		10% Impacted						30% Impacted
		(1.0,1.4)			(1.1,1.4)			(1.0,1.4)			(1.1,1.4)
Time COVID-DCO	COVID duration	Delta	% Bias	Power	Delta	% Bias	Power	Delta	% Bias	Power	Delta	% Bias	Power
0	0	0.0	0.5	80.9	0.0	0.5	80.9	0.0	0.5	80.9	0.0	0.5	80.9
6	3	0.0	0.5	81.4	0.0	0.7	80.5	0.0	0.5	80.8	0.0	0.4	81.1
6	6	0.0	0.5	80.5	0.0	0.5	81.7	0.0	0.6	81.3	0.0	0.9	80.0
21	3	2.0	0.9	80.7	1.4	0.9	80.7	2.0	1.0	80.6	1.4	1.0	79.8
21	6	3.9	1.0	80.5	2.8	1.1	79.8	3.9	1.7	78.3	2.8	1.4	79.7
39	3	2.0	1.0	79.5	1.4	0.9	79.7	2.0	1.8	77.8	1.4	1.0	80.1
39	6	3.9	1.2	79.2	2.8	1.2	79.4	3.9	1.9	78.1	2.8	1.8	78.0
39	9	5.8	1.1	80.4	4.2	1.1	79.9	5.8	2.6	76.1	4.2	2.1	78.2