Model setup

In the absence of a control arm, we observe seizure counts repeatedly after treatment begins. A longitudinal NB model can be written as:

where:

• : seizure count for patient at time
  
• : effect of follow-up time
  
• : baseline seizure frequency
  
• : coefficient for baseline
  
• : subject-specific random effect
  
• : offset accounting for observation duration (applicable when follow-up duration differs across subjects)

Here, time captures how seizure rates change after treatment initiation, while baseline explains between-patient differences.

How this is different from PCFB

PCFB asks:

How much did the outcome change from baseline to a single follow-up time?

A longitudinal NB model asks:

What is the trajectory of seizure rates over time after treatment starts?

Key differences:

• PCFB discards the original count data → NB uses count data from all visits

• PCFB embeds baseline in the outcome, introducing extra noise and variability → NB allows baseline to be modelled separately as a predictor

• PCFB distorts scale because it’s asymmetric and nonlinear → NB models counts on a natural rate scale

• trials reporting on PCFB summaries typically ignore within-subject correlation by collapsing repeated measures into a single summary → NB explicitly models repeated measures

Baseline: covariate vs reference point

A common practice in single-arm analyses is to:

compare each post-baseline visit to baseline, rather than adjusting for baseline as a covariate

While intuitive, this approach has a key flaw.

Baseline seizure frequency is highly prognostic of future counts. It is not just a reference point—it is a major source of variability across patients.

When you anchor everything to baseline:

• baseline becomes part of the outcome (as in PCFB)

• its prognostic information is no longer used to explain variability

• additional noise is introduced because the outcome depends on both baseline and post-baseline measurements

• and time effects can become confounded with baseline-dependent dynamics (including regression to the mean)

In contrast, including baseline as a covariate:

• separates between-patient heterogeneity from within-patient change over time

• improves efficiency

• and yields a cleaner interpretation of the post-treatment trajectory

In single-arm settings, apparent improvements measured as changes from baseline (including PCFB and responder rates derived from PCFB) do not identify a treatment effect. This is because the outcome is defined relative to baseline, so any observed changes, whether due to natural fluctuations, regression to the mean, or other time effects, are carried directly into the endpoint. Without a control group, these changes are indistinguishable from what would have happened in the absence of treatment. As a result, descriptive summaries such as PCFB or responder rates should not be interpreted as evidence of efficacy.

A longitudinal negative binomial model provides a more principled way to evaluate post-baseline trajectory by accounting for baseline differences and within-subject variability. Importantly, baseline is treated as a covariate rather than embedded in the outcome, so the model does not rely on baseline as a reference point for comparison.

Interpreting the time effect in a negative binomial model

If time is modelled linearly:

then:

• is the rate ratio per visit
  
• e.g., implies a 10% reduction in seizure rate per visit

This provides a simple summary of longitudinal change but it assumes the effect is constant over time.

Allowing the time effect to vary

In practice, treatment-associated trajectories are rarely perfectly linear. You may see:

• rapid early improvement

• delayed onset

• plateauing

• or rebound

These patterns can be explored by modelling time more flexibly.

count ~ factor(visit) + log_baseline + (1 | id)

count ~ factor(visit) + log_baseline + (1 | id)

count ~ visit + I(visit^2) + log_baseline + (1 | id)

count ~ visit + I(visit^2) + log_baseline + (1 | id)

library(splines)

count ~ ns(visit, df = 3) + log_baseline + (1 | id)

count ~ ns(visit, df = 3) + log_baseline + (1 | id)

Interpreting flexible time models

Once time is modelled flexibly via splines or higher-order polynomials, coefficients are no longer directly interpretable.

Instead, interpretation should focus on:

A simulated study for illustration

A single-arm longitudinal seizure study was simulated with repeated counts over six post-baseline visits, incorporating a baseline covariate, subject-level variability, and a nonlinear time effect reflecting delayed and evolving improvement. Counts were generated from a negative binomial distribution with varying observation durations. A linear time model was compared with a flexible spline model to illustrate how different assumptions about time affect the estimated trajectory. Details of the simulation are provided in the annotated code.

# ============================================================
# Simulate one single-arm longitudinal seizure study
# with NONLINEAR (visit-specific) time effects
#
# Goal:
# - illustrate a setting where the time effect is NOT constant
# - demonstrate when flexible time modelling (e.g., natural splines) is useful
# ============================================================

# -----------------------------
# 1. Load packages
# -----------------------------
library(glmmTMB)   # negative binomial mixed models
library(dplyr)     # data manipulation
library(ggplot2)   # plotting
library(splines)   # natural spline basis for time

# Set seed for reproducibility
set.seed(1234)

# -----------------------------
# 2. Define study design
# -----------------------------
n_patients <- 80                 # number of subjects
n_visits <- 6                    # visits: 1, 2, 3, 4, 5, 6
visit_times <- 1:n_visits        # post-baseline visits only

# Note:
# Here I simulate only post-baseline repeated counts.
# Baseline seizure count is included as a subject-level covariate, not as one of the longitudinal count outcomes.

# -----------------------------
# 3. Define true data-generating parameters
# -----------------------------

# Baseline mean seizure level (on log scale)
beta_0 <- log(6)

# Effect of baseline seizure count (log scale)
# We use log(28-day seizure count at baseline + 1) as the baseline covariate  
# A value of 1 is added to avoid log(0)
# In practice, this is often not an issue if the study has a minimum entry criterion (e.g., ≥5 seizures during the baseline period).
beta_baseline <- 0.35

# Subject-specific heterogeneity (random intercept SD - i.e., between-subject variability)
sd_subject <- 0.4

# Negative binomial size parameter
# Smaller values = more overdispersion
theta_nb <- 4

# -----------------------------
# 4. Define NONLINEAR time effects
# -----------------------------
# These are visit-specific log-rate effects (relative to visit 1)

true_time_effect <- c(
  "1" = 0,                # visit 1: no effect (reference)
  "2" = log(0.95),        # 5% reduction relative to visit 1
  "3" = log(0.90),        # 10% reduction relative to visit 1
  "4" = log(0.75),        # 25% reduction relative to visit 1
  "5" = log(0.75),        # plateau - maintaining a 12% reduction relative to visit 1
  "6" = log(0.6)          # 40% reduction relative to visit 1
)

# Interpretation:
# - delayed onset of seizure reduction
# - increasing seizure reduction over time
# - partial plateau

# -----------------------------
# 5. Simulate subject-level baseline information
# -----------------------------
subject_df <- data.frame(
  id = 1:n_patients,

  # Simulate baseline 28-day seizure count for each patient
  # This is a covariate, not part of the repeated outcome series
  # Mu set to 8 as a chosen average baseline seizure rate that approximately gives about 8 seizures per 28 days
  # Size parameter set to 2.5 to allow for more baseline variability 
  # Note: if baseline is collected over a longer period (e.g., 84 days),
  # a 28-day seizure frequency is often derived as:
  #   28 * (total seizures in period / number of evaluable days in period)
  baseline28 = rnbinom(n_patients, mu = 8, size = 2.5),

  # Random intercept to induce within-subject correlation
  b_i = rnorm(n_patients, mean = 0, sd = sd_subject)
) %>%
  mutate(
    # Log-transform baseline count for use as a covariate
    # +1 avoids taking log(0)
    # In practice, this is often not an issue if the study has a minimum entry criterion (e.g., ≥5 seizures during the baseline period).
    log_baseline = log(baseline28 + 1)
  )

# -----------------------------
# 6. Expand to one row per patient per visit (i.e., longitudinal data)
# -----------------------------
dat <- expand.grid(
  id = 1:n_patients,
  visit = visit_times
) %>%
  arrange(id, visit) %>%
  left_join(subject_df, by = "id")

# -----------------------------
# 7. Simulate observation duration
# -----------------------------
# Allow varying observation periods to justify offset

dat <- dat %>%
  mutate(
    exposure_days = sample(c(21, 28, 35), size = n(), replace = TRUE,
                           prob = c(0.15, 0.70, 0.15)),
    # Offset adjusts for differing observation durations.
    # Dividing by 28 expresses the model on a 28-day rate scale,
    # which aligns with typical reporting in epilepsy trials.
    # This does not affect exponentiated covariate effects (rate ratios), only the intercept.
    offset_log = log(exposure_days / 28)
  )

# -----------------------------
# 8. Generate mean seizure counts
# -----------------------------
# True linear predictor:
# log(E[count]) =
#   intercept
# + time effect
# + baseline prognostic effect
# + subject random intercept
# + offset for observation duration

dat <- dat %>%
  mutate(
    eta = beta_0 +
      true_time_effect[as.character(visit)] + # nonlinear time effect
      beta_baseline * log_baseline +
      b_i +
      offset_log,

    mu = exp(eta) # expected seizure count; this can be non-integer.
                  # Observed counts are integers drawn from a negative binomial distribution centered at mu.
  )

# -----------------------------
# 9. Simulate observed seizure counts
# -----------------------------
dat <- dat %>%
  mutate(
    count = rnbinom(n(), mu = mu, size = theta_nb)
  )

# -----------------------------
# 10. Fit a linear-time NB mixed model (misspecified under this DGP)
# -----------------------------
# This assumes a constant log-rate change per visit.
fit_linear <- glmmTMB(
  count ~ visit + log_baseline + offset(offset_log) + (1 | id),
  family = nbinom2(),
  data = dat
)

summary(fit_linear)

# -----------------------------
# 11. Interpret the linear time effect
# -----------------------------
beta_hat <- fixef(fit_linear)$cond["visit"]
rr_per_visit <- exp(beta_hat)

cat("\nLinear time model:\n")
cat("Estimated log-rate ratio per visit:", round(beta_hat, 3), "\n")
cat("Estimated rate ratio per visit:", round(rr_per_visit, 3), "\n")
cat("Estimated percent change per visit:",
    round(-(1 - rr_per_visit) * 100, 1), "%\n")

# -----------------------------
# 12. Fit a spline-time NB mixed model
# -----------------------------
# This allows the time trend to be nonlinear and more flexible.
fit_spline <- glmmTMB(
  count ~ ns(visit, df = 4) + log_baseline + offset(offset_log) + (1 | id),
  family = nbinom2(),
  data = dat
)

summary(fit_spline)

# Important:
# The spline coefficients themselves are not directly interpretable.
# Interpretation should focus on predicted means / fitted trajectory
# or contrasts between selected post-baseline visits.

# -----------------------------
# 13. Obtain model-based predictions over time
# -----------------------------
# Predict at the average baseline level and a standard 28-day period
newdat <- data.frame(
  visit = seq(1, 6, by = 0.1),
  log_baseline = mean(dat$log_baseline),
  offset_log = 0,     # ie, exposure_days=28, corresponding to a standard 28-day period
  id = NA
)

newdat$pred_linear <- predict(
  fit_linear,
  newdata = newdat,
  type = "response",
  re.form = NA      # marginal prediction: no subject-specific random effect
)

newdat$pred_spline <- predict(
  fit_spline,
  newdata = newdat,
  type = "response",
  re.form = NA
)

# -----------------------------
# 14. Summarize observed mean counts by visit
# -----------------------------
obs_summary <- dat %>%
  group_by(visit) %>%
  summarise(
    mean_count = mean(count),
    median_count = median(count),
    .groups = "drop"
  )

print(obs_summary)

True time effects used in the simulation (relative to visit 1):

• visit 1: reference (no change)

• visit 2: 5% reduction

• visit 3: 10% reduction

• visit 4: 25% reduction

• visit 5: plateau (same as visit 4)

• visit 6: 40% reduction

Below is a snippet of the simulated longitudinal dataset for the first subject:

Each row corresponds to a visit for a given subject. The dataset includes visit, baseline seizure count (baseline28), subject-specific random effects (), the log-transformed baseline seizure count (log_baseline = ), observation duration (exposure_days), its log-transformed value used as an offset (offset_log), the linear predictor (, representing the log of the expected count), the expected count (), and the observed seizure count.

Interpretation of plot:

# -----------------------------
# 15. Plot observed means and fitted trajectories
# -----------------------------

# Model predictions in long format
plot_pred <- newdat %>%
  select(visit, pred_linear, pred_spline) %>%
  tidyr::pivot_longer(
    cols = c(pred_linear, pred_spline),
    names_to = "series",
    values_to = "value"
  ) %>%
  mutate(
    series = dplyr::recode(
      series,
      pred_linear = "Linear model",
      pred_spline = "Spline model"
    )
  )

# Observed means
plot_obs <- obs_summary %>%
  transmute(
    visit,
    value = mean_count,
    series = "Observed"
  )

# Combine all series
plot_all <- bind_rows(plot_obs, plot_pred)

plot_all$series <- factor(
  plot_all$series,
  levels = c("Observed", "Linear model", "Spline model")
)

ggplot(plot_all, aes(x = visit, y = value,
                     color = series,
                     shape = series,
                     linetype = series)) +
  geom_line(linewidth = 1) +
  geom_point(size = 2) +
  scale_color_manual(
    values = c(
      "Observed" = "black",
      "Linear model" = "blue",
      "Spline model" = "red"
    )
  ) +
  scale_shape_manual(
    values = c(
      "Observed" = 16,       # circle
      "Linear model" = 17,   # triangle
      "Spline model" = NA
    )
  ) +
  scale_linetype_manual(
    values = c(
      "Observed" = "solid",
      "Linear model" = "solid",
      "Spline model" = "solid"
    )
  ) +
  guides(
    shape = guide_legend(
      override.aes = list(shape = c(16, 17, NA))
    )
  ) +
  labs(color = "", shape = "", linetype = "") +
  theme_bw()

• black line with circles = observed mean count at each visit

• blue line with triangles = linear-time NB model

• red line = spline-time NB model

The model-based predictions are slightly lower than the observed means because they represent the expected outcome for a typical subject (with average covariates and no subject-specific random effect), whereas the observed means average over heterogeneous subjects. As a result, differences between model-based predictions and raw means are expected and do not indicate poor model fit; the key comparison here is the trajectory over time rather than matching the raw means exactly.

Interpreting time effects across models:

# -----------------------------
# 16. Example contrasts between post-baseline visits
# -----------------------------

# For the spline model, compare predicted rates at selected visits.
# These are model-based comparisons across time points,
# conditional on baseline covariate being held fixed.

# Create dataset for selected visits
contrast_dat <- data.frame(
  visit = c(1, 2, 3, 4, 5, 6),   # <-- include all visits if you want all contrasts
  log_baseline = mean(dat$log_baseline),
  offset_log = 0,
  id = NA
)

# Get linear predictors (log scale)
lp <- predict(
  fit_spline,
  newdata = contrast_dat,
  type = "link",
  re.form = NA
)

# Compute rate ratios
rr_2_vs_1 <- exp(lp[2] - lp[1])
rr_3_vs_1 <- exp(lp[3] - lp[1])
rr_4_vs_1 <- exp(lp[4] - lp[1])
rr_5_vs_1 <- exp(lp[5] - lp[1])
rr_6_vs_1 <- exp(lp[6] - lp[1])

rr_4_vs_2 <- exp(lp[4] - lp[2])
rr_6_vs_4 <- exp(lp[6] - lp[4])

# cat("\nSpline model contrasts:\n")
# cat("Visit 2 vs Visit 1 rate ratio:", round(rr_2_vs_1, 3), "\n")
# cat("Visit 3 vs Visit 1 rate ratio:", round(rr_3_vs_1, 3), "\n")
# cat("Visit 4 vs Visit 1 rate ratio:", round(rr_4_vs_1, 3), "\n")
# cat("Visit 5 vs Visit 1 rate ratio:", round(rr_5_vs_1, 3), "\n")
# cat("Visit 6 vs Visit 1 rate ratio:", round(rr_6_vs_1, 3), "\n")
# 
# cat("Visit 4 vs Visit 2 rate ratio:", round(rr_4_vs_2, 3), "\n")
# cat("Visit 6 vs Visit 4 rate ratio:", round(rr_6_vs_4, 3), "\n")

# These contrasts illustrate how the implied time effect can vary
# across different parts of follow-up when time is modeled flexibly.

# -----------------------------
# 17. Summarize true vs estimated effects
# -----------------------------

# --- 1. True effects (relative to visit 1) ---
true_rr <- exp(true_time_effect)

true_df <- data.frame(
  visit = as.numeric(names(true_rr)),
  true_rr_vs_v1 = as.numeric(true_rr)
)

# --- 2. Linear model estimate ---
beta_hat <- fixef(fit_linear)$cond["visit"]
rr_linear <- exp(beta_hat)

# Linear model implies cumulative effect per visit relative to visit 1
linear_df <- data.frame(
  visit = 1:6,
  linear_rr_vs_v1 = exp(beta_hat * (1:6 - 1))
)

# --- 3. Spline model estimates ---
contrast_dat <- data.frame(
  visit = 1:6,
  log_baseline = mean(dat$log_baseline),
  offset_log = 0,
  id = NA
)

lp <- predict(
  fit_spline,
  newdata = contrast_dat,
  type = "link",
  re.form = NA
)

spline_rr <- exp(lp - lp[1])

spline_df <- data.frame(
  visit = 1:6,
  spline_rr_vs_v1 = spline_rr
)

# --- Combine all ---
summary_table <- true_df %>%
  left_join(linear_df, by = "visit") %>%
  left_join(spline_df, by = "visit") %>%
  mutate(
    true_rr = round(true_rr_vs_v1, 3),
    linear_rr = round(linear_rr_vs_v1, 3),
    spline_rr = round(spline_rr_vs_v1, 3),

    true_pct = round(-(1 - true_rr) * 100, 1),
    linear_pct = round(-(1 - linear_rr) * 100, 1),
    spline_pct = round(-(1 - spline_rr) * 100, 1),

    `True RR (% reduction)` = paste0(true_rr, " (", true_pct, "%)"),
    `Linear model RR (% reduction)` = paste0(linear_rr, " (", linear_pct, "%)"),
    `Spline model RR (% reduction)` = paste0(spline_rr, " (", spline_pct, "%)")
  ) %>%
  select(
    `Visit` = visit,
    `True RR (% reduction)`,
    `Linear model RR (% reduction)`,
    `Spline model RR (% reduction)`
  )

knitr::kable(summary_table, align = "c")

The above table shows the rate ratio (% reduction) for each post-baseline visit relative to the first post-baseline visit. The linear time model assumes a constant rate ratio per visit (i.e., a constant multiplicative change between consecutive visits). However, when comparing each visit to the first post-baseline visit, these effects accumulate over time, resulting in different rate ratios at later visits (i.e., RR for visit 3 relative to visit 1 is ). Thus, while the per-visit effect is constant, the cumulative effect relative to visit 1 is not.

Because this is a single simulated dataset, the estimated effects are not expected to recover the true visit-specific effects exactly. The purpose here is to illustrate how different model specifications capture the longitudinal trajectory.

The two models capture the trajectory differently. The linear time model imposes a constant multiplicative change per visit, which accumulates over time and leads to increasingly large reductions at later visits. As a result, it overestimates % reduction when the true effect plateaus (e.g., visits 4–5).

In contrast, the spline model allows the trajectory to vary over time and better reflects the nonlinear pattern in the data, including smaller reductions at earlier visits and a subsequent plateau. While it does not recover the true visit-specific rate ratios exactly in this single simulated dataset, it provides a more flexible and realistic representation of the underlying trajectory.

Takeaway

In single-arm settings, changes from baseline (including PCFB and responder rates which are derived from PCFB) do not identify a treatment effect, as they conflate natural fluctuations, regression to the mean, or other time effects with treatment.

A longitudinal negative binomial model provides a more principled way to evaluate post-baseline trajectories by using all available data and accounting for baseline differences and within-subject variability. Importantly, how time is modelled matters. A linear time effect imposes a constant multiplicative change per visit, which can be overly restrictive and may misrepresent the underlying trajectory. Flexible approaches, such as splines, allow the data to inform how the post-baseline trajectory evolves over time. These models should be viewed as tools for describing longitudinal patterns, using time as a proxy for treatment exposure rather than establishing a causal treatment effect.

References

Hernán, M. A., & Robins, J. M. (2020). Causal inference: What if. Chapman & Hall/CRC.

Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945–960. https://doi.org/10.1080/01621459.1986.10478354

Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701. https://doi.org/10.1037/h0037350

U.S. Food and Drug Administration. (2023). Considerations for the design and conduct of externally controlled trials for drug and biological products (Draft Guidance). https://www.fda.gov/media/164960/download

U.S. Food and Drug Administration. (2019). Demonstrating substantial evidence of effectiveness for human drug and biological products (Draft Guidance). https://www.fda.gov/media/133660/download

U.S. Food and Drug Administration. (2019). Rare diseases: Natural history studies for drug development (Guidance for industry). https://www.fda.gov/media/122425/download

Powering on responder status

Responder status is typically derived as:

What should be done instead?

For seizure endpoints:

1. Specify the estimand on the count scale using an appropriate treatment contrast (e.g., a rate ratio) and convert the result to a clinically interpretable quantity (e.g., a rate ratio of 0.7 translates to a 30% reduction in seizure frequency for active compared to placebo)

2. Specify a realistic data-generating process (e.g., negative binomial with overdispersion)

3. Simulate trials under that model

4. Fit the intended analysis model

5. Estimate empirical power

This ensures:

• Estimand coherence

• Proper variance representation

• Honest operating characteristics

Takeaway

Seizures are counts. Replacing counts with percent change or responder status changes the estimand and therefore the scientific question.

PCFB and responder analyses do not estimate a treatment effect on seizure frequency. They estimate contrasts in transformed or thresholded quantities whose statistical properties and interpretation are disconnected from the underlying count process.

When the estimand is misaligned with the seizure-generating process, power is computed for the wrong quantity and the trial answers a different question than intended.

What is placebo response?

Placebo response refers to the overall change observed in participants assigned to placebo between baseline and follow-up. This term does not refer solely to a psychobiological “placebo effect.” In methodological literature, placebo response encompasses all sources of change occurring in the placebo arm, including natural disease course and statistical artifacts, not just expectation-driven effects.

Placebo response is therefore a composite quantity.

The decomposition problem

In practice, observed change in the placebo arm reflects multiple distinct components:

1. Natural history (disease fluctuation, spontaneous remission, cyclical patterns)

2. Regression to the mean (especially when enrollment requires elevated baseline severity)

3. Measurement error (random variation in outcome assessment)

4. Study effect (being observed, monitored, structured follow-up, adherence reinforcement, behavioral change)

5. True placebo effect (expectation-driven psychobiologic response attributable to belief in treatment)

All of these are bundled together under the label “placebo response.”

And here is the problem:

What randomization actually protects

Randomization ensures that everything happening in the placebo arm is also happening in the treatment arm in expectation.

That includes:

• Regression to the mean

• Natural history

• Measurement noise

• Study participation effects

• Expectation effects (if blinding holds)

Because the estimand is the difference between groups, all shared componensts cancel in expectation and therefore in the contrast.

This is the key insight:

The logical mistake

The reasoning often goes like this:

1. The placebo arm “improved” a lot.

2. Therefore, placebo is “competing” with the drug.

3. Therefore, we must reduce placebo response.

But this logic confuses within-arm change with the between-arm contrast that defines treatment effect.

The treatment effect is the difference between randomized groups, not the magnitude of change within one arm.

To see this clearly, consider a simple simulation.

If the goal Is better signal detection, focus on efficient modeling

If variability makes treatment effects harder to detect, the principled solution is to improve statistical efficiency.

Takeaway

Precision should be achieved through robust endpoints and efficient modeling that adjusts for prognostic baseline covariates.

“Placebo response” is a descriptive summary of within-arm change. It conflates regression to the mean, measurement error, natural fluctuation, study effects, and expectation effects into a single number. These components are not separately identifiable in a parallel-group randomized trial.

Apparent within-arm improvement is not evidence of a causal effect. Change from baseline, by itself, does not justify an improvement claim without an explicit comparison arm. In a randomized superiority trial, the only causal estimand the design justifies is the treatment contrast.

References

Hernán, M. A., & Robins, J. M. (2020). Causal inference: What if. Chapman & Hall/CRC.

Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945–960. https://doi.org/10.1080/01621459.1986.10478354

International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH). (2019). ICH E9(R1) addendum on estimands and sensitivity analysis in clinical trials.

Murray, E. J. (2021). Editorial: Demystifying the placebo effect. American Journal of Epidemiology, 190(1), 2–9.
doi:10.1093/aje/kwaa162

Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701. https://doi.org/10.1037/h0037350

Senn, S. (2021). Statistical issues in drug development (3rd ed.). Wiley.

U.S. Food and Drug Administration. (2019). Enrichment strategies for clinical trials to support determination of effectiveness of human drugs and biological products: Guidance for industry.

Regulatory and Reporting Standards Emphasize Concealment, Not Disclosure of Block Size

Major regulatory and reporting guidance documents are clear about the importance of allocation concealment. Notably, they do not require disclosure of block size in protocols or SAPs.

ICH E9 (Statistical Principles for Clinical Trials)

ICH E9 emphasizes that randomization protects against selection bias and provides a valid basis for statistical inference. This protection depends on the allocation sequence remaining unpredictable at the time of enrollment.

Unpredictability is the core principle. If the block size is known to investigators, predictability can arise — directly conflicting with this requirement.

ICH E6 (Good Clinical Practice)

ICH E6 emphasizes safeguarding the randomization process:

“The investigator should follow the trial’s randomization procedures, if any, and should ensure that the code is broken only in accordance with the protocol.”

Maintaining control of the randomization schedule including operational details like block size is part of preserving its integrity.

CONSORT 2010 Statement

CONSORT distinguishes clearly between:

• Sequence generation

• Allocation concealment mechanism

Item 9 of the CONSORT 2010 checklist requires:

“Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned.”

CONSORT focuses on how concealment was ensured, not on revealing details that would undermine it.

The Purpose of Block Randomization

Block randomization is used to maintain approximate treatment balance throughout enrollment.

For example, in a 1:1 trial with block size 4, each block contains:

• 2 assignments to Treatment A

• 2 assignments to Treatment B

The order of assignments within each block is randomly permuted. Since each block contains equal numbers of assignments for each treatment, treatment counts remain balanced throughout enrollment, reducing the risk of large imbalances if the trial stops early.

The Problem: Predictability

If investigators know the block size, allocation can become predictable.

Imagine:

• 1:1 randomization

• Block size = 4

• Three patients in a block have already been assigned: A, B, A

What is the fourth assignment?

It must be B.

At that point, allocation is no longer random in practice. It is deducible.

The mere ability to anticipate the next assignment can influence enrollment decisions in small but systematic ways:

• Delaying enrollment of a “difficult” patient

• Accelerating enrollment of a “good prognosis” patient

• Selectively enrolling borderline-eligible patients

This introduces selection bias, even in a formally randomized trial. That is a failure of allocation concealment.

Allocation Concealment ≠ Blinding

Even in an open-label trial (no blinding), allocation concealment must still be protected.

Blinding protects:

• Outcome assessment

• Patient behavior

• Investigator expectations

Allocation concealment protects:

• The integrity of who gets assigned what

You can have:

• A blinded trial with broken allocation concealment

• An open-label trial with perfect allocation concealment

They are conceptually distinct safeguards.

Why Block Size Does Not Belong in the Protocol or SAP

The protocol and SAP are often widely distributed:

• Investigators

• Coordinators

• Sponsors

• Regulators

• Data monitoring committees

If the block size is written there, it is no longer protected.

Best practice is:

• The protocol states that blocked randomization will be used.

• The SAP describes the randomization method in general terms.

• The actual block size (and any random variation in block size) is known only to:

  • The unblinded statistician

  • The randomization unit/vendor/system

This protects allocation concealment while still maintaining transparency about the design.

Fixed Block Sizes

Using a fixed block size makes predictability easier.

A better approach is:

• Randomly varying block sizes (e.g., 2, 4, and 6)

This dramatically reduces predictability. But even then, disclosing the possible block sizes weakens protection.

The Statistical Consequences of Broken Allocation Concealment

Empirical evidence shows that trials with inadequate allocation concealment:

• Systematically overestimate treatment effects

• Are more likely to report statistically significant results

• Introduce selection bias at enrollment

What the Protocol Should Say

Instead of specifying block size, a protocol can say:

Participants will be randomized in a 1:1 ratio using a centralized, computer-generated permuted block randomization schedule. Details of the randomization sequence will be maintained by the unblinded statistician to ensure allocation concealment.

That is sufficient. Regulators do not require disclosure of block size in public-facing documents.

Takeaway

• Block size is an operational detail. Its disclosure does not improve reproducibility, transparency, or statistical validity. But it can undermine allocation concealment.

• And once allocation concealment is compromised, the trial is vulnerable to selection bias even if it remains fully blinded.

• Protecting allocation concealment is protecting the integrity of randomization itself.

• Block size should be known only to the unblinded randomization statistician or system.

• Regulators do not require disclosure of block size in public-facing documents.

References

International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. (1998). ICH E9: Statistical principles for clinical trials.

International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. (2019). ICH E9(R1) addendum on estimands and sensitivity analysis in clinical trials.

International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. (2016). ICH E6(R2): Guideline for good clinical practice.

Schulz, K. F., Altman, D. G., & Moher, D., for the CONSORT Group. (2010). CONSORT 2010 statement: Updated guidelines for reporting parallel group randomized trials. BMJ, 340, c332. https://doi.org/10.1136/bmj.c332

Schulz, K. F., Chalmers, I., Hayes, R. J., & Altman, D. G. (1995). Empirical evidence of bias: Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA, 273(5), 408–412. https://doi.org/10.1001/jama.1995.03520290060030

Schulz, K. F., & Grimes, D. A. (2002). Allocation concealment in randomised trials: Defending against deciphering. The Lancet, 359(9306), 614–618. https://doi.org/10.1016/S0140-6736(02)07750-4

Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., & Klassen, T. P. (1998). Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? The Lancet, 352(9128), 609–613. https://doi.org/10.1016/S0140-6736(98)01085-X

Savović, J., Jones, H. E., Altman, D. G., Harris, R. J., Jüni, P., Pildal, J., Als-Nielsen, B., Balk, E. M., Gluud, C., Gluud, L. L., Ioannidis, J. P. A., Schulz, K. F., Beynon, R., Welton, N. J., Wood, L., & Sterne, J. A. C. (2012). Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials: Combined analysis of meta-epidemiological studies. Annals of Internal Medicine, 157(6), 429–438. https://www.acpjournals.org/doi/10.7326/0003-4819-157-6-201209180-00537

Simulation setup

set.seed(1)

n <- 500
x <- rnorm(n)
y <- 2*x + rnorm(n)

d <- data.frame(x, y)

Here, the outcome depends linearly on the predictor .

Model using a continuous predictor

fit_cont <- lm(y ~ x, data = d)
summary(fit_cont)

Call:
lm(formula = y ~ x, data = d)

Residuals:
     Min       1Q   Median       3Q      Max 
-2.94113 -0.74507  0.01663  0.72882  3.11131 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) -0.04496    0.04730  -0.951    0.342    
x            1.95692    0.04678  41.832   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.057 on 498 degrees of freedom
Multiple R-squared:  0.7785,    Adjusted R-squared:  0.778 
F-statistic:  1750 on 1 and 498 DF,  p-value: < 2.2e-16

This model uses the full information contained in the predictor .

The estimated coefficient represents the expected change in for a one-unit increase in .

Model using a dichotomized predictor

d$x_cat <- ifelse(d$x >= median(d$x), 1, 0)
fit_cat <- lm(y ~ x_cat, data = d)
summary(fit_cat)

Call:
lm(formula = y ~ x_cat, data = d)

Residuals:
    Min      1Q  Median      3Q     Max 
-4.6942 -1.1305 -0.0022  1.0593  6.1757 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  -1.5672     0.1017  -15.42   <2e-16 ***
x_cat         3.1330     0.1438   21.79   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.607 on 498 degrees of freedom
Multiple R-squared:  0.4881,    Adjusted R-squared:  0.4871 
F-statistic: 474.9 on 1 and 498 DF,  p-value: < 2.2e-16

This replaces the continuous predictor with a binary variable indicating whether the value of is at or above the median versus below the median.

Instead of estimating a slope, the model now estimates the difference in mean outcome between the two groups.

Why information is lost

The continuous model uses the full range of variation in the predictor. The dichotomized model only uses whether the value falls above or below a threshold. Observations with very different values of are treated as identical if they fall in the same category.

For example, values of and would both be coded as 1 if they fall at or above the median, even though they carry very different information about the outcome.

This loss of information reduces the model’s ability to explain variability in the outcome and weakens its ability to detect real relationships.

Takeaway

Categorizing a continuous predictor throws away information and reduces statistical efficiency. Unless there is a compelling clinical reason, predictors should generally be analyzed on their original continuous scale.