The Risk of Uninformative Trials: Statistical Considerations in Radiotherapy Trials
In today’s guest post, I’m pleased to welcome Susannah Ellsworth, a radiation oncologist at the University of Pittsburgh and Residency Program Director at UPMC Hillman Cancer Center. She expands on a recent commentary we published together and its implications for past and ongoing radiotherapy trials. Timothée
“A good decision is based on knowledge and not on numbers.”
Plato, Laches, or Courage
Clinical trial design is fraught with difficult choices. Investigators must balance feasibility and budget considerations with the need to generate results and are often incentivized to minimize sample sizes, as smaller studies are cheaper and accrue faster. Several design choices affect sample size calculations, including effect size estimates and approach to significance testing. The essay below will discuss how these often-overlooked factors can affect clinical trial design and interpretation, expanding on issues raised in our recent commentary published in the European Journal of Cancer.
Relation between sample size and effect size
Sample size depends upon anticipated effect size and desired statistical power. This relationship can be illustrated by a hypothetical experiment evaluating two “unfair” coins: Coin A will turn up heads on 2/3 of tosses, whereas Coin B will land on heads on 51/100 tosses. After how many tosses will you feel confident that Coin A is unfair? What about Coin B? Intuitively, more tosses are needed to identify the smaller effect size associated with Coin B. Similarly, a larger projected effect size allows clinical trials to utilize a smaller sample, as shown in the graph below.
Underestimating effect size leads to inefficiencies by requiring overly large sample sizes, slowing accrual and inflating budgets, whereas overly ambitious projections can cause a trial to fail to meet its primary endpoint, making effect size estimates a critical aspect of trial design.
While clinical trialists obviously must make educated guesses regarding effect size – after all, if the true magnitude of benefit associated with an experimental therapy were known, running a clinical trial would be pointless – such estimates should still be grounded solidly in prior data.
Risks of inaccurate effect size estimates
Clinical trials are occasionally critiqued for inflating projected effect size by including a suboptimal control arm. A related but distinct issue is a study which retains equipoise, but whose inaccurate effect size estimates make it unlikely to reach its primary endpoint.
Illustrative of the consequences of unrealistic effect size estimates is NRG-GI-003, a recent randomized study demonstrating no benefit to proton versus photon SBRT for hepatocellular carcinoma (HCC); 2-year overall survival (OS) was 56.2% versus 69.3% with protons vs. photons, respectively.
Initial discussions of GI-003 characterized outcomes with photon therapy as a surprising overperformance relative to the study’s original projections. However, multiple other studies – all published prior to GI-003’s activation – suggest that the single paper cited to substantiate GI-003’s control-arm OS estimates was likely an outlier (Bujold 2013; Table 1). Furthermore, even as GI-003 went on accruing patients, additional data continued to emerge showing far better outcomes with photon SBRT than the trial had planned for. For example, a 2024 meta-analysis of photon SBRT for HCC reported a pooled 3-year OS estimate of 57%. The study accrual period (2017-2026) also saw FDA approvals for six systemic regimens in HCC, all of which could have influenced OS among patients on GI-003, which did not protocolize systemic therapy.

Clear parallels can be drawn between NRG-GI-003 and NRG-GI-011, which randomizes patients with unresectable pancreatic cancer to dose-escalated RT versus physician’s choice of continued chemotherapy, conventional chemoradiation, or observation.
Like GI-003, GI-011 features a poorly substantiated effect size estimate; no citation at all is provided for the control arm’s two-year OS estimate of 15%, which likely underestimates projected OS, especially since the trial only enrolls responders to induction chemotherapy. GI-011 similarly provides no mechanism to adapt to emerging advances in systemic therapy (e.g., daraxonrasib) that may affect OS, creating substantial risk that it will also fail to meet its primary endpoint.
Clinical Equipoise: One-Sided or Two-Sided Hypothesis Testing?
Sample size calculations are also affected by statistical significance considerations; statistical power decreases as sample size is reduced, assuming effect size remains constant.
Investigators can address this problem by relaxing the prespecified significance level α, albeit at the cost of an increased type I error rate. Alternatively, substituting one-sided for two-sided significance testing can permit a trial to maintain statistical power with a smaller sample size.
However, one-sided significance testing can only assess whether the magnitude of treatment effect exceeds some threshold in a single, prespecified direction. This is difficult to square with the principle of clinical equipoise, which requires genuine doubt regarding which of the arms on a randomized superiority trial is better.
GI-003 opened under the premise that equipoise was present between proton and photon SBRT for HCC; its original design required 155 evaluable patients, providing 85% power to evaluate the primary endpoint with a two-sided α of 0.05. However, immediately before closing, the trial underwent a major statistical revision. Sample size was reduced to 112; in order to maintain the original effect size projections, power was reduced to 80%, and one-sided significance testing substituted for two-sided. Ad hoc revision of the statistical approach in this way is difficult to justify, especially since this change foreclosed any possibility of assessing whether there was in fact harm from proton SBRT, which was associated with numerically inferior OS. Similarly, GI-011 utilizes one-sided significance testing, not only raising the same questions about whether this statistical approach is consistent with clinical equipoise in a randomized superiority study but likewise precluding the evaluation of whether dose-escalated SBRT for pancreatic cancer is detrimental, should the study fail to meet its primary endpoint.
Resource constraints increase the risk of uninformative trials
Design choices that minimize sample size may seem to be a pragmatic response to resource constraints.
However, overestimating effect sizes and inappropriately utilizing one-sided significance testing incur real risk that a trial will deliver uninformative – not just negative – results.
Uninformative clinical trials waste scarce resources and violate patients’ expectations that trial participation will represent a meaningful contribution to medical science. Avoiding this outcome requires careful scrutiny of a trial’s effect size projections and approach to significance testing, ideally before enrollment begins.




Discussion about this post