Comment on ESMO Magnitude of Clinical Benefit Scale.

We find the Magnitude of Clinical Benefit Scale (ESMO-MCBS) developed by Cherny et al. [1] a practical and simple-to-use tool for comparing the benefits of new cancer therapies. With a large number of oncology drugs being approved, this kind of rating of the expected treatment benefits is indeed relevant, and we commend the authors for making an important contribution. Cherny et al. propose that the lower end of the hazard ratio (HR) and 95% confidence interval (CI) is used when judging whether a treatment has meaningful efficacy when compared with the standard (comparator) treatment. However, the width of the CI depends on the number of end points accumulated, and narrows down as the trial data mature and more end points accumulate with time. For example, the trial Y depicted in Figure 1 of the article has an HR of 0.76 and a 95% CI of 0.65– 0.89, which just qualified for efficacy as the lower end of the CI hits the threshold value of 0.65 set as meaningful by the authors. When trial Y matures and more events accumulate, in the next analysis of the trial, the lower end of the 95% CI will likely no longer reach 0.65 due to the narrower CI if the HR will remain the same (0.76). Therefore, with more mature data, trial Y no longer qualifies for efficacy, although the HR remains exactly the same and there is no true change in the observed efficacy. Similarly, when the lower end of the CI is used as the criterion, small trials will qualify more easily for efficacy than large trials of identical efficacy, since small trials generally have wide 95% CIs that more frequently cross the set threshold of 0.65. We suggest that the point estimate of the HR may be a more robust measure when comparing the relative efficacy of treatments rather than the lower end of the 95% CI. The width of the 95% CI may be best viewed as further information about the HR, and meaningful threshold values should probably be sought for the HR rather than for the lower end of the 95% CI.