Five Learnings Manufacturers Should Take from the Latest NICE Guidance on Survival Modelling

Authored by BresMed, now part of Lumanity

The last time NICE provided formal guidance on the analysis of survival data was in March 2013, 3 months after it approved ipilimumab for previously treated advanced melanoma.¹ ² This was one of the first oncology interventions to demonstrate a deep and durable response in a proportion of patients. Of course, the guidance provided in 2013 was limited in its applicability to such interventions – that is, those for which different portions of the treated population might have dramatically different prognoses.

Since then, a number of effective immunotherapies have launched, not least the chimeric antigen receptor (CAR) T-cell therapies. In Technical Support Document (TSD) 21, NICE has finally provided guidance on the applicability of a range of methods that are often proposed to be more appropriate for these transformative therapies.³ Given NICE’s international reputation for methodological rigour, the guidance provided in TSD 21 is likely to have a global influence.

For those submitting to health technology assessment (HTA) bodies in the UK and beyond, the bar has now been set significantly higher – here, we draw out five learnings for manufacturers as they endeavour to adjust.

1. Clinical plausibility, clinical plausibility, clinical plausibility.

As the number of available methods continues to rise, it becomes ever more important to bear in mind NICE’s guidance that:

“Careful thought should be given to the biological and clinical justification to any statistical approach selected; the approaches detailed herein should not be considered as an extended list of survival methods to “try out” on data. Instead, care should be taken to think through the underlying mechanisms likely to be dictating short and long-term hazard/survival functions.” ³

Careful, systematic and methodical thinking will be required to manage the total number of permutations, develop a considered analysis strategy, and present a coherent overall narrative. While this is not specifically a feature of NICE guidance, it should nevertheless be considered best practice in health economics and outcomes research.

2. Explicitly focus on the hazard function.

Yes, technically we have always done this. However, typically we have predominantly focused on the visual goodness of fit to the observed survival function, rather than on the clinical coherence of our modelled hazard function (from which the survival function is derived). When selecting a model, make sure you can justify your approach relative to a) the shape of the observed hazard function in the trial, and b) the likely shape of the hazard function for each treatment in the long-term extrapolation period.

The importance of robustly estimating the evolving shape of the hazard for each treatment over time means that methods assuming simple proportional hazards are falling out of favour. Relative treatment effects within a trial will often need to be modelled implicitly (using the difference between the survival extrapolations through time), rather than explicitly via the hazard ratio.

3. Data external to the trial are only becoming more important.

External data are crucial to robustly estimating the shape of the hazard function beyond the trial. Traditional approaches to incorporating external data such as general population mortality or expert opinion will now be seen as the bare minimum. Additional options to consider include using earlierphase clinical trials with longer follow-up to inform expected outcomes for a new treatment, and using other available sources to inform expected outcomes for standard-of-care treatments (e.g. earlier published clinical trials with longer follow-up, observational cohort studies, and patient registries).

Of course, identifying, accessing and incorporating the most appropriate external evidence takes time. The guidance is clear on the need to incorporate external data, but lighter on practical solutions to do so. For assets relying on the extrapolation of time-to-event data, early HTA planning must now include the identification of relevant external data sources as potential prior information in the design and planning phases.

4. Mixture cure modelling should be used with caution.

Arguably the context was perfectly set for NICE to embrace the use of mixture cure models in certain instances (e.g. CAR T-cell therapies). Instead, NICE’s Decision Support Unit (DSU) has highlighted that mixture cure models will rarely be the most appropriate option:

“If the number at risk towards the end of follow-up is low, which is common in randomised trials, then it is highly questionable as to whether it is sensible to impose such a strong assumption as cure.”³

For interventions such as CAR T-cell therapies where a cure or a ’deep and durable response’ may be a clinically plausible aspiration, the following should be considered early in the planning process:

• Is a cure (deep and durable response) clinically plausible for this particular health condition?
• What is the mechanism by which a deep and durable response can occur for this health condition?
• What evidence will be required to support the plausibility of a deep and durable response – and how will this be collected?
• What evidence is available for this health condition regarding the extent to which people who experience a deep and durable response continue to have a heightened mortality risk relative to the general population (given, for example the burden of prior treatment)?
• What proportion of patients are expected to experience a deep and durable response?
• When are the trial data expected to reach a sufficient duration of follow-up and maturity to support observation of a deep and durable response?

Tellingly, even where there is a strong rationale for assuming the intervention causes a proportion of patients to have a prognosis similar to that of the general population, NICE notes that:

“…cure models hold few advantages compared to FPMs [flexible parametric models] that are fitted incorporating background mortality using a relative survival or excess mortality framework. Cure models perform poorly if they are applied and an assumption of ‘cure’ is not reasonable”.³

Of course, flexible parametric models won’t always be the most appropriate option – but mixture cure models rarely will.

5. Pay attention to what is not included.

The guidance is very much focused on within-trial analysis of a randomized controlled trial with two or more arms where patient-level data are available. No detail is provided on how to draw robust extrapolations through time for a range of comparators outside of the clinical trial. Is the hazard ratio between these interventions likely to remain proportional through time? Do the trial populations differ in severity, leading to population-adjustment methods being required?

It is entirely possible that population adjustment, treatment switching, and survival extrapolation may all be required in the same project. Each of these areas includes a range of different methods – and each of these methods commonly needs a variety of scenarios and sensitivity analyses to explore different approaches and requires sufficient data to do so. We would therefore strongly recommend that careful consideration is given to how to plan and present a series of analyses as a coherent overall narrative.

Closing questions

Do your current HTA preparation steps enable you to identify:

• External data to inform the longer-term extrapolation of the baseline hazard for your intervention and the wider comparators?
• Clinically informed judgements concerning the degree of persistence in your intervention’s survival benefit?
• A succinct analytical approach, integrating both trial and external data, to give a clinically plausible narrative of your intervention’s expected relative clinical benefit through time?

For more information, or to discuss how we can help you prepare for your next HTA, please contact us.