Real World Evidence for Health Technology Assessment: Strengths, Challenges, and Projections

There is a growing recognition that real world evidence (RWE) could play an increasingly important role in health technology assessment (HTA).

Similar to many trends that rapidly gain popularity in the HTA landscape, RWE has turned into an umbrella term. Its application ranges from the use of basic descriptive statistics derived from datasets with inherent selection factors to the implementation of more rigorous causal methods on large, nationally representative datasets. As such, RWE needs reframing to establish appropriate use cases, assess methodologies applied to real world data (RWD), and develop approaches to overcome inherent challenges.

In response to The Professional Society for Health Economics and Outcomes Research (ISPOR) Europe RWE Summit held in Barcelona on November 17^th, 2024, Lumanity and the University of Sheffield’s Centre for Health and Related Research (SCHARR) have collaborated to reflect on the role of RWE for HTA. This partnership brings together SCHARR’s academic expertise in RWE generation methodologies with Lumanity’s insights of industry demand and needs.

Our reflections align with insights obtained from the ISPOR RWE Summit and the associated discussions and debates. Our thoughts address four key topics: 

• Trends in the current use of RWE for HTA
• Causal inference methodology to strengthen RWE generation
• Challenges inherent to RWE generation
• Projections for future generation and use of RWE.

Trends in the current use of RWE for HTA

RWE has been utilized in various areas, both for direct submission to HTA agencies or as contextual supplementary evidence. In our opinion, two notable trends have emerged in the use of RWE within an HTA context:

1. External control arms (ECA): In situations where head-to-head comparison trial evidence is unavailable, such as rare diseases or due to ethical constraints, RWE allows for the development of ECAs to evaluate relevant comparators against new treatments. However, their validity hinges on the transportability of evidence, necessitating careful adjustments for differences in patient characteristics and data sources to match the target population, factoring in causal inference considerations.^{1, 2}
2. Surrogate endpoints: In situations where trials capture intermediate measures of treatment benefit, RWE can be valuable for deriving insights on both surrogate and longer-term outcomes in the patient population of interest, thereby allowing for the estimation of longer-term treatment benefit. For example, progression-free survival (PFS) is an intermediate measure often used in oncology clinical trials with a surrogate relationship with the longer-term outcome of overall survival (OS) which is often not captured within available follow-up. Surrogate and longer-term outcomes derived from RWE can help predict clinical trial outcomes. However, the validity of such evidence depends on validating the surrogacy relationship with longer-term outcomes through broader evidence networks, including data from the same drug class or therapeutic area, and supported by evolving methodological approaches.

RWE methodologies: causal inference methods to strengthen RWE generation

RWE has great potential to enhance HTA evidence packages by capturing insights beyond clinical trials, understanding real world practice, improving evidence generalizability, and enabling comparisons of treatment strategies that are impractical to evaluate in trials.

With the growing acceptance of RWE, there is a notable shift towards adopting novel methodologies – traditionally used in a regulatory context – within the HTA landscape. For example, we have observed the growing use of causal inference methods such as the Target Trial framework and g-methods³, which are gaining traction due to their ability to reduce bias when estimating comparative effectiveness using RWD. By structuring observational studies to replicate the key elements and design of randomized controlled trials (RCTs), these approaches facilitate the use of appropriate statistical methods to address confounding and reduce self-inflicted biases inherent in study design. Recent HTA RWE guidelines, such as the National Institute for Health and Care Excellence (NICE) RWE framework¹, emphasize the importance of exploring these techniques. We anticipate these methodologies being increasingly requested of manufacturers by HTA agencies when leveraging RWE.

Several of the authors of this paper are exploring the use of causal inference methods to develop benchmarking studies that evaluate the feasibility of using RWD to generate trial-like comparative effectiveness results for HTA submissions. Members of the ReCREATE group at SCHARR (Nicholas Latimer, Amy Chang, Saleema Rex, and colleagues) are using English Cancer Registry and US electronic health records (EHR) data to conduct benchmarking studies.^{4, 5, 6} Matthew Franklin is using English linked mental health service, hospital, and prescribing data for this purpose as part of the Target Trials in Mental Health projects funded by the National Institute for Health and Care Research (NIHR).⁷

RWE challenges: data quality and suitability

Despite the availability of advanced statistical approaches, the biggest challenge for RWE lies in the underlying data. Robust RWE requires RWD to encompass several key aspects – including representative study samples; the outcomes of interest tracked over a sufficient follow-up period; clearly detailed interventions throughout treatment; and enough information on confounders (both time-fixed and time-varying). To ensure the data are fit-for-purpose, a comparative assessment should be conducted to identify relevant data sources, thereby minimizing selection bias in both the data source and the cohorts derived from it.

Future projections

The role of RWE in HTA continues to evolve. Below are some insights we gathered at the ISPOR summit, which may indicate future trends in this field:

Dynamic treatment regimen (DTR), treatment sequences or pathways: With the expanding range of therapeutic options, understanding the clinical efficacy of DTR⁸, including treatment sequences or pathways^{4, 9} (i.e., a specific type of DTR), becomes essential. RWE addresses evidence gaps by enabling the evaluation of different treatment strategies, such as entire treatment pathways that trials typically do not assess. Furthermore, causal inference methods can be applied to generate RWE that emulates head-to-head comparative effectiveness of different treatment sequences, and potentially be adapted to create ECAs involving treatment sequences.⁹ Our co-author Amy is contributing to the development of these methods within the SCHARR ReCREATE Project, led by Professor Nicholas Latimer.⁴

Early Value Assessment (EVA): There is a growing recognition that many health technologies could be made available to the public earlier than current HTA timelines allow. NICE’s EVA for Medical Technologies (MedTech) demonstrates that technologies proven to be safe and effective can receive early recommendations for conditional use, provided there is a commitment to an evidence generation plan that emphasizes RWE.¹⁰ With increasing interest in MedTech and the digitization of healthcare systems globally, it seems reasonable to suggest that such an EVA process and associated role of RWE could garner further interest internationally. Our co-author, Matthew, is developing RWE to inform NICE’s EVA process as part of the NIHR-funded Target Trials in Mental Health projects, specifically the NIHR Invention for Innovation (i4i) & Office for Life Sciences (OLS) funded SilverCloud Target Trials focused on digital, individual, and group cognitive behavioral therapy (CBT) modalities.⁷

Heterogeneity and inequality/inequity: A key policy objective for decision makers worldwide has been to reduce health inequalities by maximizing total health while minimizing health disparities. Understanding disease burden in diverse risk groups is increasingly relevant for HTA agencies. RWE can provide valuable insights into disease susceptibility and disparities, enabling equitable assessments. As HTA evolves to address policy and political pressures to account for equality and equity in evidence-based decision making, the role of RWE to understand (in)equalities and (in)equities related to HTA will become increasingly important.

Summary

The role of RWE in HTA is still evolving, with certain areas such as ECAs and surrogates currently taking precedence. However, other areas, including treatment sequencing, EVAs, and understanding heterogeneity are gaining prominence. While appropriate methods for these tasks are being increasingly recognized and are themselves evolving, RWD must also advance to ensure it can support robust RWE generation. The ISPOR RWE Summit suggests a promising future across all these areas, with experts from organizations like Lumanity and SCHARR eager to address the challenges and opportunities ahead.