ISPOR Europe Diaries 2023

The ISPOR Europe conference was dominated by the latest news about Artificial Intelligence (AI) and Joint Clinical Assessment (JCA). Our consultants were keen to compare notes and conference discussions about AI’s place in health technology assessment. There were stimulating discussions about real-world evidence and real-world data (RWE/RWD), healthcare digitization and the European Health Data Space (EHDS).

Artificial Intelligence

The rapid rise of Artificial Intelligence (AI) and its potential impact on Health Economics and Outcomes Research (HEOR) and Health Technology Assessment (HTA) was discussed everywhere at ISPOR Europe. However, as far as we know, European HTA bodies have not issued any guidance on how AI should be used. At Lumanity, our HEOR and HTA experts will be monitoring how this develops. We will be thinking about the challenges AI might bring within our space and how those challenges might affect trust, transparency and reproducibility.

AI is powerful, but it can make mistakes and those mistakes can be hard to identify. AI is designed to deliver answers that it predicts are right based on all the data it has trained on. The discussions at ISPOR tell us we need a firmer grip on the degree of additional benefit AI can bring compared to the traditional methods we rely on in HEOR. AI and Machine Learning (ML) can generate reams of data and predictions based on what they know, so there are clear reasons to see AI’s potential applications across HEOR and HTA. Consider for example how AI and ML might apply to Cohort stratification, Causal inference, Predictive modeling, Economic evaluation, Knowledge synthesis, identifying new systematic literature review (SLR) hits proactively that will trigger data extraction when a threshold number of hits are identified, and Predicting Population, Intervention, Comparison and Outcomes (PICOs) criteria across countries. ISPOR Europe has given us much to think about. AI is coming but it cannot fully displace the need for human expertise yet.

Real-World Evidence

RWE and EU policy

When we think about the future of healthcare, we think almost automatically about how useful electronic data can be in driving product innovation and giving patients, their clinicians and others access to health data and appropriate healthcare or healthcare tools. One focus at ISPOR Europe was the European Health Data Space (EHDS), an on-going project that emerged from a European Commission regulation². The EHDS extends the General Data Protection Regulation (GDPR), and the NIS 2 Directive. It aims to create a legal framework for how health data is used for research, innovation, public health, policy-making and regulatory purposes. We are told that under strict conditions, researchers, innovators, public institutions and industry will gain access to health data, essential for developing life-saving treatments, vaccines and medical devices. Ultimately, the aim is better access to healthcare and cross boundary health systems. But there are some caveats that we are already thinking about. The HTx project is already comparing AI and traditional methods for analyzing real-world data. HTx guidelines may need to be living documents given how quickly AI is developing. We also think academics will need to support HTA decision makers but some upskilling will be needed to ensure they can do this well. 

RWE and HTA in rare diseases

RWE and randomized controlled trials (RCTs) are not mutually exclusive – they are complementary. Especially in the case of rare diseases and the challenges with data due to rare diseases having small patient populations. Evidence gaps are inevitable in rare diseases, but decision makers appreciate efforts to generate reliable and relevant data to address the decision problem. The RWE framework from NICE³ encourages best practice for planning, conducting and reporting RWE studies. Since the publication of the framework in June 2022, a range of use of RWE has been identified in different appraisals.⁴ But RWE has some limitations: in contexts where it is feasible to undertake RCTs RWE should not provide the primary basis for estimating relative efficacy due to the risk of bias on observables. Additionally, RWE sometimes cannot inform prognostic factors of newly discovered biomarkers as it does not feature in historical datasets. RWE does not/cannot exist for a treatment landscape that has recently evolved

Use of RWE to build an external comparator arm 

One of the key RWE discussions at ISPOR Europe was whether RWE can be used to build an external comparator arm in contexts where RCTs are unfeasible. Given the need for attention to causal relationships RWD analyses are more complex than RCTs. It also requires a good understanding of the clinical context and how data were generated. There is little guidance on how to assess HTA submissions based on single-arm trials and HTA authorities in some countries do not have the expertise to evaluate indirect evidence, especially using RWE sources. This means a route is needed for HTA agencies seeking to develop a principled, pragmatic approach to external controls. This route should include early interactions between the company and HTA bodies. 

RWE can provide additional information, but high-quality data is needed, and gaps/uncertainties are inevitable. The decision should consider the expected magnitude of benefit, the unmet need and the assessment of uncertainty. Managed Entry Agreements (MEA) need to be designed to address the uncertainty around the relative treatment effectiveness, not the effectiveness of the drug. In our experience, DAGitty diagrams (graphical tools used to analyze causal diagrams) may be used to decide the prognostic factors and treatment effect modifiers that you might want to control for. 

Joint Clinical Assessment

The EU HTA Regulation entered into force in January 2022 and will start to apply in January 2025. The Regulation will provide a permanent framework for JCA in the EU. JCA aims to speed up patient access to new health technologies across the EU in a consistent framework. We have been tracking the development of the EUHTA Regulation and our senior consultants have been active participants at the stakeholder meetings.⁵ At ISPOR Europe we learned that the JCA regulations have been termed the “PICO regulation”. The reasons for this are clear given the PICOs process for JCA.⁶ A simulation of a consolidated EU PICOs for a hypothetical product in oncology indicates a potential high number of PICOs and significant analysis burden for health technology developers (HTDs) and HTA bodies. 

At ISPOR Europe the consensus was that there will be no formal or practical limit on the number of PICOs, however it was acknowledged that 27 PICOs would be impractical for assessors and HTDs. Gergo Meresz of the Joint Scientific Consultation (JSC) group hoped that in time variation in population and comparators across countries can be consolidated into fewer PICOs than member states and eventually any given JCA will result in no more than 10 PICOs. Resourcing was highlighted as one of the key anticipated challenges. Given our work in HTA submissions in Ireland, we were reassured that the process in Ireland will not need to adapt too much for the JCA, given the current structure of National Centre for Pharmacoeconomics (NCPE) submissions – rapid review: full HTA assessment.

We also noted that for Agenzia Italiana del Farmaco (AIFA), the Italian Medicines Agency, the JCA could significantly simplify the work of the scientific and economic committee, “Comitato Scientifico ed Economico” (CSE) by providing the data analysis of the scientific appraisal based on agreed PICOs and allowing the CSE to focus more on price negotiation. However, there are uncertainties around timelines and the new requirements will also drive the need for resourcing and upskilling national HTA capabilities. 

Additional support or resources may also be needed to help smaller companies/biotech companies (that have small HEOR teams) to complete the process. Strong project management, living horizon scanning and early evidence generation strategy planning will be essential to success. To speed the process and reduce the resource burden, we advise using automation for analyses which will allow more time for interpretation. 

Several issues that were flagged at ISPOR Europe included:

• The need to reduce the number of PICOs to help companies provide the relevant analyses. To do this, companies will need to be involved throughout the process and the PICOs should be based on key guidelines/treatment landscape 
• Predicting the most likely PICOs is going to be essential to help inform/guide evidence generation activities, data synthesis and JCA dossier preparations. To achieve this, companies should engage in early scientific advice, as this will allow them to gain insight into initial PICOs prediction, and also provide input into pivotal trial design to ensure all relevant evidence will be generated 
• One panelist proposed that increased engagement from HTD on a base case PICO would increase the quality and the value of the EUnetHTA 21 process. During the request for JCA, the HTD can provide evidence on disease background, clinical practice and endpoints to inform PICOs. HTDs might also consider proposing sensible base case PICOs based on their ongoing research in the area

Significantly, the discussion concluded that there will not be enough data to answer some PICOs. The research question is policy driven rather than data driven. If there is a data gap, this should be transparently stated. The HTD should demonstrate that it has explored all possible avenues to fill the evidence gaps. To support this we would advise that the patient voice should be involved in the whole process to increase transparency.

Ultimately, the JCA will be beneficial to the pharmaceutical industry and will lead to faster commercialization of products, but each assessment will require meticulous planning years in advance. Companies should engage with the JSC when planning their trial design and JCA strategy. Essentially, the more information and effort that is incorporated into an RCT the better. 

Our statisticians and modelers were also drawn to the many discussions about upcoming developments in statistical methods which we will summarize in a future article – watch this space!  

References

1.           Gladwell D. BN. 2023. Available at: ISPOR Europe – a good reason to get out of the office? – Lumanity. Accessed: 21 November 2023.

2.           European Commission. 2022. Available at: Proposal for a regulation – The European Health Data Space (europa.eu. Accessed: 21 November 2023.

3.           NICE. 2022. Available at: NICE RWE Framework.

4.           Duffield S, & Jónsson, P. . The real-world impact of National Institute for Health and Care Excellence’s real-world evidence framework. Journal of comparative effectiveness research, . 2023; 12(11).

5.           Matthijsse S. Learnings from the Latest EUnetHTA21 HTD Meeting. 2023. Available at: https://lumanity.com/perspectives/learnings-from-the-latest-eunethta-21-htd-meeting/. Accessed: 21 November 2023.

6.           EUnetHTA21. D4.2 Scoping Process. 2023. Available at: https://www.eunethta.eu/d4-2/. Accessed: 21 November 2023.