Across all indications, the primary source of trial failure has been—and remains—an inability to demonstrate efficacy.¹ An analysis of 640 phase 3 trials with novel therapeutics demonstrated that 54% failed in clinical development. Among those that failed, more than half—57%—failed due to inadequate efficacy.² In neuroscience indications, an estimated 85% of late phase studies fail, with factors such as subjective endpoints, patient and disease heterogeneity, rater variability, and high placebo response rates contributing to difficulty in detecting treatment effects.

In this blog, we discuss how a combination of technology and expert clinician review can enhance risk and signal detection in neuroscience studies.

Using technology to improve risk detection

Risk-based monitoring

Risk-based monitoring has become increasingly common in neuroscience studies and involves both oversight of important and likely risks and monitoring of risks that may be less likely to occur but that could have a significant impact on data quality.

Electronic Patient Reported Outcomes

Many neuroscience trials rely on self-reporting of symptoms by study participants. Some also incorporate observational reporting by caregivers. Electronic patient reported outcomes (ePRO) technology can help optimize the objectivity of this inherently subjective data. ePRO allows for more frequent and timely data collection, helping to reduce the influence of the placebo response, improve patient adherence and engagement, and identify treatment effects. ePRO technology offers several potential advantages including:

Direct data entry to eliminate transcription errors that can occur with paper PROs
Automated input validation to ensure entries are valid and complete
Electronic time stamps to encourage more timely entries, reducing recall bias
Lower patient burden due to ease of use compared to paper PROs and potential reduction in the number of site visits and in-person assessments required
Lower site burden since ePRO can be used to automate not only the process of collecting and storing PRO data, but also the process of reminding patients about study-related tasks

Some ePRO technologies include real-time data monitoring features that can send pre-defined messages to patients if their responses meet or exceed pre-determined thresholds.

Electronic Clinical Outcomes Assessment

Electronic clinical outcome assessment (eCOA) technology is useful in neuroscience trials due its ability to gather audio, photo, video, patient, and clinician data and to integrate the full range of clinical assessments—performance-based, clinician-reported, and PROs—in a single platform. eCOA functionality can also be customized with advanced workflow and scoring automations that enable reliable, standardized data collection for cognitive endpoints.

Incorporating expert clinician review

In our experience, while technology is integral to neuroscience study success, it may not be a standalone solution for risk and signal detection. Augmenting regular data review and sponsor discussions with scale reviews by an experienced clinician reviewer further improves data quality. There are several ways to incorporate expert clinician reviews:

At the study level to look for variability and trends
At the site level to examine rating scores across each visit for each participant for unexpected inconsistencies
At the participant level to evaluate scores that are more than 1-2 standard deviations outside of the average scores per visit

Benefits of using a clinician reviewer

Ideally, an expert clinician reviewer has clinical trial experience specific to the disease under investigation and the relevant rating scale(s), either as an investigator, rater, or rater trainer. The reviewer should also have a deep understanding of the protocol and the potential pitfalls in drug development.

The benefits of utilizing a clinician reviewer include deep disease and scales knowledge relevant to the indication under investigation and ability to access—and act on, if necessary—data in near-real time. Scheduled reviews by an experienced clinician also help to reduce accidental—or rarely intentional—false entries and improve data quality, contributing to an overall increase in the probability of detecting a true treatment effect.

Case studies

In our experience, technology and expert clinician review work synergistically to enhance risk and signal detection.

In a global Phase 3 randomized, placebo-controlled study of major depressive disorder (MDD) where the Montgomery–Åsberg Depression Rating Scale (MADRS) score was the primary endpoint, regular data review revealed two patients with large total MADRS score changes (see Figure 1).

Figure 1. Large total MADRS score changes in two patients in a Phase 3 MDD study

While technology can be programmed to flag these large score changes using predefined thresholds, it is extremely helpful if an expert clinician examines the data in near-real time to see if it makes sense, to identify potential causes or explanations for unexpected data, and to remediate as appropriate.

In another global Phase 3 randomized, placebo-controlled MDD trial, low site variability in total MADRS score was observed, with four subjects having similar MADRS scores throughout the entire study (see Figure 2).

Figure 2. Four clinical trial subjects with nearly identical MADRS scores in an MDD study

This pattern would be extremely difficult for technology to detect unless the programming was very sophisticated. However, an expert clinician reviewer examining site performance would recognize that it is highly unlikely for four subjects to have such similar scores, especially since half of the study participants were given active drug and the other half were given placebo.

Key takeaway

For developers of neuroscience therapies, proactively implementing a strategy to enhance risk and signal detection is essential for improving data quality and evaluating response. Leveraging the right technology to automate workflows and incorporating an expert clinician reviewer to evaluate and analyze signals over the course of a study enables early detection and timely remediation of risks, improving the likelihood of demonstrating true treatment effects.

To learn more about optimizing risk detection in neuroscience trials, view our on-demand webinar.

[1] Fogel DB. Factors associated with clinical trials that fail and opportunities for improving the likelihood of success: A review. Contemp Clin Trials Commun. 2018;11:156-164.

[2] Hwang TJ, et al. Failure of investigational drugs in late-stage clinical development and publication of trial results. JAMA Intern Med. 2016;176:1826-1833.