Best Practices for Detecting Data Anomalies Suggestive of Fabrication or Misconduct

admin-ajaxBy Alan Kott, MUDr, Clinical Vice President and Practice Lead at Bracket

Data quality concerns are frequent in clinical trial data. At Bracket, we have always been focused on improving data quality in our programs. Many of our Data Quality Assurance programs, such as our Blinded Data Analytics tool, are designed to address problematic data before it spreads. As Risk-Based Monitoring and Centralized Statistical Monitoring have become more common, much more attention is being paid to how these programs are implemented.

The recent article from TransCelerate successfully explored the utility of statistical monitoring in identifying data fabrication in chronic obstructive pulmonary disease (COPD) clinical trial. This paper is meant to validate some of the Central Statistical Monitoring recommendations already made in earlier TransCelerate white papers.

Bracket has been employing iterations of CSM for many years. In most cases, it is applied in studies with difficult, subjective endpoints. Especially in Central Nervous System (CNS) studies, variability, inter-rater reliability, and high placebo response can negatively impact a study outcome. Being vigilant in your ongoing data monitoring can be an essential tool in ensuring your clinical trial is measuring what you set out to measure. For example, in our dataset of 14 double-blind phase 2 and 3 schizophrenia clinical trials, data quality concerns affect between 11.7 to 31.2% visits with a mean of 24.2%. Many of these data quality concerns (e.g. lack of variability, discordance, hyperconcordance, etc.) have the potential to seriously alter study results, especially if not randomly distributed in the dataset.

Fig1_Any Data Quality Concern

Many of the recommendations in this new paper are straight-forward and consistent with what Bracket and many others working on these programs are already implementing. But as the authors point out, using purely statistical approaches to identify data fabrication and for that matter other data quality concerns in subjective outcome measures may not be sufficient. At Bracket, we utilize a multi-faceted approach to the problem, combining targeted statistical analysis with various technologies such as audio/video recordings of the assessments or computer generated scores and comprehensive clinical review. Bracket has successfully implemented this methodology in a large number of global studies, closely monitoring clinical outcome data, in most cases with subjective endpoints.

Another important approach to risk mitigation is prevention. Using intelligent Electronic Clinical Outcome Assessment tools (eCOA) and predictive analytics to identify potentially problematic data before the subject gets randomized into the trial appears to be a viable approach (Kott & Daniel, 2016.) And when an intelligent eCOA is implemented carefully in a difficult study, data quality can be significantly increased (Miller and Feaster, 2015.)

Cluepoints-LogoTransCelerate has earlier recommended the use of a Risk Assessment & Categorization Tool (RACT) when implementing CSM. In addition to using clinical oversight and eCOA tools to ensure good data, Bracket has partnered with CluePoints to utilize their Central Monitoring Platform and RACT for sponsors who would like to be as vigilant as possible in protecting their study data.

Many of the findings of the TransCelerate initiative are cautious. The authors write:

Statistical monitoring can be used to identify data anomalies suggestive of fabrication, noncompliance, or other nonrandom errors that need further investigation or monitoring, and should balance sensitivity against the cost of investigating false-positive sites

False positive findings can be worrisome. At Bracket, this is managed through both careful system design, which allows for these to be tracked, and through careful clinical oversight. All of our monitoring findings are reviewed by clinicians for veracity and impact before they are disseminated to a sponsor, a CRA, or an investigator for remediation. This allows for the “human touch” on every finding, and hopefully minimizes the burden of any possible false positives, especially on the sites who are working so closely with patients.

Many of the approaches to addressing these issues have evolved quickly in recent years. Bracket, along with all of the stakeholders involved in improving these processes, will continue to be flexible in how we design these programs, and consistent in our constant evaluation of the outcomes. Understanding how these interventions are working, and continuing to publish our findings from these programs, should lead to more success in future programs.

FULL CITATION: Statistical Monitoring in Clinical Trials Best Practices for Detecting Data Anomalies Suggestive of Fabrication or Misconduct. 2016 Feb 4.
JOURNAL: Therapeutic Innovation & Regulatory Science. vol. 50 no. 2 144-154
AUTHORS: Min Lin, PhD; Shiowjen Lee, PhD; Boguang Zhen, PhD; John Scott, PhD; Amelia Horne, DrPH; Ghideon Solomon, PhD; Estelle Russek-Cohen, PhD
YEAR: 2016