Data Quality and Technology in Clinical Trials USA

Feb 21, 2017 - Feb 22, 2017, Philadelphia

Improving quality and reducing timelines in clinical trials through the use of technology and analytics

Quality Risk-Based Management by Design

We spoke to Brian Nugent of Gilead Sciences about an innovative new framework for risk-based management.



Clinical trials involve the safety of human subjects and affect the development and approval for market release for certain drugs. For this reason, quality and integrity of results is a top priority when designing clinical trials, and on-site monitoring is necessary to check if standard protocols are being followed. Sponsors also complement this with centralized monitoring where they receive and analyze clinical data remotely.

Despite these two approaches to monitoring, “Quality is a major concern,” says Brian Nugent, Associate Director, ClinOps, Process at Gilead Sciences. “The industry has far more processes and regulations now than there were maybe ten years ago. It is getting more and more difficult to get experienced folk [clinical operators]. We notice they make a lot of mistakes.”

Gilead adopts the Quality-by-Design (QbD) system where quality is the end in mind that is embedded into the entire system from objectives, processes and down to results. Quality can be achieved by addressing every single possible risk (any factor that poses a hazard to subject safety and/or data integrity and quality), but this could prove costly and inefficient. Nugent, who will be participating in the Conference for Data Quality & Technology in Clinical Trials, Philadelphia in April 2016, says, “Risk-based monitoring (RBM) is one way to control that.”

RBM allows clinical operators and sponsors to focus their monitoring efforts on clinical areas, sites or studies that need it the most based on their risk profiles. Gilead takes it a step further by adopting a unified Risk-Based Quality Management (RBQM) framework within clinical operations, which can help them identify, assess, control, review and communicate risks that potentially affect the quality of clinical results.

Adopting a structure for risk identification, analysis, and measurement

Nugent says RBM planning must be based on the most critical factors, such as key risk indicators (KRIs), key performance indicators (KPIs), key quality indicators (KQIs), reduced Source Document Verifications (SDVs) and other parameters essential in developing a structure for identifying, analyzing and measuring risk.

He suggests using QbD and other data tools to identify the areas to concentrate on. “We first need to identify the risk. At the start of a trial, we use a QbD tool and find out where our critical and high risk areas are,” Nugent says, “but what we really want to know is how we manage the risk. Do we assign, accept or control the risk?” At Gilead, they either accept or control risk, but they follow certain accepted risk tolerances, which they make sure are communicated down to all clinical sites.

According to Nugent, they also use a risk tolerance tool, which helps them avoid emphasizing areas they aren’t supposed to and failing to address those they should be concerned with, such as data integrity, subject safety, regulatory concerns, and oversight plans. “It’s really a way for us to focus on what’s important,” he says, explaining that this tool is what they use to uniformly analyze risk across the firm.

Leadership sets the risk tolerances for every study based on certain categories. “It takes only a few minutes and is part of the work flow of the tool itself,” shares Nugent. Afterwards, their study team inputs these into the risk tolerance tool to look at the uncontrolled risk and apply control if needed. The effects of such controls are then analyzed. When analyzing and measuring risk, the typical formula is multiplying impact of risk with likelihood and detectability, which pertains to whether they can detect a change. However, Nugent says change can be detected by having the right processes in place. So, instead of detectability, Gilead uses vulnerability, which Nugent says poses the questions: “Do we have the processes, experience and models in place? Are we vulnerable to this risk? Do we having nothing in place to detect and control this risk?” He observes that people understand the term vulnerability better. “Are we vulnerable? If yes, then we have a process that doesn’t work or no process in place at all. Of course, we are vulnerable. We have no way of detecting or controlling this.”

Managing risk-related information

One of the main facets of establishing risk tolerances is providing the ability to manage risk-related information. “There’s a tremendous amount of data that needs to be manipulated,” says Nugent. “We have to limit that data. Our data management group does a lot of work looking at the data and filtering it down for our clinical teams,” he says, explaining how in the past moving from central to physical monitoring usually meant an influx of data. “We couldn’t work that way because we were just flooding the team with information. So we backed up from that and we’ve been very cautious for the last two years.” With their new approach, Gilead provides clinical personnel with 10 to 12 indicators, which they find are more manageable for teams.

Quality control of risk mitigation

Nugent put together a Quality Control group tasked for ongoing maintenance of the QbD system and to ensure that quality is embedded. He cites the example of how it is a common practice for laboratories across the globe to fill out tubes with genetic sampling even when they don’t have the consent to do so. “Our vendors and labs won’t change,” he says. “So it comes down to monitoring. We can tell monitors to pull out those tubes, but we need to control by building it into the template and making sure that all unnecessary tubes be pulled and all sites be trained on every visit to not engage in unconsented processes or procedures.”

However, the QC arm does more than audit. For Nugent, the success of mitigation can be measured in terms of the effectivity of risk controls. QC monitors the processes that support risk and builds or tweaks them (either individually or comprehensively) for continuous improvement. “There are all those things that surround the processes and the processes themselves, which are the weak link,” he asserts. “If we don’t have a robust way to check these to make sure that everything supporting the clinical trial processes is functioning correctly, QC lets us know that the processes used for the controls and mitigation are working.”

Nugent also talks about residual risks, which are the risks that are seen after controls have been applied. They are a parameter that helps QC identify whether controls are effective or not. More specifically, they are given a range of whether risks are well controlled, over controlled, or under controlled, which helps them focus on what they need to do and how best to react. In this way, according to Nugent, “RBM helps us focus our energy on what is important. When you control a high risk factor, you can monitor it going forward. You do not need to spend too much time and energy when you put something effective in place.” He emphasizes that communicating residual risks upwards is critical because it also pairs with vulnerability in guiding their risk analysis.

Integration of QbD and RBM into culture

Achieving an optimally performing quality-focused RBM requires integration into the organization’s culture. Some of the drivers of this integration include effective recruitment, robust training, leadership, and culture change.

For Nugent, a successful mitigation model needs experienced clinical staff who have “been in the industry for five years or more, with a broad range of experience in starting up, maintenance, and closing up of trials, as well as the ability to understand the scientific aspect of trials. They understand regulations not only here in the United States but also globally.”

We want to be able to understand how do our clinical staff work? How can these tools best be part of their everyday life? How can we prevent redundancy? The results of my QbD have to automatically integrate into the risk tolerance tool. We have to integrate our key risk factors into our central documents.

Gilead also carries out several training programs, one of which is referred to as the ‘champion training,’ where key management are trained on critical functions and management tools, models, and methodologies. Across the organization, they also encourage employees to ask basic questions: “We’ve trained our staff to ask: What could go wrong?; What needs to go right?; How are you going to control this?; What if the risks become events?; and, How do I prioritize and control those risks?”

According to Nugent, they incentivize staff by influencing them: “We try to build expectations for RBM into our culture - not only for us, but also for our regulators in the US and globally. We incentivize staff with the expectation of the industry that we do this type of thing and help them understand that this is not only important but is something doable, which they can put into their own work balance.”

Nugent also observes how there are many tools for RBM discussed during conferences he attends, “but not a lot of talk on how to integrate those into the culture of the organization.” Gilead makes use of QbD tools to integrate RBM into the daily workflow. He says, “We want to be able to understand how do our clinical staff work? How can these tools best be part of their everyday life? How can we prevent redundancy? The results of my QbD have to automatically integrate into the risk tolerance tool. We have to integrate our key risk factors into our central documents.”

Integrating QbD and high risk indicators

Nugent further shares Gilead’s plans of developing a central dashboard for study management teams and executive leaders, where QbD results and high-risk indicators can be integrated. Such a dashboard will mean they are no longer required to look into multiple places to get a comprehensive view of risk.

When people ask Nugent how long the shift to the RBM approach will take, he always finds himself predicting five years, explaining that culture shift really takes a long time:“We have a 500-person group, and to get these people to shift the way they work, they’re not used to it. It slows them down to begin with.” He emphasizes the need for strong endorsement from leadership, saying,“You need to understand your organization, and to do it large scale without strong leadership means it would be doomed.”

However, with the integration of QbD tools and the RBM approach, Gilead has created a framework that has helped automate the mitigation of risk and protection of quality. Nugent says people have responded well in the organization, and it has made processes easier for them. A complete shift will take a long time. Nugent recognizes they are still discovering many aspects of their framework and how it functions in the organization and the industry as a whole, but it remains an effective method for integrating different quality and risk-assessment tools into their daily work, relationships with laboratories, and need for continuous improvement. Despite the regulatory changes in the industry, Nugent believes this framework is poised to help maintain quality and identify, control, and monitor risks more efficiently.


Brian Nugent will be presenting at Data Quality & Technology in Clinical Trials, April 18th-19th, 2016 in Philadelphia. 



Data Quality and Technology in Clinical Trials USA

Feb 21, 2017 - Feb 22, 2017, Philadelphia

Improving quality and reducing timelines in clinical trials through the use of technology and analytics