For the clinical trial industry, adequate user acceptance testing (UAT) for computerized systems such as electronic Trial Master Files (eTMFs) is crucial for both regulatory approval and optimized study management. The validation process for an eTMF system can represent a heavy lift for vendors and sponsors alike, particularly as new features and functionality are introduced to respond to new regulation or clinical need.

Addressing this evolving complexity will require new UAT models that encourage greater collaboration, reach more end users, and alleviate some of the burden that accompanies UAT activities. Crowdsource UAT, a validation model that enables testing via real, external users, can offer several potential advantages for the clinical trial industry. While more traditional methods of UAT validation are likely to result in a steady increase in workload with each new upgrade cycle, crowdsourced validation can serve to expedite UAT through:

• objective evidence exchange;
• decreased individual workloads for participating sponsors;
• increased access to users;
• faster testing;
• more reliable flagging of issues and bugs.

This, coupled with the advantages users themselves can experience through the in-system training offered by testing, may help position organizations for a more streamlined, comprehensive UAT and technology deployment.

A New Frontier for UAT Validation

Despite its popularity in the broader tech space, crowdsource UAT is a tool that has gone unleveraged in the clinical trial industry. Yet establishing a codified, optimized crowdsource UAT model has the potential to transform eTMF validation, offering organizations crucial time and cost savings. This validation is not just important to ensure that these systems operate as intended – it is also necessary to remain compliant with regulators. Subpart C of the FDA’s CFR 21 Part 11 states that any executed validation for a computerized system should be based on a justified, documented risk assessment; this requirement is echoed by guidance issued under an ICH GCP addendum released in 2018. The EMA has likewise required that any qualification and validation activities conducted for computerized systems be based on a documented risk assessment.

IQVIA, a leading health information technology and clinical research company, has launched a crowdsource UAT model aimed at driving joint cross-client UAT activities during upgrade cycles. Its process for crowdsource validation differs from traditional system validation in a few key ways:

1. For the crowdsource process, participating clients utilize a single multi-client tenant, and the vendor leads UAT training for all testers.
2. Following the release and provision of all validation documentation associated with a crowdsource project, the vendor is responsible for assigning UAT scripts to each client for execution.
3. At the conclusion of the project, the vendor reviews, consolidates, and releases all of the objective evidence to all participating clients.

As part of its pilot, IQVIA standardized its training method and outputs to ensure its testers were executing the same requirements; as each test case or script was completed, IQVIA reviewers evaluated each to ensure that there were no errors or missing evidence. This evaluation was performed on an ongoing basis in order to maintain project timelines and circumvent the potential for multiple issues being flagged at the end of the process, compounding the potential for rework.

While the relative novelty of this model for the clinical trial industry did result in some hesitancy from clients, others were excited by the potential reductions in validation workload. Continuing to recruit clients to participate in future UAT validation activities is crucial to proving the value of the model for the industry as a whole. IQVIA’s long-term plan is to continue to build confidence in the model through increasing client buy-in, generating more client feedback, and continuing to scale, with a focus on achieving iterative improvements.

Building Confidence in Crowdsource Validation for the Clinical Trial Space

For clients reluctant to adopt a new model for UAT validation, their primary concerns are often tied to regulation – how can a sponsor, wholly responsible for the success of its validation activities, rely on outside data to achieve these requirements? Yet most regulatory language, such as that emanating from the EMA, notes that sponsors can rely on a vendor’s documents for computerized system qualification.

While a sponsor can (and should) review a vendor’s validation process to determine whether it is sufficient to meet requirements, the participating sponsors within a crowdsource validation exercise function in much the same way as third-party testers, essentially serving as outsourced testing and subject to the same rigor. Moreover, the framework that IQVIA provides for crowdsource UAT, combined with the quality of objective evidence made possible by its larger testing cohorts, make it a viable alternative to traditional testing methods.

Another concern for participants centers on confidentiality. As part of its pilot, IQVIA worked to obscure the identities of participants from one another, both during the exercise and when objective evidence exchange occurred. Because the model is vendor-led, IQVIA was able to ensure adequate “blinding” throughout; as the model progresses, IQVIA will likely develop client-to-client NDAs in order to further secure this confidentiality.

Although the crowdsource model has only been tested for eTMF platforms, it has more wide-ranging potential for the space, including for electronic data capture (EDC) platforms and other similar technologies. IQVIA plans to continue its testing with interested clients, refining its approach in order to achieve proof of concept for its model by demonstrating both the improved utility of crowdsource UAT and its potential for faster, more cost-effective validation.

To learn more about IQVIA’s eTMF platform visit iqvia.com/eTMF.