Insights
11 June 2020
Remote cognitive testing: what are the scientific and operational considerations?
Remote testing solutions most commonly involve participants completing assessments via an online platform in the comfort of their home, on their own device. Conducting trials in this way can often improve study efficiency by reducing the time and costs required to collect large volumes of reliable data. Most importantly, web-based assessments significantly reduce the participation burden for patients versus travelling to site. Here we will discuss some of the scientific and operational factors that should be considered when deciding on the suitability of using remote cognitive testing solutions to conduct a trial.
Is remote testing suitable for all study populations?
Remote testing supports patients to engage in research from the comfort of their own home and allows researchers to collect valuable data from participants who otherwise may not be able to partake in in-clinic testing. However, with less face-to-face interaction how can we best guide patients through their participation? For many patients voice-over instructions are sufficiently helpful to enable their participation. However, for the very young or those living with significant cognitive impairment, remote administration could lead to unreliable results. For some patients, the following activities can help to overcome this barrier to web-based assessment:
- Completing an initial testing session in clinic to familiarise participants with the testing procedures
- Providing support for task administration via video call
Can the same cognitive assessments and endpoints be used for web-based visits compared to those conducted in-clinic?
When deciding whether it is feasible to conduct testing remotely, the suitability of the cognitive test battery should be considered carefully. Computerised cognitive assessment technologies like CANTAB collect cognitive processing speed measures, which offer increased sensitivity to detect cognitive dysfunction or improvements compared to traditional paper and pencil cognitive tests. However, these latency measures require the technology to detect millisecond differences in response speed and when participants use their own devices to conduct cognitive assessments there is the risk of inter-device variability for latency measures. For this reason, cognitive tasks with a reaction time component should be avoided where possible in web-based testing and non-latency endpoints should always be chosen for analysis. However, to mediate this issue, researchers may also choose to provision devices for participants allowing for the collection of precise latency measures by removing this inter-device variability.
Keeping patient burden as low as possible promotes patient engagement and compliance with the testing protocol. To support with this, provisions should be put in place to support participants to comply with study protocols and complete assessments on the right day and at the right time. Technologies such as in-built reminders sent by text or email and device alarms can help improve these adherence rates [1].
Is cognitive data collected remotely as valid and reliable as data collected in-clinic?
A cognitive assessment conducted using the same hardware and software combination should be equivalent in the clinic and the home. However, without in-person supervision it can be difficult to ascertain the participant’s behaviour and their motivation to complete the assessments.
To overcome this issue, some cognitive testing technologies including CANTAB solutions, allow for in-built participant compliance checks as well as additional subject authentication checks.
To investigate this issue further, we conducted a study of supervised in-person vs web-based testing in a home setting to examine participant behaviour and motivation in an unsupervised setting. The results showed no significant effect of platform or test setting on any of the individual key outcome measures individually. The full findings are available to view here.
Is collecting cognitive data remotely as secure as collecting data in-clinic?
Cybersecurity is often a pressing concern for both researchers and participants when using web-based testing solutions. However, electronic data protection is heavily regulated. For example, the EU General Data Protection Regulation (GDPR) was passed in 2018 to provide cohesive conventions for the protection of people’s data across the EU, and enforce large fines from companies who are non-compliant [2].
Although remote testing applications are as secure at storing data as in-clinic testing platforms, participants often have concerns about the security of their Patient Identifiable Information (PII). These concerns should be taken seriously, and the participant should have ample opportunity to discuss them with a study representative. Studies may choose to invite participants to site for an initial enrolment day to help resolve these concerns, or conduct phone or video chats.
Many web-based testing platforms, including CANTAB solutions, provide in-built features to obtain informed consent when an initial on-site visit is not feasible.
What are the operational factors that should be considered when conducting a remote study?
So patients are using a device that is already integrated into their day-to-day life, it is becoming increasingly popular for studies to employ a ‘Bring Your Own Device’ (BYOD) policy whereby participants are given the software to complete the assessments from their own personal device.
Most web-based testing applications require internet connectivity to either complete the cognitive tasks or to upload results to site once the testing has been completed. Often participants can have difficulty completing sessions and adhering to the study protocol when they do not have a reliable internet connection so implementing screening for this during enrolment should also be considered.
Summary
Web-based testing solutions have the potential to provide high-quality, reliable study data without the need for participants to visit a clinic site. However, these studies are fundamentally different in design to in-clinic studies and therefore provisions must be put in place to ensure the successful implementation and outcomes from these types of trials.
To find out more about Cambridge Cognition’s remote testing solutions:
References
[1] Cormack F, McCue M, Taptiklis N, Skirrow C, Glazer E, Panagopoulos E, van Schaik TA, Fehnert B, King J, Barnett JH. Wearable Technology for High-Frequency Cognitive and Mood Assessment in Major Depressive Disorder: Longitudinal Observational Study. JMIR Ment Health 2019;6(11):e12814
[2] The European Parliament and The Council of the European Union. REGULATION (EU) 2016/ 679 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL – of 27 April 2016 – on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data, and Repealing Directive 95/ 46/ EC (General Data Protection Regulation).; 2016. http://ec.europa.eu/justice/data-protection/reform/files/regulation_oj_en.pdf. Accessed January 16, 2018
Author:
