Memorability of Word-Pairs: Developing a Method for Generation of Calibrated Stimulus Sets for Repeat, Remote, Automated Testing

NeuroVocalix^TM uses text to speech (TTS) and automatic speech recognition (ASR) to deliver voice-based cognitive assessments across multiple platforms including smartphones, tablets, and traditional telephones. We have created a method for generating word-pair stimuli with well-understood memorability characteristics for use with NeuroVocalix^TM. We sampled 192 word-pairs from the British National Corpus to be used in a Verbal Paired Associates (VPA) task. We then used NeuroVocalixTM to run VPA with 188 English-speaking participants worldwide. We modelled task performance to develop a difficulty model using Logistic Mixed Effects, which we then in turn used to produce optimised pair-lists from a set of all possible pairs in our corpus. Using this method we have produced 12 sets of word-pairs optimised both for difficulty and for automated speech recognition use. Word-pair difficulty is well-understood and well-matched across sets to enable longitudinal data collection. This method has the potential to be used for automated production of other sets in the future, for example in different languages or for use with people whose vocabulary is more limited, such as young children.

Validating a set of Verbal-Paired Associates (VPA) Word-Pair lists for Repeat, Remote, Automated Testing

Having generated a set of 12 word-pair lists we are now validating these stimuli. Lists of word-pairs for use with VPA should be of a similar difficulty to one another to enable repeat, longitudinal testing. To be sensitive to a broad range of memory abilities, and to allow for declines or increases in performance over the course of a study, our aim for this set was that among healthy older adults, participants should on average recall half of the word-pairs at their first attempt, with a broad variation in performance. In this cross-sectional between-groups study, 181 older adults carried out automated VPA using NeuroVocalix^TM and we analysed their performance. Our results show a good spread of task performance, suggesting good task sensitivity. Overall, our results indicate that the 12 eight-pair lists elicit task performance within a desirable range, both at the individual list level and across the set of lists. This study supports release of this set of word-pair stimuli lists for use in NeuroVocalix^TM VPA.