In this project, we use a switching hidden Markov model (EMSHMM) approach to analyze eye movement data in cognitive tasks involving cognitive state changes. We used a switching hidden Markov model (SHMM) to capture a participant’s cognitive state transitions during the task, with eye movement patterns during each cognitive state being summarized using a regular HMM.
We applied EMSHMM to a face preference decision-making task with two pre-assumed cognitive states—exploration and preference-biased periods—and we discovered two common eye movement patterns through clustering the cognitive state transitions. One pattern showed both a later transition from the exploration to the preference-biased cognitive state and a stronger tendency to look at the preferred stimulus at the end, and was associated with higher decision inference accuracy at the end; the other pattern entered the preference-biased cognitive state earlier, leading to earlier above-chance inference accuracy in a trial but lower inference accuracy at the end.
gaze cascade plot
preference prediction accuracy
This finding was not revealed by any other method. As compared with our previous HMM method, which assumes no cognitive state change (i.e., EMHMM), EMSHMM captured eye movement behavior in the task better, resulting in higher decision inference accuracy. Thus, EMSHMM reveals and provides quantitative measures of individual differences in cognitive behavior/style, making a significant impact on the use of eyetracking to study cognitive behavior across disciplines.
Downloads
This is the MATLAB toolbox for eye movement analysis with switching HMMs (EMSHMM). It uses a few functions from the EMHMM toolbox, which is included.
- Files: EMSHMM toolbox v0.1 (66MB).
- If you use this toolbox please cite:
Eye movement analysis with switching hidden Markov models.
,
Behavior Research Methods, 52:1026-1043, June 2020. [appendix]
Publications
- Eye movement analysis with switching hidden Markov models.
,
Behavior Research Methods, 52:1026-1043, June 2020. [appendix] - Mind reading: Discovering individual preferences from eye movements using switching hidden Markov models.
,
In: The Annual Meeting of the Cognitive Science Society (CogSci), Philadelphia, Aug 2016.
Links
- EMHMM toolbox – eye movement analysis with hidden Markov models
Acknowledgements
We are grateful to the Research Grant Council of Hong Kong (project 17609117 to J.H.H. and CityU 110513 to A.B.C.) and to JST.CREST (to S.S.).