计算机集成制造系统 ›› 2024, Vol. 30 ›› Issue (5): 1668-1682.DOI: 10.13196/j.cims.2023.HI12

• • 上一篇    下一篇

基于隐式意图脑电解码的人机交互多任务建模研究

苗秀1,2,侯文军3,4,5+   

  1. 1.北京邮电大学现代邮政学院(自动化学院)
    2.内蒙古科技大学建筑与艺术设计学院
    3.北京邮电大学数字媒体与设计艺术学院
    4.北京邮电大学网络系统与网络文化北京市重点实验室
    5.北京邮电大学交互技术与体验系统文化和旅游部重点实验室
  • 出版日期:2024-05-31 发布日期:2024-06-12
  • 作者简介:苗秀(1989-),女,内蒙古包头人,北京邮电大学博士研究生,内蒙古科技大学副教授,硕士生导师,研究方向:人机交互、脑机接口,E-mail:18547290825@163.com; +侯文军(1963-),女,山西太原人,教授,博士,博士生导师,研究方向:人机交互、知识图谱,通讯作者,E-mail:hou1505@163.com。
  • 通讯作者简介:侯文军(1963-),女,山西太原人,教授,博士,博士生导师,研究方向:人机交互、知识图谱,通讯作者,E-mail:hou1505@163.com
  • 基金资助:
    2022年度内蒙古党委、内蒙古自治区哲学社会科学规划资助项目(2022NDB244);2023年度内蒙古自治区直属高校基本科研业务费资助项目(2023XKJX026)。

Human-computer interaction multi-task modeling based on implicit intent EEG decoding

MIAO Xiu1,2,HOU Wenjun3,4,5+   

  1. 1.School of Modern Post(School of Automation),Beijing University of Posts and Telecommunications
    2.School of Architecture and Artistic Design,Inner Mongolia University of Science and Technology
    3.School of Digital Media & Design Arts,Beijing University of Posts and Telecommunications
    4.Beijing Key Laboratory of Network System and Network Culture,Beijing University of Posts and Telecommunications
    5.Key Laboratory of Interactive Technology and Experience System,Ministry of Culture and Tourism,Beijing University of Posts and Telecommunications
  • Online:2024-05-31 Published:2024-06-12
  • Supported by:
    Project supported by the Inner Mongolia Party Committee,Philosophy and Social Science Planning of Inner Mongolia Autonomous Region in 2022,China(No.2022NDB244),and the Basic Research Funds for Universities Directly under the Inner Mongolia Autonomous Region in 2023,China(No.2023XKJX026).

摘要: 在短期内具有完全自主水平的机器智能无法实现的情况下,人仍是人机系统的重要组成部分。智能系统感觉并预测用户的意图,有助于实现人机之间自然动态地协同,提高人机系统安全和效率。然而,人机交互的过程中充斥着大量模糊、隐蔽的隐式意图,传统的心理或行为分析方法解析意图无法保证时效性和准确性。随着传感技术发展,基于生理信号识别用户意图成为主流方法,但现有关于隐式意图的研究存在可分模式少、识别精度低、面向领域实境研究不足等问题。面向工控复杂系统领域,在生理视域下基于被动脑机接口技术,提出一种将人自然地纳入到智能人机系统回路中的方法。首先,抽取操作者与工业软件交互过程中的典型任务,在多属性任务组(MATB)多任务范式基础上设计意图生理信号诱发实验程序;接着,采用共空间模式(CSP)算法提取多任务意图脑电空域特征;最后,通过被试间交叉验证和5折参数寻优,构建机器学习意图模型,实现对隐式交互意图的自动识别。研究发现,相比传统特征提取方法,基于改良后的多分类CSP算法对意图特征提取更为有效,结合三维空间特征可视化进一步证实了特征的有效性;脑电信号能够作为判断人机交互隐式意图的依据,CSP+SVM算法模型能够有效提高人机交互隐式意图脑电的解码性能。对隐式意图信息的转译对构建基于意图的高效人机交互模型,以及发展人机系统、提高人机协作效率具有重要意义。

关键词: 隐式意图, 人机交互, 电解码, 多任务建模

Abstract: In the short term with fully autonomous level of machine intelligence can not be achieved,human are still an important part of the Human Computer Interaction (HCI) systems.Intelligent systems should be able to "feel" and "predict" human intentions to achieve multi-channel natural and dynamic collaboration between human and machine,which is very important to improve the safety and efficiency of HCI system.However,the process of HCI is filled with a lot of fuzzy and hidden implicit intentions,and the analysis efficiency and accuracy using traditional psychological or behavioral analysis methods are poor.With the development of sensing technology,the intention recognition based on physiological signals has become the main method,but the existing research basis has some problems,such as few separable modes,low recognition accuracy and insufficient research on domain context.A method to integrate human into human-machine coordination loop naturally based on passive brain-computer interface technology was presented in the field of industrial control complex system.The typical interactive tasks of Human-Machine Interaction (HMI) were extracted based on Multi-Attribute Task Battery (MATB) multi-task paradigm.Then,the common space pattern algorithm was used to extract the Electroencephalogram(EEG) spatial features of multi-task intention.A machine learning intention model was constructed to decode implicit intention EEG and realize automatic recognition of implicit intention through subject-wise cross-validation and 5-fold parameter optimization.It was proved that EEG signals could be used as the basis for judging the implicit intention of HCI,and  CSP+SVM algorithm model could effectively improve the EEG decoding performance of implicit intention.The translation of implicit intent information was of significance for the study of intent-based efficient HCI model,the development of HCI systems and the improvement of human-machine collaboration efficiency.

Key words: implicit intention, human-computer interaction, electroencephalogram decoding, multi-task modeling

中图分类号: