关键词: 人机交互, 大语言模型, 视觉语言导航, 智能制造, 工业5.0

Abstract: In the manufacturing field,the wide application of mobile robots has become the key to improving operational safety and efficiency.However,most existing robotic systems can only complete predefined navigation tasks,and cannot be adapted to the unstructured environment.To overcome this bottleneck,an interactive navigation method for mobile inspection robots based on large language models was introduced,which replaced operators in conducting inspections within hazardous industrial areas,and to execute complex navigation tasks based on verbal instructions.The High-Resolution Net (HRNet) model was utilized for semantic scene segmentation,integrating the segmentation results into the reconstructed 3D scene mesh during the point cloud fusion phase to create a comprehensive 3D semantic map.A large language model was used to make the robot comprehend human natural language instructions and generate Python code based on the 3D semantic map to complete navigation tasks.A series of experiments had been conducted to validate the effectiveness of the proposed system.

Key words: human-robot interaction, large language model, vision and language navigation, smart manufacturing, Industry 5.0