Abstract: Aiming at the outbound container space assignment problem in automated container terminals,considering the characteristics of three-dimensional stacking,yard crane and multi-area cooperation,and a mathematical programming model was established to reduce task imbalance and container flip rate.To improve the quality of the solution,a reinforcement learning based hyper-heuristic was developed,which used several heuristic and intelligent algorithms with different characteristics as low-level heuristics,a novel policy-based reinforcement learning as high-level selection strategy and deep learning to extract hidden patterns more efficiently.Based on actual data of Yangshan Phase IV Automated Container Yard,numerical experiments were conducted.The results showed that the proposed algorithm could effectively solve this problem and was superior to state-of-art intelligent algorithms,which could improve work efficiency and provide decision support for automated container yard.

Key words: automated container terminal, container space assignment problem, reinforcement learning, hyper-heuristic algorithm

摘要： 针对实际场景下的自动化码头集装箱堆场出口箱箱位分配问题,考虑到三维码放、场桥接力和多箱区协同等特性,并以降低任务的不均衡性和后续的翻箱率为优化目标,构建了自动化码头出口箱箱位分配数学规划模型。为了提高求解质量,开发了一种基于强化学习的超启发式方法,该方法将具有不同特征的启发式算法和智能算法作为低层启发式策略,采用新颖的基于策略的强化学习方法作为高层决策方法,并使用深度学习更高效地提取状态中的隐藏模式。最后,根据洋山四期自动化集装箱堆场历史数据设计了算例,并将所提算法与常规智能算法进行了对比,证明了所提算法的有效性和优越性,同时表明所提算法能够提高堆场作业效率,为自动化集装箱堆场提供决策支持。

关键词: 自动化码头, 箱位分配, 强化学习, 超启发式算法