  The Bin Packing Problem (BPP) has attracted enthusiastic research interest
recently, owing to widespread applications in logistics and warehousing
environments. It is truly essential to optimize the bin packing to enable more
objects to be packed into boxes. Object packing order and placement strategy
are the two crucial optimization objectives of the BPP. However, existing
optimization methods for BPP, such as the genetic algorithm (GA), emerge as the
main issues in highly computational cost and relatively low accuracy, making it
difficult to implement in realistic scenarios. To well relieve the research
gaps, we present a novel optimization methodology of two-dimensional (2D)-BPP
and three-dimensional (3D)-BPP for objects with regular shapes via deep
reinforcement learning (DRL), maximizing the space utilization and minimizing
the usage number of boxes. First, an end-to-end DRL neural network constructed
by a modified Pointer Network consisting of an encoder, a decoder and an
attention module is proposed to achieve the optimal object packing order.
Second, conforming to the top-down operation mode, the placement strategy based
on a height map is used to arrange the ordered objects in the boxes, preventing
the objects from colliding with boxes and other objects in boxes. Third, the
reward and loss functions are defined as the indicators of the compactness,
pyramid, and usage number of boxes to conduct the training of the DRL neural
network based on an on-policy actor-critic framework. Finally, a series of
experiments are implemented to compare our method with conventional packing
methods, from which we conclude that our method outperforms these packing
methods in both packing accuracy and efficiency.
