Reinforcement Learning for Freight Booking Control Problems
Abstract
Booking control problems are sequential decisionmaking problems that occur in the domain of revenue management. More precisely, freight booking control focuses on the problem of deciding to accept or reject bookings: given a limited capacity, accept a booking request or reject it to reserve capacity for future bookings with potentially higher revenue. This problem can be formulated as a finitehorizon stochastic dynamic program, where accepting a set of requests results in a profit at the end of the booking period that depends on the cost of fulfilling the accepted bookings. For many freight applications, the cost of fulfilling requests is obtained by solving an operational decisionmaking problem, which often requires the solutions to mixedinteger linear programs. Routinely solving such operational problems when deploying reinforcement learning algorithms may be too time consuming. The majority of booking control policies are obtained by solving problemspecific mathematical programming relaxations that are often nontrivial to generalize to new problems and, in some cases, provide quite crude approximations. In this work, we propose a twophase approach: we first train a supervised learning model to predict the objective of the operational problem, and then we deploy the model within reinforcement learning algorithms to compute control policies. This approach is general: it can be used every time the objective function of the endofhorizon operational problem can be predicted, and it is particularly suitable to those cases where such problems are computationally hard. Furthermore, it allows one to leverage the recent advances in reinforcement learning as routinely solving the operational problem is replaced with a single prediction. Our methodology is evaluated on two booking control problems in the literature, namely, distributional logistics and airline cargo management.
 Publication:

arXiv eprints
 Pub Date:
 January 2021
 arXiv:
 arXiv:2102.00092
 Bibcode:
 2021arXiv210200092D
 Keywords:

 Mathematics  Optimization and Control;
 Computer Science  Machine Learning