RoboPath AI: Warehouse Robot Path Optimization using Tabular Q-Learning with Multi-Goal Task Decomposition

Thettu Mokshagna Theja, M.Gowthami

Reinforcement Learning; Q-Learning; Warehouse Automation; Path Optimization; Grid-Based Navigation; Multi-Q-Table Architecture.

The rapid growth of warehouse automation has increased the demand for intelligent robotic systems capable of efficient and safe navigation in dynamic environments. Traditional path-planning algorithms perform well in static conditions but are less effective in environments with moving obstacles and multi-step tasks. To address this challenge, this paper presents RoboPath AI, a reinforcement learning-based warehouse robot path optimization system using tabular Q-learning. The proposed system models the warehouse as a 12×12 grid environment where the robot learns to navigate, collect multiple target items, avoid obstacles and human workers, and return to its starting position. A structured reward function is designed to encourage efficient movement and safe navigation. To improve learning performance, a multi-Q-table architecture is implemented, enabling the agent to handle sequential subtasks effectively. An epsilon-greedy strategy is used to balance exploration and exploitation during training. The system also incorporates real-time visualization, policy representation, and reward analysis to monitor learning progress. Additionally, Optuna-based hyperparameter tuning is applied to optimize performance. Experimental results demonstrate that the agent successfully learns optimal navigation strategies over time, showing significant improvement in efficiency and convergence.

[1] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. MIT Press, 2018.
[2] C. J. C. H. Watkins and P. Dayan, “Q-Learning,” Machine Learning, vol. 8, no. 3–4, pp. 279–292, 1992.
[3] S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach. Pearson, 2021.
[4] V. Mnih et al., “Human-Level Control through Deep Reinforcement Learning,” Nature, vol. 518, pp. 529–533, 2015.
[5] D. Silver et al., “Mastering the Game of Go with Deep Neural Networks and Tree Search,” Nature, 2016.
[6] R. Bellman, Dynamic Programming. Princeton University Press, 1957.
[7] L. Busoniu et al., “Multi-Agent Reinforcement Learning: An Overview,” Innovations in Multi-Agent Systems, 2010.
[8] T. Schaul et al., “Prioritized Experience Replay,” ICLR, 2016.
[9] J. Schulman et al., “Proximal Policy Optimization Algorithms,” arXiv preprint, 2017.
[10] F. Hutter et al., “Sequential Model-Based Optimization for General Algorithm Configuration,” LION, 2011.
[11] T. Akiba et al., “Optuna: A Next-Generation Hyperparameter Optimization Framework,” KDD, 2019.
[12] M. L. Puterman, Markov Decision Processes. Wiley, 2014.
[13] S. Thrun et al., Probabilistic Robotics. MIT Press, 2005.
[14] P. Abbeel and A. Ng, “Apprenticeship Learning via Inverse Reinforcement Learning,” ICML, 2004.
[15] H. Kober et al., “Reinforcement Learning in Robotics: A Survey,” IJRR, 2013.