Reinforcement Learning-Based Intelligent Agents for Improved Productivity in Container Vessel Berthing Applications

This chapter introduces the use of hybrid intelligent agents in a vessel berthing application. Vessel berthing in container terminals is regarded as a very complex, dynamic application, which requires autonomous decision-making capabilities to improve the productivity of the berths. In this chapter, the dynamic nature of the container vessel berthing system has been simulated with reinforcement learning theory, which essentially learns what to do by interaction with the environment. Other techniques, such as Belief-Desire-Intention (BDI) agent systems have also been implemented in many business applications. The chapter proposes a new hybrid agent model using an Adaptive Neuro Fuzzy Inference System (ANFIS), neural networks, and reinforcement learning methods to improve the reactive, proactive and intelligent behavior of generic BDI agents in a shipping application. IDEA GROUP PUBLISHING This paper appears in the book, Business Applications and Computational Intelligence edited by Kevin E. Voges and Nigel K. L. Pope © 2006, Idea Group Inc. 701 E. Chocolate Avenue, Suite 200, Hershey PA 17033-1240, USA Tel: 717/533-8845; Fax 717/533-8661; URL-http://www.idea-group.com ITB12002 156 Lokuge & Alahakoon Copyright © 2006, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited. Introduction Competition among container ports continues to increase as the worldwide container trade grows (Ryan, 1998). Managers in many container ports are trying to attract more vessel lines by automating the handling of equipment, and providing and speeding up various port-related services. One of the important applications in container terminals is the vessel berthing system, where system functionalities include the optimal allocation of berths to vessels, allocation of cranes, labor, and trucks of containers (loading and discharging) guaranteeing the high productivity of the container terminals. The research described in this chapter is motivated by a berth assignment problem faced by terminal operators in large container hub ports. It aims to investigate the possibility of using intelligent agents for the efficient management of vessel berthing operations. In a dynamic environment, a vessel berthing system is a very complex dynamic application system, which requires dealing with various uncertainties to assure improved productivity and efficiency in the container terminals. Numerous studies have been conducted in vessel and port-related operations in the past. Most of the research focuses on a static vessel berthing system, where the main issue is to find a good plan for assigning vessels. Brown, Lawphongpanich, and Thurman (1994) used an integer-programming model for assigning one berth to many vessels in a naval port. Operations and the dynamic nature of a container port are not considered in the vessel berthing program. Lim (1998) addressed the vessel planning problem with a fixed berthing time; Li, Cai, and Lee (1998) addressed the scheduling problem with a single processor and multiple jobs and assumed that vessels had already arrived; Chia, Lau, and Lim (1999) used an Ant Colony Optimization approach to solve the berthing system by minimizing the wharf length; Kim and Moon (2003) used simulated annealing in berth scheduling. We suggest that the use of experience with dynamic decisionmaking capabilities would help to ease the burden of operational complexities at container terminals. We argue that the application systems should always interact with the environment to observe changes at different time intervals and should react promptly by suggesting alternative solutions. These features would essentially improve the autonomous behavior of current vessel berthing and planning application systems. The BDI agent model is possibly the best known and best studied model of practical reasoning for implementations (Georgeff, Pell, Pollack, & Wooldridge, 1998), for example, IRMA (Bratman, Israel, & Pollack, 1998) and the PRS-like systems and dMARS. In some instances the criticism regarding the BDI model has been that it is not well suited to certain types of behavior. In particular, the basic BDI model appears to be inappropriate for building complex systems that must learn and adapt their behaviors. Such systems are becoming increasingly important for business applications. The hybrid BDI model suggested in this article discusses a new agent model to overcome some of the limitations of the generic BDI model. A hybrid-agent model for container terminals is introduced with only a few intelligent tools, such as neural networks and an adaptive neuro-fuzzy inference system (ANFIS). This greatly improves agent behavior in complex applications, such as a vessel berthing systems. Further, it enhances the capabilities of learning, social behavior, and adaptability in planning, especially in dynamically changing environments. 28 more pages are available in the full version of this document, which may be purchased using the "Add to Cart" button on the product's webpage: www.igi-global.com/chapter/reinforcement-learning-basedintelligent-agents/6024?camid=4v1 This title is available in InfoSci-Books, Business Intelligence, Business-Technology-Solution, InfoSci-Business Technologies, Business, Administration, and Management, InfoSci-Business and Management Information Science and Technology. Recommend this product to your librarian: www.igi-global.com/e-resources/libraryrecommendation/?id=1