Alliance between agents and optimization in warehouse logistics

Nowadays, logistics has become an important part in the supply chain due to the effects of globalization. A warehouse is a large building where goods are stored, and where they may be shipped, catalogued, or received. When considering the level of effort involved in warehouse operations, the greatest expenditure of effort is in the picking process. In warehouse, the major goal is to improve the efficiency, accurancy and safety of warehouse operations. Supply chain managers focus to gain maximum efficiency for minimum cost. Therefore, Warehouse Management Systemd (WMS) first appeared on the market in the 1980’s and have been developed to handle warehouse resources and track and monitor warehouse real-time operations. There are many application of Radio Frequency Identification (RFID) that can be found embedded in items especially in supply chain management. Verification is one of the crucial issues of RFID technology. The purpose of this study is to demonstrate the feasibility of a location using the RFID as an active technology in warehouse management field. In this content, our objective is to combine multi-agent systems and optimization algorithms in order to localize the operator by integrating the indoor positioning system and then optimize operator roote during the picking mission. In this paper, we present an Optimizer Based Agent Communication of Operator Work plan in Warehouse (OBACO2W). Order pickers, using the pick by voice technique, receive and execute the orders sent by the WMS. The purpose of logistics is to establish efficient and satisfy customers’ requirement. So, we aim to enhance workers’ productivity by assigning to the order pickers the shortest path for the picking mission. We must first locate the picker adopting the RFID technology. Multi-agent modeling problem can be divided into skills and knowledge to autonomous entities called agents, to provide links and dialogue between these entities by means of communication and interaction. In a multi-agent system, the change of status of certain objects in the environment of change agents can affect their behavior and decisions. Thus, to optimize their choices and to guide their taken decisions, these agents can be equipped with optimization approaches suited to their skills and knowledge. Hence the alliance between the multiagent systems and optimization methods; these two approaches are perfectly complementary.


INTRODUCTION
There are different types of geolocation technology such as the Global Positioning System (GPS), positioning systems via cell phone, Wi-Fi, RFID, etc. All these technologies have different applications and limitations. The indoor location is currently a very open field. Among many logistics technologies, RFID has been identified as an important one to improve logistics operations and supply chain management [1]. The contribution of this work is the combination of optimization algorithms with multi-agent system while integrating RFID technology in logistic field. The RFID technology is used in many applications including manufacturing and distribution of products [2] [3], Business-to-Business (B2B) logistics, Business-to-consumer (B2C) marketing and aftersales service [4]. Due to globalization, logistics has become a strategic factor in creating competitive advantage in supply chains [1]. French Logistic Association (ASLOG) defines logistic as "the set of activities that they intend to place to the minimum cost a guaranteed amount of product in the place and moment that is demanded". "A warehouse is a storage facility that receives goods and products for eventual distribution to consumers or other businesses. A warehouse can also be called a distribution center; warehouse management is the process of coordination of incoming goods, subsequent storage and cargo tracking, and finally, the distribution of goods to their appropriate destinations" [5]. WMS are available from the first computer systems where they ensured simple features for the storage location. Nowadays, WMS can be autonomous. They may include a complex technology like RFID and Voice Recognition. The basic principle of WMS is providing information to allow effective control of the movement of materials within the warehouse. In order picking tasks, workers collect sets of items from an assortment in a warehouse according to a work order. They then deliver them to the next station in a precisely designed material flow process [6]. Experiences from practice show that about a half of the total operating expenses of a warehouse is spent by order picking [7]. This is caused by the complexity and the labor intensity of order picking processes [8]. In previous research, Yan et al [9] worked on the combination of WMS and RFID technology in order to improve the competitive power of enterprises and the efficiency of the supply chain. Taljanovic et al [10] worked also on the logistic based on multi-agent system in order to reduce cost and time to process a wave by reducing distance traveled by order picker, retrieval time per item and replenishment costs. Wang [11] studied on logistic system based on multi-agent. He worked on the problem of logistics supply chain coordination based on the multi agent system in order to effectively process distributed large scale data. Cossentino et al [12] proposed a multi agent system which is used in a real environment within a project developed with a company working on logistics. Multi-agent systems are used when we are confronted to different organizations with different goals and proprietary information in order to handle their interactions. In general, the reasons of use of multi-agents systems are parallelism, robustness, scalability etc. In this work, we integrate an optimization approach in our multi-agent system. This paper is organised as follows: Section 2 discusses the use of multi-agent system in the literature and proposed the current research works in RFID. Section 3 details a description of the problem. Section 4 describes the multi-agent architecture. We present the OA behavior in section 5. Section 6 illustrates our approach through different simulations result. Finally, section 7 gives a conclusion to this paper and outlines the future research.

Multi-agent system
Multi-agent systems are now a new technology for the design and control of complex systems. A multi-agent system is a system composed of autonomous hardware or software entities called agents. The multi-agent approach is based on several theories and concepts that have their roots in several disciplines such as sociology, psychology, distributed systems, software engineering. Several research works were based on multiagent systems in manufacturing and supply chain. Lorena et al [13] worked on the disturbances that threaten the supply chain. The goal of their work was to enable the supply chain to respond to disruptive events to minimize their impact using an agent-based system for Supply Chain Event Management (SCEM) architecture. Lim et al [14] presented an iterative agent bidding mechanism, which performs dynamic integration of process planning and production. Ghiassi et al [15] proposed agent-based techniques to coordinate the activities of internet-based supply chain system for mass customization markets. Li and Fong [16] proposed agent-based architectures to facilitate the formation and organization of virtual enterprises for order management.

Radio-frequency Identification technology
Radio Frequency Identification, known as RFID, is an automatic identification technology which was available in the late 1960's and raised in the 1990's. It uses non-contact two-way radio communication means to achieve the purpose of recognition. The RFID technology is an increasingly popular technology for tracking and tracing goods in the supply chain activities in relation to logistics, warehousing and manufacturing. The RFID has been widely applied in different industry sectors in the domains such as logistics, warehousing, manufacturing, security, counterfeit, patient safety, etc. Manufacturing is one of the major RFID application fields [17]. There are a number of research works focusing on improving production flow. Scholz-Reiter et al [18] used RFID to support Kanban system for the production industries, Guo et al [19] proposed an intelligent production control decision F e b r u a r y , 2 0 1 4 support system that consists of production routing rules to improve assembly line. Chen and Tu [20] proposed an agent-based framework to respond to RFID events in real time to better manage production flows and improve the traceability and visibility of manufacturing processes, and Chen et al [21] introduced a RFID framework to smoothen mass customization production flow. In logistics, Royo et al [22] presented an implementation of an RFID system which aims to determine the accuracy of the measurements obtained in a loading dock by contrasting it with the known information of the goods transported. Trappey et al [23] proposed an Genetic algorithm dynamic performance evaluation for RFID reverse logistic management.
Hellstrom and Johansson [24] examined the impact of different control strategies on the returnable transport items with a simulation model, Ola and Henrik [25] and Lin [26] conducted surveys to assess the use of RFID and its impact on logistics performance. Lee and Chan [27] developed an efficient reverse logistics system with RFID incorporated. In warehousing, an empirical study was conducted to evaluate the efficiency of RFID and the implementation strategy was proposed to promote RFID adoption. Poon et al [28] proposed a radio frequency case-based logistics resource management system (R-LRMS) for formulating and suggesting the appropriate material handling solutions in a warehouse environment. Several studies have combined the multi-agent architecture and RFID technology in logistic's field. Lim et al [17] proposed a multi agent system, incorporating RFID technology, which aim to facilitate and add value to operations across the supply chain. Zhang et al [29] introduced an agent-based workflow management to facilitate interactions between RFID-enabled manufacturing resources. Chow et al [30] proposed a dynamic logistics process knowledge based on an RFID multi-agent approach. Lockman et al [31] presents a multi-agent architecture for verification and validation of RFID architecture taxonomy based on tree-node diagrams starting with the top tier and detailed branch structures.

DESCRIPTION OF THE PROBLEM
Order picking consists on collecting set of items stored in the warehouse and regrouping them before delivering to customers. Optimizing this functionality increases productivity and improves the quality of service. "Traditionally, order picking is accomplished by providing workers with printed-out pick lists of articles, describing their position in the warehouse, the amount to be collected, and short descriptions. To increase efficiency and to reduce the number of picking errors, industrial setups start using two techniques: Pick-by-Light and Pick-by-Voice systems" [32]. Order pickers receive order from the WMS. This order contains the order name, the order number, the items number, location and name of each item. WMS send the order and assign it to the first available picker. Actually, there are two major problems: a) WMS assigns randomly the order to the first available picker without taking into account its location in the warehouse. b) The picker accomplishes the picking mission respecting the order of items sent by the WMS. The location of items is not optimized. So, picker may spend more time. Delivery may be delayed causing a loss of customers and money.
To overcome these problems, we propose an architecture based on the alliance between the multi-agent system and optimization approaches. The solution aims to minimize operator path. This optimization will be done on two levels. First, we assign the order to the nearest picker and then we find the shortest path for the picking mission. For the first level, we establish a study that estimate the picker position using the RFID technology in order to locate the operator and assign to it the nearest order. Figure 1 represents a general idea of the proposed solution.

PROPOSED AGENT BASED ARCHITECTURE
A multi-agent system is a combination of different agents which work in collaboration pursuing assigned tasks to achieve the overall goal of the system and the focus of intense attention in many fields in computer science and artificial intelligence [33]. It has become an increasingly powerful tool in developing complex systems that takes advantages of agent properties: autonomy, sociality, reactivity and pro-activity [34]. Ferber [35] defines an agent as an entity that can perceive its environment and reacts to these perceptions.

System architecture
To resolve the problem described previously, we propose a system based on the coordination of five kinds of software agents. The architecture of the multi-agent system is presented in Figure 2.   Optimizer Agents (OA): the OA receives the order assigned to a resource from the IdA. For this, the OA assigns an order for each Mobile Agent (MA) described below according to its location in warehouse. The localization of MA is determinated by RFID technology. The shortest path is assigned to the MA in order to avoid traveling kilometers and wasting time. The OA receives confirmation from MA once this later is in the right position for picking ( Figure 5).

Figure 5. Optimizer Agent behavior
 Mobile Agents (MA): the MA is the software representation of the order picker. It patrols warehouse and confirms to OA that it is in the right position when it is in front of the rack. Once the picking is achieved, the MA sends a message to the AMA described below informing him the status of the order ( Figure 6).

Communication protocol
The IA sends an order to the IdA which verifies resources and assigns them to adequate order. The order affected to the resource (palet, trolley..) is sent to OA which localizes MA using RFID technology. F e b r u a r y , 2 0 1 4 Figure 8. Sequence diagram MA receives the order affected to the resource and undertakes the picking missing. It sends the order status to AMA. If the mission is successfully completed, the AMA sends an answer to IA. Else, the order is restarted by sending it to IdA. Figure 8 below represents a sequence diagram in the UML 1 . It represents the communication protocol of our agent based system.

THE BEHAVIOR OF the OA
The purpose of this work is to improve order picker's work by offering them the adequate condition which allows them to improve the quality of services and increase productivity. The OA assigns the order to the nearest picker. The proposed optimization algorithm is integrated in the behavior of the OA to localize the MA and then find the shortest path for the picking mission.

The system location system features
The model developed here presents some hypothesis that should be verified in a real implementation:  The first of these hypothesis is the morphology of the modeling area. We consider that there are no walls blocking the waves emission.
 The second concerns the equipment used for the RFID tags. We consider that the RFID reader worn by the picker transmits with the same power omnidirectional. This power allows it to broadcast to 6m.
 The third concerns the RFID tags. The tags represents the items in the warehouse. We consider that when they capture a signal from a reader, the reader gets to know the strength of the signal returned by the tag and that this force depends on the distance and a white noise. In addition, we consider that the location of the tags is known.


The latter hypothesis is the fact that we consider the plane problem, i.e. we consider that the tags are at the same height as the RFID reader.

The positioning algorithm
After receiving the order from the IdA, the OA assigns a picking mission to each MA depending on its location in warehouse. The localization of these MA is done thanks to the RFID technology.
Knowledge of distances to tags is not equivalent to the knowledge of the position of the picker. It is therefore necessary to develop an algorithm to estimate the position. The proposed algorithm is not based on the triangulation, trilateration or others. It is based on the algorithm of gradient descent 2 [36], innovative approach to the localization.


Operation of the algorithm: The operation of the algorithm is illustrated in Figure 9. First, a starting point is generated. The distances between the picker (RFID reader) and tags are then calculated. The estimated position is moved to the direction given by the straight line formed by the picker and the tag for which the difference between the acquired distance and the distance to the estimated position is less (gradient algorithm). The algorithm repeats these steps until the termination criterion in the case of a static position or repeats endlessly.

Equation 1
Where ( , ) and ( , ) correspond respectively to the initial and the end points of the model area.

Calculating the distances (2):
The distance between the RFID reader (the picker) and a tag is calculated using a relation between the strength of the received signal (RSSI 3 ) and the distance because, in practice, we have access to RSSI and not directly to the distance. To obtain this relation, we proceed by "fingerprint", i.e. we create a large experimental database linking the RSSI to the distance in order to create a "Wireless Signal Attenuation Curve", a curve of signal attenuation. So it depends on the place of use of the technology.
From this curve, we can, by acquiring the RSSI for each tag, calculate the distance to each of these tags.
Method Calculating_the_distances (picker i) CONSTANT: WASC -1 2 Gradient descent methode called also method of steepest descent is an algorithm for finding the nearest local minimum of a function which presupposes that the gradient of the function can be computed. 3      The adjustment values in equation 4 ( (t), (t)) are the products of the adjustment rate ( , ) determined by experimentation, estimated at time (t) and position of the gradient as shown in equation 5. The gradient corresponds to the gradient of the tag k for which is maximum can be calculated using Equation 6. ̅̅̅̅ .
Estimating the position is gradually adjusted using these equations. The average error of the position of the picker i for the n tags captured by the RFID reader may be calculated by the method of least squares according to equation 7. Termination of the algorithm: The terminal condition of the algorithm can be a number of iterations or (and) an average error less than a certain threshold. Defining a number of iterations ensures completion of the algorithm in a short time while defining a threshold for the average error is used to ensure precision in the location of the picker. In the case of a dynamic positioning (our case), the algorithm must run the same loop even if it reaches zero error since the picker can move and thus generate an error. The operators are located in real time by a geolocation system using the RFID. The emplacement of different items and pickers are represented in Figure 10. The optimization approach is applied on two steps: First, OA finds the best picker and the first item. Then, the first item is used as a source to apply Dijkstra's algorithm in order to resolve the shortest path problem. The distance between an operator and an item allows fixing the best operator and the first item. It is calculated as follows:

Integrated behavior algorithm
 Optimization_displacement_operator: The handling of the order is in function of time. If we are in the beginning of the workday, the pickers connect to the WMS. The assignment of the order to a picker is random. F e b r u a r y , 2 0 1 4 However, we must find the nearest item to assigned picker. From this item, we calculate the shortest path using Dijkstra algorithm. Else if the picker is on the yard with an empty resource and his status is available then we find the best picker and we apply Dijkstra algorithm to assign to the picker optimal order picking route.    First_item: This function calculates the distance between the picker p and the items i of the order o and returns the distance and the index of the nearest item of the picker p. In the beginning of the workday, all pickers are at the same position. So in this case, the assignment will be random. Then, we must find the first item in order to apply Dijkstra algorithm. "Item" is a structure representing the distance and the index of an item. F e b r u a r y , 2 0 1 4

Modified Dijkstra algorithm
Dijkstra's algorithm has a very wide range of applications, such as: multi-point routing [37], surveying and mapping science [38], the shortest path of logistics and transport [39], the intelligent transportation system [40], the expressway network toll collection [41]. In this work, OA assigns the order to the nearest MA which represents the picker. The Dijkstra's algorithm is applied once the first item is determined to find the shortest path between items of the order. The different items are represented in a graph with non-negative edge path costs. MAs patrol in warehouse and follow a work plan fixed by the modified Dijkstra algorithm.

SIMULATION RESULTS
We are developing our system, with JADE platform (Java Agent Development framework)*. JADE is a middleware which permits a flexible implementation of multi-agent systems. It was developed by Telecom Italia (CSELT) in 1998. It offers an efficient transport of ACL (Agent Communication Language) messages for agents' communications which complies with FIPA specifications**. JADE is written in java language, supports mobility, evolves rapidly and until there, it is the only existent multi-agent platform (Greenwood, 2005).Jade structure includes Agent Management System) and DF (Directory Facility). A platform includes one container or many containers. In this paper, we used a JADE graphical tool which sniffs message exchange between agents. This tool is useful to debug a conversation between agents.

Agents communication
Graphical tools offered by jade allow perceiving the communications and behaviors of agents. The sniffer graphic tool on Figure 11 presents interactions instances between different agents according to the communication protocol presented in paragraph 4.2. Figure 11. Sniffer Agent GUI

Localization operator result
To test the positioning algorithm, we created a simulation in Java. We can test the algorithm in different situations by varying the size of the simulated area, the spacing tags, the amplitude of the measurement noise, the speed of pickers' movement, the pickers' number, etc. The movement of the picker is based on the "wiggle" method: the picker has a direction that varies between -45 ° and 45 ° at each step and then advances a little. This creates an impression of movement corresponding to a real movement. The figure shows an example with tag spacing of 3m and a movement speed of 5m / s.  The noise's amplitude in the first curve is constant (1 m). We note that to a tags spacing of 6m, the accuracy is very good (about 10 cm). A 6 m spacing tags corresponds to the limit of the range of the RFID reader and therefore, the reader fails to capture enough tags so that the algorithm works optimally even if the precision still greater than 30 cm ( figure 14). F e b r u a r y , 2 0 1 4 Figure 14. Curve of the precision depending on spacing of tags The tags spacing in the second curve is constant (1 m) ( figure 15). We note that the accuracy does not vary much despite noise amplitude up to 3m and is quite satisfactory (about 12 cm). For amplitude of 50 cm, the precision is worse than for amplitude of 100 cm. It is due to the setting of the adjustment rate (equation5).

Simulation scenario First scenario:
The order is composed of 21 articles. The type of resource assigned by IdA to the order is double. The type of resource can be double or simple. It depends on items number. We fixed 7 as the number from which the type of resource affected to the order is double.

Second scenario:
The order is composed of 7 items. IA sends the order to the IdA. IdA checks the availability of resources and assigns the adequate resource to the order. In this example, the type of resource that will be affected to the order is simple. In fact, the type of resource is chosen in function of items number. This request is send to the OA whose function is finding the shortest path for picking. The optimization is done in two steps: the first consist on finding the best MA and the second on scheduling items to minimize MA's route.
Beginning of working day Figure 16 presents all pickers which are at the same position in the beginning of working day. Figure 16. Picker emplacement at t=o  Order sent by WMS The WMS sends an order composed of 7 items. The picking must be done respecting the order in which the items were sent. Figure 17 represents the route of picker shown in red in the beginning of a working day. Figure 177. Picker route at t=o before optimization The order is assigned to the picker1.The distance traveled by the picker since the first item is 4,456 kilometers.
 Applying optimization approach The IdA sends the order assigned to a resource to the OA. At the beginning of the day, the assignment of the order is random because all pickers are at the same position after logging. This position is fixed for the simulation at the coordinates (0, 0). So, the OA has to find the first item in order to apply Dijkstra's algorithm ( figure 18) .   Applying optimization approach The order is assigned to the picker 3. The distance traveled by the picker since the first item is 4,179 kilometers (figure 21). Figure 21. Picker route at t=t+∆t using optimization F e b r u a r y , 2 0 1 4 Table 1 represents the gain for a mission per hour and the gain per day. Pickers work seven hour per day. The histogram below shows that the distance traveled by the picker using the proposed OBACO2W is lower than the distance traveled by the picker when the order is sent randomly by WMS ( Figure 22).

Conclusion and future work
In this paper, we propose an alliance between multi-agent system and optimization in order to localize the operator, affect the order to the nearest picker and assign to it the shortest path for picking mission. The positioning algorithm gives very good results while being robust supporting measurement noise up to 50%. With tags spacing of 3m, It would take about 1000 tags to cover an area of 10000m ² (size of a small warehouse). This spacing corresponds to the spacing required between readers if we want to make passive RFID positioning. A tag is about 1 euro while a reader is about 100 euros. So we immediately notice the economic benefit of passive RFID positioning since it would have more than 1000 persons so that the passive positioning is economic. In addition, the expansion of the surface in which you want to perform positioning is not a problem, just add a few tags. The results show that the optimization approach integrated in the OA behaviour allows to the operator to travel fewer miles. By assigning the nearest picker to the order and controlling items order, pickers earn approximately in terms of kilometres 11% and the gain can reach 28%. Reducing distance travelled by order picking and time of order picking enhance warehouse performance. For future work, we aim to improve our optimization approach and compare it to others algorithms. After picking, the operator places items on pallets in order to deliver them to customers. We aim to concentrate on this putting phase by optimizing the pallet using knapsach problem in order to satisfy customer orders.