Research Topic: Cooperative Multi Robot Systems for Contemporary Shopping Malls

The research aim is design and devlopment of methods and technologies for integration of cognitive characteristics of cooperative robots with advanced recognition techniques (vision, RFID). The particular objectives includes the development of methods for cooperative perception and SLAM, the development of task allocation algorithms and navigation strategies, as well as development of cooperative learning. The porposed multi-robot platform enables the different functional scenarios: cooperative distribution stock monitoring, cooperative RFID inventory control, cooperative robot formations, human-robot assistance.

The main concept of multi-robot systems for shopping malls is based on the representation of a multi-purpose robotic system for service applications with advanced perception and action capabilities [20-23]. The proposed concept is strictly oriented towards the design of multi-robotic systems whose entities have shared and common goals. The second important characteristics of the multi-robot system are related to the capability of awareness of other robot entities. With the word aware, we refer to whether entities reason about the actions and intentions of their teammates. Finally, a single entity’s actions help to advance the goals of other teammates.

The previously defined characteristics define the multi-robot systems according to the type of interactions between robot entities. The proposed system represents a type of cooperative interaction, in which robot entities are aware of other robot entities. They share goals and their actions are beneficial to their teammates. In these systems, robots may at times be working on different parts of the higher level goal, and thus may at times have to ensure that they share the workspace without interfering with each other. However, the majority of the work of the robots is focused on working together to achieve a common goal.

However, the proposed system considers also collaborative interaction between robot team mates. When robots have individual goals, they are aware of their teammates, and their actions do help advance the goals of others. Each team member has his/her own goal of performing task, but by working together with others with complementary expertise, each can help other members to better achieve their individual goals. Of course, most of these collaborations are also cooperative, and it is possible to turn a collaborative team into a cooperative team by simply viewing the team goals from a higher perspective.

The general concept of proposed multi-robotic cooperative-collaborative robotic system is based on a distributed dynamic hybrid architecture presented in Fig. 1. It consists of individual robot-agents and groups of agents that are networked, capable of acquiring and processing information from their surroundings, communicating between themselves, and sharing knowledge within the team. Robot agents, as individuals, possess advanced cooperative perception, cooperative and communicative action characteristics.

Distributed Hybrid Architecture of an multi-agent system for mega store scenarios

Fig. 1. Distributed Hybrid Architecture of an multi-agent system for mega store scenarios.

Hybrid control architecture combines local controllers with higher-level control approaches to achieve both robustness and the ability to influence the entire team’s actions through global goals, plans, or control. This approach is based on layered architectures, where each robot’s control architecture consists of a planning or social deliberative layer that decides how to achieve high-level goals; an executive or relational layer that synchronizes agents, sequences tasks, and monitors task execution; and a behavioural layer that interfaces to the robot’s sensors and effectors. Each of these layers interacts with those above and below it. Additionally, robots can interact with each other via direct connections at each of the layers.

On the social deliberative level, system behaviour that allows the team to cope with the environmental changes provides a strategy that can be adopted to reorganize the team members’ tasks. On the reactive layer, every single robot in the team copes with the environmental changes by providing a specific solution to reorganize its own task in order to fulfill the assigned goal. This layer enables realization of single primitive tasks or of composite tasks (primitive tasks linked by logical conditions on event). For the relational layer, it is important to maintain relationships between the robot and its teammates; for the social deliberative layer, the main aim is to find an appropriate strategy for task decomposition and task allocation. The proposed system represents a MR-ST-IA taxonomy according to task allocation classification. MR (multi-robot tasks) denotes that there is more than one robot working on the same task at the same time. ST (single-task robots) denotes work of a single robot on only one task, while IA (instantaneous task allocation) denotes that tasks are assigned to optimize the instantaneous allocation of tasks.

The collaborative behaviour of multi-robot teams is based on formal models and techniques that have been developed to build successful cooperative multi-robot systems and to provide solutions for several types of problems. For the problem of cooperative spatial perception based on distributed sensors, new formal models based on probabilis­tic (Bayesian) approaches together with qualitative and logic-based representations will be considered. As a second characteristic, formal models for multi-robot plans provide a significant step forward in defining suitable solutions for cooperation and collaboration.

The high-level tasks in the social deliberative layer will be achieved using specific paradigms of robust distributed intelligence. The fundamental challenge is to develop an appropriate paradigm for determining how best to achieve global coherence from the interaction of entities at the local level. The first basic paradigm of our architecture is the behaviourist approach to autonomous multi-robot control. Rather than decomposing the robot control system based on information processing functions, the behaviourist approach decomposes the high control behaviours into tasks achieving local reactive behaviours, such as obstacle avoidance, exploration, and map building. The result is a series of autonomous robots that can survive in a dynamic world, avoiding obstacles, exploring the environment, following walls, building maps, and so forth.

The organizational/social paradigms are additional techniques that have been used in to create similar higher-level, intentional cooperation and/or collaboration in multi-robot teams. Organizational and social paradigms are typically based on an organizational theory derived from human systems. In these approaches, agent/robot interactions are designed by modelling individual and group dynamics as part of an organization. The proposed system will consider three different methods of organizational/social paradigms for the realization of collective behaviour of multi-robot system. The first one is the use of roles. Use of Roles are often used to divide a system into manageable working areas that can each be assigned to a different robot in the team. An easy division of work is achieved by assigning roles according to the skills and capabilities of the individual team members. The second one is using Market economies as a paradigm for task allocation. Multi-robot task allocation is the problem of mapping tasks to robots, such that the most suitable robot is selected to perform the most appropriate task. Market-based approaches to task allocation make use of the theory of market economies to determine how best to allow robots to negotiate responsibilities in the mission. The last method is using Teamwork models that allow agents/robots to explicitly reason about coordination and communication. In dynamic environments, the ability to reason about the interactions of agents/robots can enable the team members to reorganize themselves as needed to address new situations that arise. Protocols for establishing team member commitments are determined as part of this general model.

Behaviour of multi-robot systems in different shopping environments and conditions is the subject of permanent, on-line cooperative perception, cooperative planning and action, cooperative learning and experience that robots acquire or get from humans. By being able to communicate, robot-agents collect and share information obtained from their sensor systems, as well as knowledge obtained through learning of cooperative activities. Learning will be especially important in behaviour-based systems to adapt task assignment in robot teams and to deal with individual robot capa­bilities that change over time. Specific temporal-difference approaches and reinforcement learning are very important in this context.

Advanced Mega Store Scenarios

The proposed multi-robot system is oriented toward generating enabling new technology in realistic environments, such as advanced mega stores with shopping scenario models. Such environments show different characteristics of collaborative behaviour depending on environment-specific conditions or scenario-specific situations. These scenarios will typically be set in real-world environments with a need for special functionalities such as: exploration, monitoring, robust and adaptive navigation, obstacle avoidance, and human robot interaction.

The proposal for new cognitive methods and technologies in cognitive multi-robots systems will collaborate with megastores, such as the METRO Group Future Mega Store [11] and can build on and benefit from initial results of the Future Store Initiative. So far mega stores have realized simple solutions for simple problems but in order to make progress in collaborative perceptional robotics and monitoring, a research quantum jump is needed in terms of necessary research on vision, navigation, communication and collaboration.

The proposed cognitive architecture aims at integrating the area of multi-robots systems with learning sensor fusion for vision and RFID technology developed and trial this generic technology in the Future Store Initiative. In particular modern real time image recognition techniques that use huge database of 200000 products may become possible with acceptable costs. In this way, our embedded and innovative multi-robot system enabling technology can drive important service application such as retailing for the future, in order to make this a tangible experience and to highlight the benefits for the business community as well as for consumers.

The Main Cooperation Tasks–Scenarios [Fig.2]

  1. Cooperative Vision Distribution Stock Monitoring. Multiple mobile robots patrol the shopping floor, use visual processing to identify product gaps on shelves, collect information and provide it via a collaborative agent to the store manager at regular intervals so that store operations can optimize the resources to replan all identified gaps. This most important monitoring function could be assisted with a barcode scheme, that is shown on each shelf label indicating the “to-be” product in that area, while a visual check of those products available in that area in comparison with the product picture stored in the merchandise management system, would provide evidence if that product is still available.
  2. Cooperative RFID Inventory Monitoring. Multiple mobile robots will be equipped with an RFID-based infrastructure which provides all RFID readings from a shelf area that is marked with a product barcode or RFID tags. RFID can realize an automated inventory count dependent on item tagging. In this way, the autonomous inventory system each day gets the exact picture of products in the store. That means that ordering, product placement and category management can be supported much better towards their optimal performance level.
  3. Complementary human robot assistance is performed by a robot-info/delivery agent. A robot is a guide for a human to find desired goods. The subtask is tracking the human-customer agent and assisting him/her to carry the goods. The human customer chooses the articles and puts them into the trolley (small wagon pulled by the robot) while being tracked by the robot.
  4. Cooperative traffic control. Negotiation and consensus decisions happen when some robot-agents meet each other at a narrow corridor (e.g. warehouse door). Negotiation and consensus algorithms enable agents to “agree” about the order of passing. In this situation, the system priorities determine the way of solving the problem. Cooperative formation is performed by the collective movement of robot team members from warehouse to shopping floor, and vice versa.

Advanced Mega Store Scenarios

Fig. 2. Advanced Mega Store Scenarios.