Efficient Evaluation Functions for Multi-rover Systems

Evolutionary computation can successfully create control policies for single-agent continuous control problems. This paper extends single-agent evolutionary computation to multi-agent systems, where a large collection of agents strives to maximize a global fitness evaluation function that rates the performance of the entire system. This problem is solved in a distributed manner, where each agent evolves its own population of neural networks that are used as the control policies for the agent. Each agent evolves its population using its own agent-specific fitness evaluation function. We propose to create these agent-specific evaluation functions using the theory of collectives to avoid the coordination problem where each agent evolves neural networks that maximize its own fitness function, yet the system as a whole achieves low values of the global fitness function. Instead we will ensure that each fitness evaluation function is both “aligned” with the global evaluation function and is “learnable,” i.e., the agents can readily see how their behavior affects their evaluation function. We then show how these agent-specific evaluation functions outperform global evaluation methods by up to 600% in a domain where a collection of rovers attempts to maximize the amount of information observed while navigating through a simulated environment.