Seminar on Computational Learning and Adaptation

  An Introduction to Collective Intelligence

Kagan Tumer
Compuational Science Division
NASA Ames Research Center
kagan@ptolemy.arc.nasa.gov

This talks introduces the concept of ``COllective INtelligence'' (COIN) and the crucial steps involved in COIN design. A COIN is a large multi-agent system where:

1. There is no centralized communication among agents;

2. There is no centralized control among agents;

3. There is a well-specified global objective, and we are confronted with the inverse problem of how to configure the system to achieve that objective.

4. Agents are ``greedy'' in that they act to try to optimize their own utilities, without explicit regard to cooperation with other agents.

In particular, we are interested in COINs in which each agent runs a reinforcement learning (RL) algorithm. Rather than use a conventional modeling approach (e.g., model the system dynamics, and hand-tune agents to cooperate), we aim to solve the COIN design problem implicitly, via the ``adaptive'' character of the RL algorithms of each of the agents. This approach introduces an entirely new, profound design problem: Assuming the RL algorithms are able to achieve high rewards, what reward functions for the individual agents will, when pursued by those agents, result in high world utility? In other words, what reward functions will best ensure that we do not have phenomena like the tragedy of the commons, Braess's paradox, or the liquidity trap?

Although still very young, research specifically concentrating on the COIN design problem has already resulted in successes in artificial domains, in particular in packet-routing, the leader-follower problem, and in variants of Arthur's El Farol bar problem. It is expected that as it matures and draws upon other disciplines related to COINs, this research will greatly expand the range of tasks addressable by human engineers. Moreover, in addition to drawing on them, such a fully developed science of COIN design may provide insight into other already established scientific fields, such as economics, game theory, and population biology.

Joint work with David Wolpert.

Return to the seminar schedule