A Cognitive Architecture for Physical Agents

The Icarus project focuses on two complementary goals: the AI community's original dream of creating complete intelligent agents and Allen Newell's vision of a unified theory of the cognitive architecture. We are especially concerned with agents that exist in a physical environment, whether actual or simulated, and thus must integrate cognition with perception and action. This focus grew originally out of the World Modelers Project, a joint effort with Jaime Carbonell and others at CMU during the 1980s to create a realistic simulated environment to serve as a testbed for research on physical agents.

The Icarus architecture has gone through a number of different versions, the most recent developed at DaimlerChrysler Research & Technology Center between 1998 and 2000, and which, naturally, we tested in a simulated driving domain. This incarnation of Icarus encodes knowledge as reactive skills, each of which specifies the goal-relevant reactions to a class of situations. A skill consists of three elements stated in terms of logical expressions: a set of objectives, a set of requirements or preconditions, and a set of alternate means for accomplishing the objective under those conditions. Each objective, requirement, or means can refer to primitive actions/sensors or to other Icarus skills, thus imposing a hierarchical organization on long-term memory. Each skill also has an associated utility cast as a linear function of sensory attributes.

The basic Icarus interpreter operates on a recognize-act cycle but, unlike many architectures, focuses on reactive execution of existing skills rather than on problem-space search. Given a top-level skill to pursue, on each cycle the system first checks the objective field for that skill. If the objectives are true, nothing further needs to be done, but, if not, the interpreter examines the requirements to determine if the preconditions for action are met. If not, Icarus invokes a subskill associated with the failed requirement in an effort to satisfy it; otherwise, it selects one of the alternate means and calls on the primitive action or subskill associated with it. The architecture selects the alternative with the highest expected utility as predicted by the linear function associated with each skill.

We have recently extended the Icarus architecture to support explicit short-term memories and modules for conceptual inference, means-ends problem solving, and cumulative learning of skills. The new version relies on five distinctive assumptions:

Primacy of action and perception over cognition;

Separation of categories from skills;

Hierarchical structure of long-term memory;

Correspondence between long-term and short-term structures; and

Value-driven nature of agent behavior.

Our recent papers explain these assumptions in more detail. Our research plans include incorporating mechanisms for forward chaining, model-based learning from delayed reward, storing/retrieving episodic traces, and learning from other agents' behaviors. We are also looking at new domains, such as game playing, in which to evaluate the framework.

This research is funded by DARPA IPTO and the National Science Foundation. Support for earlier work came from the Air Force Office of Scientific Research, NASA Ames Research Center, and DaimlerChrysler Research and Technology.

Current Contributors to the Project

Langley, P., & Choi, D. (in press). A unified cognitive architecture for physical agents. Proceedings of the Twenty-First National Conference on Artificial Intelligence. Boston: AAAI Press.
Nejati, N., Langley, P., & Konik, T. (in press). Learning hierarchical task networks by observation. Proceedings of the Twenty-Third International Conference on Machine Learning. Pittsburgh, PA.
Asgharbeygi, N., Stracuzzi, D., & Langley, P. (in press). Relational temporal difference learning. Proceedings of the Twenty-Third International Conference on Machine Learning. Pittsburgh, PA.
Langley, P. (in press). Cognitive architectures and general intelligent systems. AI Magazine.
Langley, P., & Choi, D. (2006). Learning recursive control programs from problem solving. Journal of Machine Learning Research, 7, 493-518.
Langley, P. (2005). An adaptive architecture for physical agents. Proceedings of the 2005 IEEE/WIC/ACM International Conference on Intelligent Agent Technology (pp. 18-25). Compiegne, France: IEEE Computer Society Press.
Langley, P., Choi, D., & Rogers, S. (2005). Interleaving learning, problem solving, and execution in the Icarus architecture (Technical Report). Computational Learning Laboratory, CSLI, Stanford University, CA.
Choi, D., & Langley, P. (2005). Learning teleoreactive logic programs from problem solving. Proceedings of the Fifteenth International Conference on Inductive Logic Programming (pp. 51-68). Bonn, Germany: Springer.
Asgharbeygi, N., Nejati, N., Langley, P., & Arai, S. (2005). Guiding inference through relational reinforcement learning. Proceedings of the Fifteenth International Conference on Inductive Logic Programming (pp. 20-37). Bonn, Germany: Springer.
Langley, P., & Rogers, S. (2005). An extended theory of human problem solving. Proceedings of the Twenty-seventh Annual Meeting of the Cognitive Science Society. Stresa, Italy.
Langley, P., & Rogers, S. (2004). Cumulative learning of hierarchical skills. Proceedings of the Third International Conference on Development and Learning. San Diego, CA: IEEE Press.
Langley, P. (2004). Cognitive architectures and the construction of intelligent agents. Proceedings of the AAAI-2004 Workshop on Intelligent Agent Architectures (pp. 82). Stanford, CA.
Langley, P., Arai, S., & Shapiro, D. (2004). Model-based learning with hierarchical relational skills. Proceedings of the ICML-2004 Workshop on Relational Reinforcement Learning. Banff, Alberta.
Langley, P., Cummings, K., & Shapiro, D. (2004). Hierarchical skills and cognitive architectures. Proceedings of the Twenty-Sixth Annual Conference of the Cognitive Science Society (pp. 779-784). Chicago, IL.
Choi, D., Kaufman, M., Langley, P., Nejati, N., & Shapiro, D. (2004). An architecture for persistent reactive behavior. Proceedings of the Third International Joint Conference on Autonomous Agents and Multi Agent Systems (pp. 988-995). New York: ACM Press.
Langley, P., Choi, D., & Shapiro, D. (2004). A cognitive architecture for physical agents (Technical Report). Computational Learning Laboratory, CSLI, Stanford University, CA.
Ichise, R., Shapiro, D., & Langley, P. (in press). Structured program induction from behavioral traces. IEICE Transactions on Information and Systems (in Japanese).
Langley, P., Shapiro, D., Aycinena, M., & Siliski, M. (2003). A value-driven architecture for intelligent behavior. Proceedings of the IJCAI-2003 Workshop on Cognitive Modeling of Agents and Multi-Agent Interactions (pp. 10-18). Acapulco, Mexico.
Langley, P., & Laird, J. E. (2002). Cognitive architectures: Research issues and challenges (Technical Report). Institute for the Study of Learning and Expertise, Palo Alto, CA.
Ichise, R., Shapiro, D. G., & Langley, P. (2002). Learning hierarchical skills from observation (pp. 247-258). Proceedings of the Fifth International Conference on Discovery Science.
Shapiro, D., & Langley, P. (2002). Separating skills from preference: Using learning to program by reward. Proceedings of the Nineteenth International Conference on Machine Learning (pp. 570-577). Sydney: Morgan Kaufmann.
Shapiro, D., Langley, P., & Shachter, R. (2001). Using background knowledge to speed reinforcement learning in physical agents. Proceedings of the Fifth International Conference on Autonomous Agents (pp. 254-261). Montreal: ACM Press.
Shapiro, D., & Langley, P. (1999). Controlling physical agents through reactive logic programming. Proceedings of the Third International Conference on Autonomous Agents (pp. 386-387). Seattle: ACM Press.
Langley, P. (1997). Learning to sense selectively in physical domains. Proceedings of the First International Conference on Autonomous Agents (pp. 217-226). Marina del Rey, CA: ACM Press. For more information, please send email to dgs@stanford.edu .

A Cognitive Architecture for Physical Agents

Current Contributors to the Project

Past Contributors to the Project

Related Papers