Seminar on Computational Learning and Adaptation

   Using Partially Observable Markov Decision Processes
         for Dialog Management in Spoken Dialog Systems

 Jason D. Williams
        Machine Intelligence Laboratory
        University of Cambridge

In a spoken dialog system, the role of the dialog manager is to decide what actions to take over time to help a user achieve their goal.  This task is difficult in large part because speech recognition errors are common, introducing uncertainty in the current state of the conversation.  In our research, we seek to model this uncertainty explicitly, and to apply machine learning techniques to generate dialog managers that cope with this uncertainty.  Partially Observable Markov Decision Process, or POMDPs, present an attractive framework in this pursuit.

In this talk, a method for formulating a dialog manager as a POMDP is presented.  In the first part of the talk the motivation for the POMDP approach is discussed.  By factoring the elements of the POMDP, a model of user behavior and speech recognition errors are directly incorporated.  Results show that, on a small dialog management task, the POMDP approach outperforms a typical baseline from the literature.

To date, POMDPs for dialog management have scaled poorly, and have been limited to artificially small "toy" problems.  In the second part of the talk, a novel approach -- called a "Summary POMDP" -- is presented, which scales POMDPs to handle slot-based dialog management problems of a realistic size.  The technique is evaluated with a user model estimated from real dialog data, and results demonstrate the operation and scalability of the method.

This talk includes joint work with Pascal Poupart from University of Waterloo and Steve Young from University of Cambridge

Return to the seminar schedule