Home Schedule Assignments Special Topics Readings Resources Project Ideas Policies

Special Topic: Reinforement learning and Simulated Users for Dialogue Management

Required Readings

  1. Jason D. Williams and Steve Young. Scaling POMDPs for spoken dialog management. IEEE Transactions on Audio, Speech, and Language Processing 2007.
  2. Kallrroi Georgila, James Henderson, and Oliver Lemon. User Simulation for Spoken Dialogue Systems: Learning and Evaluation Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), pp. 1065-1068, Pittsburgh, USA, 2006.
  3. Oliver Lemon, Kallirroi Georgila, and James Henderson. Evaluating Effectiveness and Portability of Reinforcement Learned Dialogue Strategies with Real Users: The TALK TownInfo Evaluation Proceedings of the IEEE/ACL Workshop on Spoken Language Technology (SLT), pp. 178-181, Aruba, 2006.
  4. Jost Schatzmann and Steve Young. The Hidden Agenda User Simulation Model IEEE Transactions on Audio, Speech, and Language Processing, 17(4):733-747, 2009.
  5. Florian L. Kreyssig, Inigo Casanueva, Pawel Budzianowski, and Milica Gasic. Neural User Simulator for Corpus-based Policy Optimisation for Spoken Dialogue Systems SIGDIAL 2018.
  6. Layla El Asri, Jing He, and Kaheer Suleman. A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systems Interspeech 2016.
  7. Pei-Hao Su, Paweł Budzianowski, Stefan Ultes, Milica Gasic, and Steve Young. Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Management SIGDIAL 2017.
  8. Baolin Peng, Xiujun Li, Jianfeng Gao, Jingjing Liu, and Kam-Fai Wong. Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning ACL 2018.

Other Readings
  1. James Henderson, Oliver Lemon, and Kallirroi Georgila. Hybrid reinforcement/supervised learning of dialogue Policies from fixed datasets. Computational Linguistics 2008.
  2. Jost Schatzmann, Blaise Thomson, Karl Weilhammer, Hui Ye, and Steve Young. Agenda-based user simulation for bootstrapping a POMDP dialogue system. NAACL 2007
  3. Kallirroi Georgila, James Henderson, and Oliver Lemon. Learning user simulations for information state update dialogue systems. Interspeech 2005.
  4. Steve Young, Milica Gasic, Simon Keizer, Francois Mairesse, Jost Schatzmann, Blaise Thomson, Kai Yu. The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management. Computer Speech and Language 2010.
  5. Tiancheng Zhao and Maxine Eskenazi. Towards end-to-end learning for dialogue state tracking and management using deep reinforcement learning. SIGDIAL 2016.
  6. Inigo Casanueva, Pawel Budzianowski, Pei-Hao Su, Stefan Ultes, Lina Rojas-Barahona, Bo-Hsiang Tseng, and Milica Gasic. Feudal reinforcement learning for dialogue management in large domains. NAACL 2018.
  7. Ramesh Manuvinakurike, David DeVault, and Kallirroi Georgila. Using reinforcement learning to model incrementality in a fast-paced dialogue game. SIGDIAL 2017.
  8. Kallirroi Georgila and David Traum. Reinforcement learning of argumentation dialogue policies in negotiation. Interspeech 2011.
  9. Jason D. Williams and Steve Young. Partially observable Markov decision processes for spoken dialog systems. Computer Speech and Language, 21:393-422, 2007
  10. Milica Gasic and Steve Young. Gaussian processes for POMDP-based dialogue manager optimization. IEEE Transactions on Audio, Speech, and Language Processing, 22(1):28-40, 2014.
  11. Jost Schatzmann, Kallirroi Georgila, and Steve Young. Quantitative Evaluation of User Simulation Techniques for Spoken Dialogue Systems. Proceedings of the 6th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL), pp. 45-54, Lisbon, Portugal, 2005.
  12. Kallirroi Georgila, James Henderson, and Oliver Lemon. Learning User Simulations for Information State Update Dialogue Systems. Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), pp. 893-896, Lisbon, Portugal, 2005.
  13. Ramesh Manuvinakurike, David DeVault, and Kallirroi Georgila. Using Reinforcement Learning to Model Incrementality in a Fast-Paced Dialogue Game. Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL), pp. 331-341, Saarbruecken, Germany, 2017.
  14. Alexandros Papangelis and Kallirroi Georgila. Reinforcement Learning of Multi-Issue Negotiation Dialogue Policies. Proceedings of the 16th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL), pp. 154-158, Prague, Czech Republic, 2015.
  15. Kallirroi Georgila, Maria Wolters, and Johanna D. Moore. Simulating the Behaviour of Older versus Younger Users when Interacting with Spoken Dialogue Systems. Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics - Human Language Technologies (ACL-HLT), Short Papers, pp. 49-52, Columbus, Ohio, USA, 2008. Lu Chen, Zhi Chen, Bowen Tan
  16. Sishan Long, Milica Gasic, and Kai Yu. AgentGraph: Towards Universal Dialogue Management with Structured Deep Reinforcement Learning. IEEE/ACM Transactions on Audio, Speech, and Language Processing 2019.
  17. Heriberto Cuayahuitl, Simon Keizer, and Oliver Lemon. Strategic Dialogue Management via Deep Reinforcement Learning. NIPS Workshop on Deep Reinforcement Learning 2015.
  18. Pararth Shah, Dilek Hakkani-Tur, and Larry Heck. Interactive reinforcement learning for task-oriented dialogue management.
  19. Baolin Peng, Xiujun Li, Lihong Li, Jianfeng Gao, Asli Celikyilmaz, Sungjin Lee, and Kam-Fai Wong. Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning. EMNLP 2017.