Building Spoken-Language Collaborative Interface Agents

This article reports on experiences with collaborative interface agents using spoken dialogue to collaborate with users working with graphical user interface applications. Collaborative interface agents provide users with the means to manage tasks and leave many of the details to the agent. The article presents four different collaborative agents and associated applications. It reports on lessons learned in building these agents, including the importance of choosing tasks that relieve the user of unnecessary detail, and providing speech capabilities that are useable for a wide range of users. In particular, the article reports on the success in developing a subset language for speech understanding in one of the agents. Finally, the article discusses the advantages of using the explanation capabilities in collaborative agents to help users learn new interface functionality.