Dissertation
Learning from Human Teachers: Supporting How People Want to Teach in Interactive Machine Learning
Doctor of Philosophy (PhD), Washington State University
01/2018
Handle:
https://hdl.handle.net/2376/16413
Abstract
As the number of deployed robots grows, there will be an increasing need for humans to teach robots new skills that were not pre-programmed, without requiring these users to have any experience with programming or artificial intelligent systems. To enable this, we need to better understand how people want to teach and to support the ways in which people want to teach. To learn from human teachers, we consider the case where a human could provide online evaluative feedback, or design a sequence of tasks for the agent to learn on. With respect to learning from evaluative feedback, this dissertation demonstrates that learning algorithms that treat human feedback as a complex, discrete mode of communication can be better suited to learning from human trainers, rather than simply a numeric utility function to be optimized. Our empirical results indicate that humans, when teaching agents, deliver discrete feedback and follow different training strategies. We develop a novel model of evaluative feedback that captures knowledge about a teacher's training strategy. Based on this model, we develop two Bayesian learning algorithms that can learn from real users more efficiently than previous approaches that interpret feedback as numeric. To address limited evaluative feedback, we design a new representation of the learning agent. We demonstrate empirically that by changing the speed of the agent according to its confidence level, human trainers can be implicitly motivated to provide more explicit feedback when the learner has more uncertainty about how to act. We believe this can potentially be an effective way for the agent to interact with end-users, when taking into account human factors such as frustration. Finally, we consider the case where a human could design a sequence of tasks for the agent to learn on. We investigate how non-experts design curricula and how we can adapt machine-learning algorithms to better take advantage of this non-expert guidance. We empirically show that non-experts can design curricula that result in better overall agent performance than learning from scratch. We also demonstrate that by leveraging some principles people use when designing curricula, we can significantly improve our curriculum-learning algorithm.
Metrics
68 File views/ downloads
37 Record Views
Details
- Title
- Learning from Human Teachers
- Creators
- Bei Peng
- Contributors
- Matthew E. Taylor (Advisor)Lawrence Holder (Committee Member)Janardhan R. Doppa (Committee Member)
- Awarding Institution
- Washington State University
- Academic Unit
- School of Electrical Engineering and Computer Science
- Theses and Dissertations
- Doctor of Philosophy (PhD), Washington State University
- Number of pages
- 161
- Identifiers
- 99900581508901842
- Language
- English
- Resource Type
- Dissertation