Autotelic Reinforcement Learning: Exploring Intrinsic Motivations for Skill Acquisition in Open-Ended Environments |
||
 |
 |
|
| © 2025 by IJCTT Journal | ||
| Volume-73 Issue-1 |
||
| Year of Publication : 2025 | ||
| Authors : Prakhar Srivastava, Jasmeet Singh | ||
| DOI : 10.14445/22312803/IJCTT-V73I1P104 | ||
How to Cite?
Prakhar Srivastava, Jasmeet Singh, "Autotelic Reinforcement Learning: Exploring Intrinsic Motivations for Skill Acquisition in Open-Ended Environments," International Journal of Computer Trends and Technology, vol. 73, no. 1, pp. 32-43, 2025. Crossref, https://doi.org/10.14445/22312803/IJCTT-V73I1P104
Abstract
This paper presents a comprehensive overview of autotelic Reinforcement Learning (RL), emphasizing the role of intrinsic motivations in the open-ended formation of skill repertoires. We delineate the distinctions between knowledge-based and competence-based intrinsic motivations, illustrating how these concepts inform the development of autonomous agents capable of generating and pursuing self-defined goals. The typology of Intrinsically Motivated Goal Exploration Processes (IMGEPs) is explored, with a focus on the implications for multi-goal RL and developmental robotics. The autotelic learning problem is framed within a reward-free Markov Decision Process (MDP), WHERE agents must autonomously represent, generate, and master their own goals. We address the unique challenges in evaluating such agents, proposing various metrics for measuring exploration, generalization, and robustness in complex environments. This work aims to advance the understanding of autotelic RL agents and their potential for enhancing skill acquisition in a diverse and dynamic setting.
Keywords
Skill Acquisition, Reinforcement learning, Social autonomous agents, Open-Ended environments, Social Learners.
Reference
[1] Richard S. Sutton, and Andrew G. Barto, Reinforcement Learning: An Introduction, MIT press, pp. 1-552, 2018.
[Google Scholar] [Publisher Link]
[2] Dean A. Pomerleau, “Efficient Training of Artificial Neural Networks for Autonomous Navigation,” Neural computation, vol. 3, no. 1, pp. 88-97, 1991.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Michael L. Littman, “Markov Games as a Framework for Multiagent Reinforcement Learning,” Machine Learning Proceedings 1994, pp. 157-163, 1994.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Volodymyr Mnih et al., “Human-level Control through Deep Reinforcement Learning,” Nature, vol. 518, pp. 529-533, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[5] David Silver et al., “Mastering the Game of Go with Deep Neural Networks and Tree Search,” Nature, vol. 529, pp. 484-489, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Marc G. Bellemare et al., “Autonomous Navigation of Stratospheric Balloons using Reinforcement Learning,” Nature, vol. 588, pp. 77 82, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[7] Ashish Vaswani et al., “Attention is all you Need,” 31st Conference on Neural Information Processing Systems, pp. 1-15, 2017.
[Google Scholar]
[8] Andrew J. Elliot, and James W. Fryer, “The Goal Construct in Psychology,” Handbook of Motivation Science, vol. 18, pp. 235-250, 2008.
[Google Scholar]
[9] J.M. Hunt, “Intrinsic Motivation and Its Role in Psychological Development,” Nebraska Symposium on Motivation, vol. 13, pp. 189-282, 1965.
[Google Scholar] [Publisher Link]
[10] Joar Vittersø, “Mihaly Csikszentmihalyi, Finding Flow. The Psychology of Engagement with Everyday Life,” Journal of Happiness Studies, vol. 1, pp. 121-123, 2000.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Jelena Luketina et al., “A Survey of Reinforcement Learning Informed by Natural Language,” Machine Learning, pp. 6309-6317, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[12] Natasha Jaques et al., “Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning,” Proceedings of the 36th International Conference on Machine Learning, pp. 3040-3049, 2019.
[Google Scholar] [Publisher Link]
[13] L. Steels, “Language Games for Autonomous Robots,” IEEE Intelligent Systems, vol. 16, no. 5, pp. 16-22, 2001.
[CrossRef] [Google Scholar] [Publisher Link]
[14] Dean A. Pomerleau, “Alvinn: An Autonomous Land Vehicle in a Neural Network,” Advances in Neural Information Processing Systems, 1988.
[Google Scholar] [Publisher Link]
[15] Stefan Schaal, “Learning from Demonstration,” Advances in Neural Information Processing Systems, 1996.
[Google Scholar] [Publisher Link]
[16] Takayuki Osa et al., “An Algorithmic Perspective on Imitation Learning,” Foundations and Trends® in Robotics, vol. 7, no. 1-2, pp. 1 179, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[17] Y. LeCun et al., “Gradient-based Learning Applied to Document Recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
[CrossRef] [Google Scholar] [Publisher Link]