Intrinsic Rewards for Maintenance, Approach, Avoidance and Achievement Goal Types

Paresh Dhakan, Ignacio Rano, N Siddique

Research output: Contribution to journalArticle

Abstract

In reinforcement learning, reward is used to guide the learning process. The reward is often designed to be task-dependent, and it may require significant domain knowledge to design a good reward function. This paper proposes general reward functions for maintenance, approach, avoidance, and achievement goal types. These reward functions exploit the inherent property of each type of goal and are thus task-independent. We also propose metrics to measure an agent's performance for learning each type of goal. We evaluate the intrinsic reward functions in a framework that can autonomously generate goals and learn solutions to those goals using a standard reinforcement learning algorithm. We show empirically how the proposed reward functions lead to learning in a mobile robot application. Finally, using the proposed reward functions as building blocks, we demonstrate how compound reward functions, reward functions to generate sequences of tasks, can be created that allow the mobile robot to learn more complex behaviors.
LanguageEnglish
Number of pages16
JournalFrontiers in Neurorobotics
DOIs
Publication statusPublished - 9 Oct 2018

Fingerprint

Reinforcement learning
Mobile robots
Robot applications
Learning algorithms

Keywords

  • intrinsic reward function
  • goal types
  • open-ended learning
  • autonomous goal generation
  • reinforcement learning

Cite this

@article{a9052fc21c904efaaa6cca2c8877ea4d,
title = "Intrinsic Rewards for Maintenance, Approach, Avoidance and Achievement Goal Types",
abstract = "In reinforcement learning, reward is used to guide the learning process. The reward is often designed to be task-dependent, and it may require significant domain knowledge to design a good reward function. This paper proposes general reward functions for maintenance, approach, avoidance, and achievement goal types. These reward functions exploit the inherent property of each type of goal and are thus task-independent. We also propose metrics to measure an agent's performance for learning each type of goal. We evaluate the intrinsic reward functions in a framework that can autonomously generate goals and learn solutions to those goals using a standard reinforcement learning algorithm. We show empirically how the proposed reward functions lead to learning in a mobile robot application. Finally, using the proposed reward functions as building blocks, we demonstrate how compound reward functions, reward functions to generate sequences of tasks, can be created that allow the mobile robot to learn more complex behaviors.",
keywords = "intrinsic reward function, goal types , open-ended learning , autonomous goal generation , reinforcement learning",
author = "Paresh Dhakan and Ignacio Rano and N Siddique",
year = "2018",
month = "10",
day = "9",
doi = "10.3389/fnbot.2018.00063",
language = "English",
journal = "Frontiers in Neurorobotics",
issn = "1662-5218",

}

Intrinsic Rewards for Maintenance, Approach, Avoidance and Achievement Goal Types. / Dhakan, Paresh; Rano, Ignacio; Siddique, N.

In: Frontiers in Neurorobotics, 09.10.2018.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Intrinsic Rewards for Maintenance, Approach, Avoidance and Achievement Goal Types

AU - Dhakan, Paresh

AU - Rano, Ignacio

AU - Siddique, N

PY - 2018/10/9

Y1 - 2018/10/9

N2 - In reinforcement learning, reward is used to guide the learning process. The reward is often designed to be task-dependent, and it may require significant domain knowledge to design a good reward function. This paper proposes general reward functions for maintenance, approach, avoidance, and achievement goal types. These reward functions exploit the inherent property of each type of goal and are thus task-independent. We also propose metrics to measure an agent's performance for learning each type of goal. We evaluate the intrinsic reward functions in a framework that can autonomously generate goals and learn solutions to those goals using a standard reinforcement learning algorithm. We show empirically how the proposed reward functions lead to learning in a mobile robot application. Finally, using the proposed reward functions as building blocks, we demonstrate how compound reward functions, reward functions to generate sequences of tasks, can be created that allow the mobile robot to learn more complex behaviors.

AB - In reinforcement learning, reward is used to guide the learning process. The reward is often designed to be task-dependent, and it may require significant domain knowledge to design a good reward function. This paper proposes general reward functions for maintenance, approach, avoidance, and achievement goal types. These reward functions exploit the inherent property of each type of goal and are thus task-independent. We also propose metrics to measure an agent's performance for learning each type of goal. We evaluate the intrinsic reward functions in a framework that can autonomously generate goals and learn solutions to those goals using a standard reinforcement learning algorithm. We show empirically how the proposed reward functions lead to learning in a mobile robot application. Finally, using the proposed reward functions as building blocks, we demonstrate how compound reward functions, reward functions to generate sequences of tasks, can be created that allow the mobile robot to learn more complex behaviors.

KW - intrinsic reward function

KW - goal types

KW - open-ended learning

KW - autonomous goal generation

KW - reinforcement learning

U2 - 10.3389/fnbot.2018.00063

DO - 10.3389/fnbot.2018.00063

M3 - Article

JO - Frontiers in Neurorobotics

T2 - Frontiers in Neurorobotics

JF - Frontiers in Neurorobotics

SN - 1662-5218

ER -