A Comparative Study of Deterministic and Stochastic Policies for Q-learning

Y Bi; Adam Thomas-Mitchell; Wei Zhai; Naveed Khan

doi:10.1109/airc57904.2023.10303208

A Comparative Study of Deterministic and Stochastic Policies for Q-learning

Y Bi, Adam Thomas-Mitchell, Wei Zhai, Naveed Khan

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

9 Downloads (Pure)

Abstract

Q-learning is a form of reinforcement learning that employs agents to perform actions in an environment under a policy to reach ultimate goals. Q-learning is also thought as a goal-directed learning to maximize the expected value of the cumulative rewards via optimizing policies. Deterministic and scholastic policies are commonly used in reinforcement learning. However, they perform quite different in Markov decision processes. In this study, we conduct a comparative study on these two policies in the context of a grid world problem with Q-learning and provide an insight into the superiority of the deterministic policy over the scholastic one.

Original language	English
Title of host publication	the 4th International Conference on Artificial Intelligence, Robotics and Control (AIRC 2023)
Publisher	IEEE
Pages	1-5
Number of pages	5
ISBN (Electronic)	979-8-3503-4824-8, 979-8-3503-4823-1
ISBN (Print)	979-8-3503-4825-5
DOIs	https://doi.org/10.1109/airc57904.2023.10303208
Publication status	Published online - 2 Nov 2023
Event	the 4th International Conference on Artificial Intelligence, Robotics and Control (AIRC 2023) - British University, Cairo, Egypt Duration: 9 May 2023 → 11 May 2023

Conference

Conference	the 4th International Conference on Artificial Intelligence, Robotics and Control (AIRC 2023)
Abbreviated title	AIRC 2023
Country/Territory	Egypt
City	Cairo
Period	9/05/23 → 11/05/23

Keywords

Reinforcement Learning
Q-Learning
Markov Decision Process

Access to Document

10.1109/airc57904.2023.10303208

AIRC2023-ybAuthor Accepted Manuscript, 718 KBLicence: Unspecified

Cite this

@inproceedings{e8e8cc848bfb4c82a4cc3c5ecce835c9,

title = "A Comparative Study of Deterministic and Stochastic Policies for Q-learning",

abstract = "Q-learning is a form of reinforcement learning that employs agents to perform actions in an environment under a policy to reach ultimate goals. Q-learning is also thought as a goal-directed learning to maximize the expected value of the cumulative rewards via optimizing policies. Deterministic and scholastic policies are commonly used in reinforcement learning. However, they perform quite different in Markov decision processes. In this study, we conduct a comparative study on these two policies in the context of a grid world problem with Q-learning and provide an insight into the superiority of the deterministic policy over the scholastic one.",

keywords = "Reinforcement Learning, Q-Learning, Markov Decision Process",

author = "Y Bi and Adam Thomas-Mitchell and Wei Zhai and Naveed Khan",

year = "2023",

month = nov,

day = "2",

doi = "10.1109/airc57904.2023.10303208",

language = "English",

isbn = "979-8-3503-4825-5",

pages = "1--5",

booktitle = "the 4th International Conference on Artificial Intelligence, Robotics and Control (AIRC 2023)",

publisher = "IEEE",

address = "United States",

note = "the 4th International Conference on Artificial Intelligence, Robotics and Control (AIRC 2023), AIRC 2023 ; Conference date: 09-05-2023 Through 11-05-2023",

}

Bi, Y, Thomas-Mitchell, A, Zhai, W & Khan, N 2023, A Comparative Study of Deterministic and Stochastic Policies for Q-learning. in the 4th International Conference on Artificial Intelligence, Robotics and Control (AIRC 2023). IEEE, pp. 1-5, the 4th International Conference on Artificial Intelligence, Robotics and Control (AIRC 2023), Cairo, Egypt, 9/05/23. https://doi.org/10.1109/airc57904.2023.10303208

TY - GEN

T1 - A Comparative Study of Deterministic and Stochastic Policies for Q-learning

AU - Bi, Y

AU - Thomas-Mitchell, Adam

AU - Zhai, Wei

AU - Khan, Naveed

PY - 2023/11/2

Y1 - 2023/11/2

N2 - Q-learning is a form of reinforcement learning that employs agents to perform actions in an environment under a policy to reach ultimate goals. Q-learning is also thought as a goal-directed learning to maximize the expected value of the cumulative rewards via optimizing policies. Deterministic and scholastic policies are commonly used in reinforcement learning. However, they perform quite different in Markov decision processes. In this study, we conduct a comparative study on these two policies in the context of a grid world problem with Q-learning and provide an insight into the superiority of the deterministic policy over the scholastic one.

AB - Q-learning is a form of reinforcement learning that employs agents to perform actions in an environment under a policy to reach ultimate goals. Q-learning is also thought as a goal-directed learning to maximize the expected value of the cumulative rewards via optimizing policies. Deterministic and scholastic policies are commonly used in reinforcement learning. However, they perform quite different in Markov decision processes. In this study, we conduct a comparative study on these two policies in the context of a grid world problem with Q-learning and provide an insight into the superiority of the deterministic policy over the scholastic one.

KW - Reinforcement Learning

KW - Q-Learning

KW - Markov Decision Process

U2 - 10.1109/airc57904.2023.10303208

DO - 10.1109/airc57904.2023.10303208

M3 - Conference contribution

SN - 979-8-3503-4825-5

SP - 1

EP - 5

BT - the 4th International Conference on Artificial Intelligence, Robotics and Control (AIRC 2023)

PB - IEEE

T2 - the 4th International Conference on Artificial Intelligence, Robotics and Control (AIRC 2023)

Y2 - 9 May 2023 through 11 May 2023

ER -

A Comparative Study of Deterministic and Stochastic Policies for Q-learning

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this