A Comparative Study of Deterministic and Stochastic Policies for Q-learning

Y Bi; Adam Thomas-Mitchell; Wei Zhai; Naveed Khan

doi:10.1109/airc57904.2023.10303208

A Comparative Study of Deterministic and Stochastic Policies for Q-learning

Y Bi, Adam Thomas-Mitchell, Wei Zhai, Naveed Khan

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

89 Downloads (Pure)

Abstract

Q-learning is a form of reinforcement learning that employs agents to perform actions in an environment under a policy to reach ultimate goals. Q-learning is also thought as a goal-directed learning to maximize the expected value of the cumulative rewards via optimizing policies. Deterministic and scholastic policies are commonly used in reinforcement learning. However, they perform quite different in Markov decision processes. In this study, we conduct a comparative study on these two policies in the context of a grid world problem with Q-learning and provide an insight into the superiority of the deterministic policy over the scholastic one.

Original language	English
Title of host publication	2023 4th International Conference on Artificial Intelligence, Robotics and Control, AIRC 2023
Publisher	IEEE
Pages	110-114
Number of pages	5
ISBN (Electronic)	9798350348248
ISBN (Print)	979-8-3503-4825-5
DOIs	https://doi.org/10.1109/airc57904.2023.10303208
Publication status	Published online - 2 Nov 2023
Event	the 4th International Conference on Artificial Intelligence, Robotics and Control (AIRC 2023) - British University, Cairo, Egypt Duration: 9 May 2023 → 11 May 2023

Publication series

Name	2023 4th International Conference on Artificial Intelligence, Robotics and Control, AIRC 2023

Conference

Conference	the 4th International Conference on Artificial Intelligence, Robotics and Control (AIRC 2023)
Abbreviated title	AIRC 2023
Country/Territory	Egypt
City	Cairo
Period	9/05/23 → 11/05/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

Reinforcement Learning
Q-Learning
Markov Decision Process
Deterministic and stochastic policies
GridWorld

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/airc57904.2023.10303208

AIRC2023-ybAuthor Accepted Manuscript, 718 KBLicence: Unspecified

Cite this

Bi, Y., Thomas-Mitchell, A., Zhai, W., & Khan, N. (2023). A Comparative Study of Deterministic and Stochastic Policies for Q-learning. In 2023 4th International Conference on Artificial Intelligence, Robotics and Control, AIRC 2023 (pp. 110-114). (2023 4th International Conference on Artificial Intelligence, Robotics and Control, AIRC 2023). IEEE. Advance online publication. https://doi.org/10.1109/airc57904.2023.10303208

@inproceedings{e8e8cc848bfb4c82a4cc3c5ecce835c9,

title = "A Comparative Study of Deterministic and Stochastic Policies for Q-learning",

abstract = "Q-learning is a form of reinforcement learning that employs agents to perform actions in an environment under a policy to reach ultimate goals. Q-learning is also thought as a goal-directed learning to maximize the expected value of the cumulative rewards via optimizing policies. Deterministic and scholastic policies are commonly used in reinforcement learning. However, they perform quite different in Markov decision processes. In this study, we conduct a comparative study on these two policies in the context of a grid world problem with Q-learning and provide an insight into the superiority of the deterministic policy over the scholastic one.",

keywords = "Reinforcement Learning, Q-Learning, Markov Decision Process, Deterministic and stochastic policies, GridWorld",

author = "Y Bi and Adam Thomas-Mitchell and Wei Zhai and Naveed Khan",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; the 4th International Conference on Artificial Intelligence, Robotics and Control (AIRC 2023), AIRC 2023 ; Conference date: 09-05-2023 Through 11-05-2023",

year = "2023",

month = nov,

day = "2",

doi = "10.1109/airc57904.2023.10303208",

language = "English",

isbn = "979-8-3503-4825-5",

series = "2023 4th International Conference on Artificial Intelligence, Robotics and Control, AIRC 2023",

publisher = "IEEE",

pages = "110--114",

booktitle = "2023 4th International Conference on Artificial Intelligence, Robotics and Control, AIRC 2023",

address = "United States",

}

Bi, Y, Thomas-Mitchell, A, Zhai, W & Khan, N 2023, A Comparative Study of Deterministic and Stochastic Policies for Q-learning. in 2023 4th International Conference on Artificial Intelligence, Robotics and Control, AIRC 2023. 2023 4th International Conference on Artificial Intelligence, Robotics and Control, AIRC 2023, IEEE, pp. 110-114, the 4th International Conference on Artificial Intelligence, Robotics and Control (AIRC 2023), Cairo, Egypt, 9/05/23. https://doi.org/10.1109/airc57904.2023.10303208

A Comparative Study of Deterministic and Stochastic Policies for Q-learning. / Bi, Y; Thomas-Mitchell, Adam; Zhai, Wei et al.
2023 4th International Conference on Artificial Intelligence, Robotics and Control, AIRC 2023. IEEE, 2023. p. 110-114 (2023 4th International Conference on Artificial Intelligence, Robotics and Control, AIRC 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Comparative Study of Deterministic and Stochastic Policies for Q-learning

AU - Bi, Y

AU - Thomas-Mitchell, Adam

AU - Zhai, Wei

AU - Khan, Naveed

PY - 2023/11/2

Y1 - 2023/11/2

N2 - Q-learning is a form of reinforcement learning that employs agents to perform actions in an environment under a policy to reach ultimate goals. Q-learning is also thought as a goal-directed learning to maximize the expected value of the cumulative rewards via optimizing policies. Deterministic and scholastic policies are commonly used in reinforcement learning. However, they perform quite different in Markov decision processes. In this study, we conduct a comparative study on these two policies in the context of a grid world problem with Q-learning and provide an insight into the superiority of the deterministic policy over the scholastic one.

AB - Q-learning is a form of reinforcement learning that employs agents to perform actions in an environment under a policy to reach ultimate goals. Q-learning is also thought as a goal-directed learning to maximize the expected value of the cumulative rewards via optimizing policies. Deterministic and scholastic policies are commonly used in reinforcement learning. However, they perform quite different in Markov decision processes. In this study, we conduct a comparative study on these two policies in the context of a grid world problem with Q-learning and provide an insight into the superiority of the deterministic policy over the scholastic one.

KW - Reinforcement Learning

KW - Q-Learning

KW - Markov Decision Process

KW - Deterministic and stochastic policies

KW - GridWorld

UR - http://www.scopus.com/inward/record.url?scp=85178550245&partnerID=8YFLogxK

U2 - 10.1109/airc57904.2023.10303208

DO - 10.1109/airc57904.2023.10303208

M3 - Conference contribution

SN - 979-8-3503-4825-5

T3 - 2023 4th International Conference on Artificial Intelligence, Robotics and Control, AIRC 2023

SP - 110

EP - 114

BT - 2023 4th International Conference on Artificial Intelligence, Robotics and Control, AIRC 2023

PB - IEEE

T2 - the 4th International Conference on Artificial Intelligence, Robotics and Control (AIRC 2023)

Y2 - 9 May 2023 through 11 May 2023

ER -

Bi Y, Thomas-Mitchell A, Zhai W, Khan N. A Comparative Study of Deterministic and Stochastic Policies for Q-learning. In 2023 4th International Conference on Artificial Intelligence, Robotics and Control, AIRC 2023. IEEE. 2023. p. 110-114. (2023 4th International Conference on Artificial Intelligence, Robotics and Control, AIRC 2023). Epub 2023 Nov 2. doi: 10.1109/airc57904.2023.10303208

A Comparative Study of Deterministic and Stochastic Policies for Q-learning

Abstract

Publication series

Conference

Bibliographical note

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this