Shaping Reward Signals in Reinforcement Learning Using Constraint Programming
(2025) CPAIOR 2025
Files
978-3-031-95976-9.pdf
Open Access - Adobe PDF
- 19.91 MB
Details
- Authors
- Abstract
- Reinforcement learning is a machine learning paradigm in which an agent learns through interaction a policy, that is what sequence of actions to take in some environment in order to maximize the reward it receives. Reward shaping modifies the reward function in an attempt to accelerate agent training. Under some conditions, potential-based reward shaping (pbrs) preserves the optimal policies of the original problem. We propose a pbrs method that automatically derives shaped rewards from a constraint programming (cp) model of the environment and from the probability mass functions over the domains of its variables, as provided by the cpbp framework. In the context of an ai planning task, we investi- gate the effect of cp modeling choices on the effectiveness of our reward shaping method. Our experiments show that our method significantly shortens training while being rather insensitive to modeling choices, and that the resulting agent’s performance scales well beyond instance sizes seen during training.
- Affiliations
Citations
Yin, C., Cappart, Q., & Pesant, G. (2025). Shaping Reward Signals in Reinforcement Learning Using Constraint Programming. CPAIOR 2025, Part II, 15763, 252-269. https://doi.org/10.1007/978-3-031-95976-9_16 (Original work published 2025)