Logo image
Back
CONSTRAINT-AWARE REWARD RELABELING FOR OFFLINE SAFE REINFORCEMENT LEARNING
Thesis

CONSTRAINT-AWARE REWARD RELABELING FOR OFFLINE SAFE REINFORCEMENT LEARNING

Yassine Chemingui
Master of Science (MS), Washington State University
2026
pdf
MS_Thesis_Yassine (1)DownloadView
Open Access CC BY V4.0

Abstract

Offline Reinforcement Learning Offline Safe Reinforcement Learning
Offline safe reinforcement learning (OSRL) considers the problem of learning rewardmaximizing policies for a predefined cost constraint from a fixed dataset. This thesis proposes a simple and effective approach referred to as Constraint-aware Reward (Re)Labeling (CARL), that can be wrapped around existing offline RL algorithms. CARL is an iterative approach that alternates between two steps for each sampled batch of data to ensure state-action-wise safety constraints. First, update the cost evaluation function using an off-policy evaluation procedure. Second, update the policy using relabeled rewards (assign large penalty) for state-action pairs that are detected unsafe based on cost estimates. CARL is a minimalist approach, does not introduce any additional task-specific hyperparameters, and allows us to leverage strong off-the-shelf offline RL algorithms to solve OSRL problems. Experimental results on the DSRL benchmark tasks demonstrate that CARL reliably enforces safetyconstraints under small cost budgets, while achieving high rewards.

Metrics

1 Record Views

Details

Logo image