A major challenge in offline reinforcement learning (RL) is: how to learn an effective policy using only a previously collected dataset, without further interaction with the environment. Since the agent cannot explore and collect new data, it may generate out-of-distribution (OOD) actions, actions that are poorly represented or completely absent in the training dataset. Such actions can lead to unreliable or overly optimistic decisions. In general, the problem can be summarized as:
How can an offline RL agent generate high-reward actions while remaining within the distribution of actions supported by the available dataset?
