You will be shown an excerpt from a computer science scientific research paper, followed by a question about the paper. Please answer the question. Do not include any text in your response other than the answer.

Context: Training a task-oriented conversational agent from a dialogue corpus can be naturally formulated as ofﬂine reinforcement learning (RL) problem (Levine et al., 2020; Fujimoto et al., 2019; Jaques et al., 2020), which offers the prospect to optimize the policy solely from the ﬁxed dataset without online environment interaction. Most of the existing ofﬂine RL methods are built on the off-policy Actor- Critic framework, which performs iterative optimization of the policy (i.e. actor) and the action- value function (i.e. critic) (Fujimoto et al., 2019; Janner et al., 2019; Kumar et al., 2020). Yet, a naive application of these ofﬂine RL methods generally results in poor dialogue strategies which generate responses in no way similar to human language (Lewis et al., 2017; Zhao et al., 2019; Jang et al., 2020). Ofﬂine Reinforcement Learning. There have been extensive studies on ofﬂine RL (Fujimoto et al., 2019; Levine et al., 2020; Kumar et al., 2020; Wang et al., 2020). Most of prior works are built on the off-policy actor-critic framework, and they focus on the overestimation issue by taking the OOD actions (Kumar et al., 2019; Lee et al., 2020; Fujimoto et al., 2019; Jaques et al., 2020; Kumar et al., 2020). However, a naive application of these ofﬂine RL methods suffer from the issue of diverging from human language in the task-oriented dialogues (Lewis et al., 2017; Zhao et al., 2019; Jang et al., 2020). On the other hand, there are a number of recent works on weighted behavior cloning, where a policy is trained by a variant of supervised learning loss (Wang et al., 2020; Peng et al., 2019; Siegel et al., 2020). The weighted behavior cloning approaches ﬁlter out bad actions, then perform behavior cloning on high-quality data. However, in the task-oriented dialogues, simply dropping the unsuccessful dialogues from the corpus is undesirable, since they may contain some task-speciﬁc information that is useful to properly respond to user requests. Our GPT-Critic aims to revise unsuccessful dialogues into successful ones, which is in contrast to the weighted behavior cloning on the ﬁxed training dataset, where the action choice is restricted to the support in the dataset (Wang et al., 2020; Peng et al., 2019; Siegel et al., 2020). More recently, Chen et al. (2021) introduce Decision Transformer, a Transformer-based architecture that casts the problem of RL as conditional sequence modeling. These ofﬂine RL methods based on behavior cloning are directly applied to the task-oriented dialogues without aforementioned issue, but their results are similar to that of behavior cloning in the task-oriented dialogues.

Question: What are the examples of offline RL algorithms that are not straightforwardly applicable to the task-oriented dialogue domain?
Most of the algorithms based on the policy gradient such as actor-critic are not straightforwardly applicable to the task-oriented dialogue domain