<instructions>Your goal is to analyze the following comment and estimate how highly it will be upvoted by the Hacker News community.</instructions><comment><author>leobg</author><text>I think the ideas is that the LLM cannot think internally. It’s output &lt;i&gt;is&lt;/i&gt; its thinking process. Especially with an auto regressive architecture like GPT, where each output token becomes part of the input.&lt;p&gt;I imagine it like handing the LLM a piece of scratch paper.</text><parent_chain></parent_chain></comment><story><title>When do you need chain-of-thought prompting for ChatGPT?</title><url>https://arxiv.org/abs/2304.03262</url></story>
<instructions>Your goal is to analyze the following comment and estimate how highly it will be upvoted by the Hacker News community.</instructions><comment><author>belter</author><text>&amp;quot;...Chain-of-Thought (CoT) prompting can effectively elicit complex multi-step reasoning from Large Language Models~(LLMs). For example, by simply adding CoT instruction ``Let&amp;#x27;s think step-by-step&amp;#x27;&amp;#x27; to each input query of MultiArith dataset, GPT-3&amp;#x27;s accuracy can be improved from 17.7\% to 78.7\%. However, it is not clear whether CoT is still effective on more recent instruction finetuned (IFT) LLMs such as ChatGPT. Surprisingly, on ChatGPT, CoT is no longer effective for certain tasks such as arithmetic reasoning while still keeping effective on other reasoning tasks. Moreover, on the former tasks, ChatGPT usually achieves the best performance and can generate CoT even without being instructed to do so...&lt;p&gt;...Hence, it is plausible that ChatGPT has already been trained on these tasks with CoT and thus memorized the instruction so it implicitly follows such an instruction when applied to the same queries, even without CoT. Our analysis reflects a potential risk of overfitting&amp;#x2F;bias toward instructions introduced in IFT, which becomes more common in training LLMs...&amp;quot;</text><parent_chain></parent_chain></comment><story><title>When do you need chain-of-thought prompting for ChatGPT?</title><url>https://arxiv.org/abs/2304.03262</url></story>