<instructions>Your goal is to analyze the following comment and estimate how highly it will be upvoted by the Hacker News community.</instructions><comment><author>milkey_mouse</author><text>&amp;gt; I do have a feeling of dejavú, like I&amp;#x27;ve seen this before on hn.&lt;p&gt;You&amp;#x27;re either thinking of speculative encoding more generally, or Medusa: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;abs&amp;#x2F;2401.10774&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;abs&amp;#x2F;2401.10774&lt;/a&gt;</text><parent_chain><item><author>SonOfLilit</author><text>Good idea. Generate 8 tokens with a small model, then give them to a large model as one batch (much faster) and if tokens 1..3 are in agreement but 4..8 nut, take only 1..4 and start again. Most tokens are easy to guess so you&amp;#x27;ll get x3.5 gains.&lt;p&gt;I do have a feeling of dejavú, like I&amp;#x27;ve seen this before on hn.</text></item></parent_chain></comment><story><title>Lossless Acceleration of LLM via Adaptive N-Gram Parallel Decoding</title><url>https://arxiv.org/abs/2404.08698</url></story>
<instructions>Your goal is to analyze the following comment and estimate how highly it will be upvoted by the Hacker News community.</instructions><comment><author>johntb86</author><text>They mention previous work on speculative decoding using similar techniques, but &amp;quot;ANPD dynamically generates draft outputs
via an adaptive N-gram module using real-time
statistics, after which the drafts are verified by the
LLM. This characteristic is exactly the difference
between ANPD and the previous speculative decoding methods.&amp;quot;</text><parent_chain><item><author>SonOfLilit</author><text>Good idea. Generate 8 tokens with a small model, then give them to a large model as one batch (much faster) and if tokens 1..3 are in agreement but 4..8 nut, take only 1..4 and start again. Most tokens are easy to guess so you&amp;#x27;ll get x3.5 gains.&lt;p&gt;I do have a feeling of dejavú, like I&amp;#x27;ve seen this before on hn.</text></item></parent_chain></comment><story><title>Lossless Acceleration of LLM via Adaptive N-Gram Parallel Decoding</title><url>https://arxiv.org/abs/2404.08698</url></story>