<instructions>Your goal is to analyze the following comment and estimate how highly it will be upvoted by the Hacker News community.</instructions><comment><author>gwern</author><text>Discussion &amp;amp; links to various implementations: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.reddit.com&amp;#x2F;r&amp;#x2F;MachineLearning&amp;#x2F;comments&amp;#x2F;eg1wr3&amp;#x2F;reformer_the_efficient_transformer_anonymous_et&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.reddit.com&amp;#x2F;r&amp;#x2F;MachineLearning&amp;#x2F;comments&amp;#x2F;eg1wr3&amp;#x2F;ref...&lt;/a&gt;</text><parent_chain></parent_chain></comment><story><title>Reformer, the Efficient Transformer</title><url>https://ai.googleblog.com/2020/01/reformer-efficient-transformer.html</url></story>
<instructions>Your goal is to analyze the following comment and estimate how highly it will be upvoted by the Hacker News community.</instructions><comment><author>lapink</author><text>There is no argument for why the LSH would work well, especially at the beginning of training. As the weights are initially random, bucket assignment  would be random as well. If predicting at position A requires info from position B,  but they are not in the same bucket, there will be no gradient to get the query embedding of A closer to the key embedding of B. The reversible layer trick is neat though.</text><parent_chain></parent_chain></comment><story><title>Reformer, the Efficient Transformer</title><url>https://ai.googleblog.com/2020/01/reformer-efficient-transformer.html</url></story>