<instructions>Your goal is to analyze the following comment and estimate how highly it will be upvoted by the Hacker News community.</instructions><comment><author>JPalakapilly</author><text>This is a great point. We completely agree that high-quality results is essential for adoption. It&amp;#x27;s basically table stakes for any tool like this to be useful. We&amp;#x27;ve had several versions of this tool that weren&amp;#x27;t quite &amp;quot;good enough&amp;quot; and never saw any real use. Our latest version seems to meet the first quality threshold for actual work use.&lt;p&gt;Our method of evaluating quality is not super systematic right now. For this competitive landscape task, we have a &amp;quot;test suite&amp;quot; of ~10 companies and for each we have a sort of &amp;quot;must-include&amp;quot;, &amp;quot;should-include&amp;quot;, &amp;quot;could-include&amp;quot; set of competitors that should be surfaced. We run these through our tool and others and look at precision and recall on the competitor sets.&lt;p&gt;In terms of errors, right now our results are a little noisy, since we&amp;#x27;re biased towards being exhaustive vs selective. There are obviously irrelevant companies in the results that no human would have ever included. Our users can fairly easily filter these out by reading the one sentence overviews of the companies but it&amp;#x27;s still not a great UX. Actively working on this.</text><parent_chain><item><author>swatcoder</author><text>&amp;gt; delivering results 50x faster at 50x less cost&lt;p&gt;What about &lt;i&gt;quality&lt;/i&gt; of results? Are you measuring that too? Did you do so for the traditional reference practice? Using what sort of methodology? How did it your technique compare in quality? What kind of errors was it most likely to make? What techniques have you devised for spotting those errors? Are they the same kind of errors that users would experience when outsourcing? Are the errors easier or harder to spot for one than the other? Are they faster to remediate with one?&lt;p&gt;I see a clever &lt;i&gt;concept&lt;/i&gt; but given the state of LLM&amp;#x27;s and the nature of how they work, I don&amp;#x27;t know that nominal cost and speed differences are really enough to sell on. Not for something &amp;quot;crucial to big business decisions.&amp;quot; I&amp;#x27;d want to know that my failure&amp;#x2F;miss rate is no worse than when outsourcing and that my &lt;i&gt;net&lt;/i&gt; cost and time (including error identification and recovery) still end up ahead. I don&amp;#x27;t see either of those vital issues touched upon here.</text></item></parent_chain></comment><story><title>Launch HN: Meticulate (YC W24) – LLM pipelines for business research</title><text>Hi HN, Meticulate gives finance professionals easy access to world class business research—think competitive landscapes, market sizings, customer segmentation etc. Here&amp;#x27;s a video of a competitive landscape generation in action (&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;youtu.be&amp;#x2F;aJ-slHcp32c)and&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;youtu.be&amp;#x2F;aJ-slHcp32c)and&lt;/a&gt; we’ve taken down the signup wall today so you can try it directly at &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;meticulate.ai&amp;#x2F;&quot;&gt;https:&amp;#x2F;&amp;#x2F;meticulate.ai&amp;#x2F;&lt;/a&gt;.&lt;p&gt;Some background on “business research”: investment and consulting teams sink many hours a week into researching companies, markets, and products. This work is time-sensitive and exhausting, but crucial to big decisions like company acquisitions or pricing model changes.&lt;p&gt;At large financial services firms, much of this work is offshored to external providers, who charge thousands of dollars per project and are often slow and low-quality. Small teams lack the budget and consistent flow of work to employ these resources. We’re building an automation solution that brings a fast, easily accessible, and defendable research resource.&lt;p&gt;Meticulate uses LLMs to emulate analyst research processes. For example, to manually build a competitive landscape like this one: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;meticulate.ai&amp;#x2F;workflow&amp;#x2F;65dbfeec44da6238abaaa059&quot;&gt;https:&amp;#x2F;&amp;#x2F;meticulate.ai&amp;#x2F;workflow&amp;#x2F;65dbfeec44da6238abaaa059&lt;/a&gt;, an analyst needs to spend ~2 hours digging through company websites, forums, and market reports. Meticulate replicates this same process of discovering, researching, and mapping companies using ~1500 LLM calls and ~500 webpages and database pulls, delivering results 50x faster at 50x less cost.&lt;p&gt;At each step, we use an LLM as an agent to run searches, select and summarize articles, devise frameworks of analysis, and make small decisions like ranking and sorting companies. Compared to approaches where an LLM is being used directly to answer questions, this lets us deliver results that (a) come from real time searches and (b) are traceable back to the original sources.&lt;p&gt;We’ve released two workflows: building competitive landscapes and market maps. We designed it with an investor running diligence on a company as the target use case but we’re seeing lots of other use cases that we didn’t originally have in mind—things like founders looking for alternative vendors for a product they’re purchasing; sales reps searching for more prospects like one they’ve already sold to; consultants trying to understand a new market they are unfamiliar with, and more.&lt;p&gt;The main challenges we’ve been overcoming are preventing quality degradation along multi-step LLM pipelines where an error on one step can propagate widely, and dealing with a wide range of data quality. We’re working hard on our next set of workflows and would love for you to give it a try at &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;meticulate.ai&quot;&gt;https:&amp;#x2F;&amp;#x2F;meticulate.ai&lt;/a&gt; and would appreciate feedback at any level!</text></story>
<instructions>Your goal is to analyze the following comment and estimate how highly it will be upvoted by the Hacker News community.</instructions><comment><author>madiator</author><text>I am sure most people never asked these questions to a human doing this research.</text><parent_chain><item><author>swatcoder</author><text>&amp;gt; delivering results 50x faster at 50x less cost&lt;p&gt;What about &lt;i&gt;quality&lt;/i&gt; of results? Are you measuring that too? Did you do so for the traditional reference practice? Using what sort of methodology? How did it your technique compare in quality? What kind of errors was it most likely to make? What techniques have you devised for spotting those errors? Are they the same kind of errors that users would experience when outsourcing? Are the errors easier or harder to spot for one than the other? Are they faster to remediate with one?&lt;p&gt;I see a clever &lt;i&gt;concept&lt;/i&gt; but given the state of LLM&amp;#x27;s and the nature of how they work, I don&amp;#x27;t know that nominal cost and speed differences are really enough to sell on. Not for something &amp;quot;crucial to big business decisions.&amp;quot; I&amp;#x27;d want to know that my failure&amp;#x2F;miss rate is no worse than when outsourcing and that my &lt;i&gt;net&lt;/i&gt; cost and time (including error identification and recovery) still end up ahead. I don&amp;#x27;t see either of those vital issues touched upon here.</text></item></parent_chain></comment><story><title>Launch HN: Meticulate (YC W24) – LLM pipelines for business research</title><text>Hi HN, Meticulate gives finance professionals easy access to world class business research—think competitive landscapes, market sizings, customer segmentation etc. Here&amp;#x27;s a video of a competitive landscape generation in action (&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;youtu.be&amp;#x2F;aJ-slHcp32c)and&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;youtu.be&amp;#x2F;aJ-slHcp32c)and&lt;/a&gt; we’ve taken down the signup wall today so you can try it directly at &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;meticulate.ai&amp;#x2F;&quot;&gt;https:&amp;#x2F;&amp;#x2F;meticulate.ai&amp;#x2F;&lt;/a&gt;.&lt;p&gt;Some background on “business research”: investment and consulting teams sink many hours a week into researching companies, markets, and products. This work is time-sensitive and exhausting, but crucial to big decisions like company acquisitions or pricing model changes.&lt;p&gt;At large financial services firms, much of this work is offshored to external providers, who charge thousands of dollars per project and are often slow and low-quality. Small teams lack the budget and consistent flow of work to employ these resources. We’re building an automation solution that brings a fast, easily accessible, and defendable research resource.&lt;p&gt;Meticulate uses LLMs to emulate analyst research processes. For example, to manually build a competitive landscape like this one: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;meticulate.ai&amp;#x2F;workflow&amp;#x2F;65dbfeec44da6238abaaa059&quot;&gt;https:&amp;#x2F;&amp;#x2F;meticulate.ai&amp;#x2F;workflow&amp;#x2F;65dbfeec44da6238abaaa059&lt;/a&gt;, an analyst needs to spend ~2 hours digging through company websites, forums, and market reports. Meticulate replicates this same process of discovering, researching, and mapping companies using ~1500 LLM calls and ~500 webpages and database pulls, delivering results 50x faster at 50x less cost.&lt;p&gt;At each step, we use an LLM as an agent to run searches, select and summarize articles, devise frameworks of analysis, and make small decisions like ranking and sorting companies. Compared to approaches where an LLM is being used directly to answer questions, this lets us deliver results that (a) come from real time searches and (b) are traceable back to the original sources.&lt;p&gt;We’ve released two workflows: building competitive landscapes and market maps. We designed it with an investor running diligence on a company as the target use case but we’re seeing lots of other use cases that we didn’t originally have in mind—things like founders looking for alternative vendors for a product they’re purchasing; sales reps searching for more prospects like one they’ve already sold to; consultants trying to understand a new market they are unfamiliar with, and more.&lt;p&gt;The main challenges we’ve been overcoming are preventing quality degradation along multi-step LLM pipelines where an error on one step can propagate widely, and dealing with a wide range of data quality. We’re working hard on our next set of workflows and would love for you to give it a try at &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;meticulate.ai&quot;&gt;https:&amp;#x2F;&amp;#x2F;meticulate.ai&lt;/a&gt; and would appreciate feedback at any level!</text></story>