Jose Kuttan, founder, Aviyel. Building products in AI and open source.
AI has existed in software for some time, but large language models (LLMs) have changed the landscape in consumer and enterprise software. For the first time, AI is a genuinely general-purpose computing system.
For businesses, LLMs can perform a wide range of functions that were impossible a few years back. It is now possible to generate a two-page letter, an entire tutorial or even a book given a short textual prompt. It is also possible to generate and write code. Let's consider a few use cases from specialized fields:
• A financial analyst can give an LLM an Excel file and ask for a summary of the data. Or the analyst can ask a specific sales-related query to the data warehouse.
• In healthcare, LLMs decipher the risks of a patient developing a condition.
• LLMs can be used to answer a question that requires multiple steps—like, for example, customer issues at a call center.
How Businesses Can Prepare To Integrate LLMs
We will likely see many more new applications of LLMs in the months and years to come. With specialized technical skills being handled by technology, employees' time and effort can instead be put into better understanding the business and customer needs.
This will change the way many organizations work. For example, as more and more software incorporates LLMs, many job responsibilities will evolve into verifying and approving the steps suggested by the LLMs rather than having humans complete the tasks.
Because the roles of employees will be changing with the adoption of LLMs, business leaders will need to work to help their teams adapt to these new roles. With this in mind, here are three considerations for businesses as they look to gain the most advantage of the new levels of productivity offered by LLMs:
Of course, the tricky bit about LLMs is hallucinations—i.e., making up facts—which can make them unpredictable at times and a little bit risky to adopt. If it can make up facts on the fly, can the multi-step plans and answers it gives be trusted?
There are researchers working on solutions—AutoGPT (ReAct ), Tree of Thoughts and other frameworks—to improve LLMs' accuracy in complex tasks using grounding and an approach called reason and action (or chain of thoughts). The idea is to add real data and have multiple intermediate steps to make the LLM less prone to hallucinations. It might still hallucinate, but it can be made more accurate, stable and reproducible in production systems.
In the meantime, one way to manage hallucinations while these frameworks are being developed is few-shot prompting, a process in which you provide example answers in the prompt, which can sometimes be very effective. If you are using LLMs in a domain-specific application, fine-tuning the model with data can also help reduce hallucinations.
Regardless, an important design principle is to always put the user in control and in charge. Make sure that the LLM’s output is always verified by a user. This brings us back to my point earlier that roles will move toward verifying and approving AI outputs. Business leaders must ensure that the user can always verify the output and override when necessary. As the models get better and the applications become more stable, verification might end up being all that is required.
A significant difference between LLMs and other AI models is that most other AI were built and trained for specialized use cases. LLMs, on the other hand, are often very flexible and general purpose. You don't need specialized techniques or training to integrate it into your system. It's an AI model that can quickly be shaped to the user's needs rather than vice versa.
While LLMs are general purpose, you still need to build the data pipelines to expose your internal data to the model. For example, if you are a law firm using LLMs internally to automate basic tasks, you’ll have to make sure all your documents are extracted correctly and stored in a vector database.
Accuracy will be incredibly important in some industries. In the healthcare industry, for example, there is no room for error. A lot of engineering effort will have to go into ensuring accuracy, though maybe not the specialized army of ML engineers like before.
Finally, you have to ensure your employees have the best LLM-enabled tools available. Code suggestion tools for developers, image generation tools for designers, text generation tools for your sales and non-engineering staff, etc. Help them embrace change and get them ready to make use of the advantages that are going to come along.
That said, you also need to ensure that they understand data security and the risks of sharing sensitive data with LLMs. Data security is improving with solutions like the anonymization of data that is being sent to LLMs. By understanding the risk, developing best practices for sharing your company's data with LLMs and ensuring that data has been anonymized, you can minimize the risk involved with sharing data with LLMs.
Business leaders should be excited about how AI native applications will help supercharge productivity in our organizations. This decade will bring many changes to enterprises and enterprise structures as the way we work shifts from figuring out the steps ourselves to verifying the proper set of tasks and ensuring they are executed correctly.