LLMOps: Is Your Enterprise Architecture Function Ready?

LLMOps is a business function working to implement generative AI and large language models for enterprise. Discover how enterprise architecture is perfectly placed to support this new field.

Large language model operations (LLMOps) is an emerging field that is rising to meet the challenge of leveraging generative artificial intelligence (AI) large language models (LLMs). While it's clear that tools such as ChatGPT are revolutionary, how to best manage the practical implementation of these applications is still an open question.

We believe that enterprise architects can provide the intelligence and strategic planning capabilities that LLMOps needs to leverage generative AI. Let's consider what LLMs are and why they're so important and challenging for enterprise.

To find out more about how LeanIX can support you in developing LLMOps, book a demo:

What Is An LLM?

Large language models (LLMs) are sophisticated artificial intelligence (AI) algorithms that can understand and generate text comparable to the standard that a human can. This allows them to instantly generate reports and communications, and answer questions based on the data you connect them to.

LLMs have been trained by processing millions of documents and interactions from the internet until they gained an understanding of natural language. Rather than having to use a programming language or user interface (UI) to get answers from a computer in kind, you can simply ask an LLM to do something and it will understand and reply in a way that anyone can understand.

With an Excel spreadsheet, for example, you normally need to code a 'vlookup' to see how many of your data points are meeting their service level agreements (SLAs), then interpret that data. With a properly implemented LLM you could simply ask it to look at the spreadsheet and give you an instant yes or no answer.

LLMs Use Cases

Large language models (LLMs) only became available to the public with the launch of OpenAI's ChatGPT less than a year ago. Already, Massachusetts Institute of Technology (MIT) studies are showing a 40% increase in productivity when using generative AI to carry out simulated tasks.

While instantly generating documents is a huge time-saver, there are a variety of other benefits that could prove to be more important. The advantages include:

LLM For Communication

Rather than composing differently worded emails for a diverse set of stakeholders, LLMs can simply be asked to notify your stakeholders on a development and can compose different communications for each. Conversely, an LLM can read all the emails in your inbox and then summarize any important information for you.

LLM For Sentiment Analysis

Just as an LLM can summarize your emails, they can read and summarize all the communications throughout your organization, or all the social media conversations your customers are having about you. This can give you a quick run-down of your reputation, which can be tracked over time.

LLM For Instant Translation

LLMs have been used to translate what a person is saying into another language in real time. As such, you could have a Zoom call with someone in English while they experience it in French.

LLM For Malware Analysis

So-called Zero-Day vulnerabilities are a crucial issue for software developers, as it's impossible to be prepared for a vulnerability that was first discovered by a cyber attacker. Companies like Google are using LLMs to read through code and report back on any script that looks like it might have a malicious application, even if it hasn't been seen before.

LLM For Citizen Development

Since LLMs can understand computer code and natural language, it's possible to ask an LLM to code you a custom software application to complete a specific task. This democratizes software development, but will also enhance the challenges that come with so-called 'shadow IT'.

Infinite Use Cases

Large language models (LLMs) have an almost infinite number of use cases, and are set to revolutionize the workplace and your IT landscape. Specialized LLM applications are being developed for software development, the legal industry, HR, finance, and the medical industry.

As such, it's no surprise that a vast number of LLM startups have entered the market in the last year. Back in September, Baidu CEO Robin Li claimed more than 70 LLMs had been released in China alone.

The below image from Insight Partners begins to give an indication of the complexity of leveraging LLMs for your organization:

LLMOps Is Your Enterprise Architecture Function Ready - complexity of generative AI implementation

With so many LLM products entering the market for such a variety of use cases, choosing which to implement in which of your business processes will be a challenge in itself. That's why so many businesses are quickly developing an LLM operations function.

What Is LLMOps?

Large language model operations (LLMOps) is a quickly developing discipline for selecting, implementing, and maintaining large language model (LLM) applications to support your business processes. Having a dedicated function working with these complex tools is key for leveraging artificial intelligence (AI) innovation.

This requires both having a responsible person or persons to monitor the market and your IT and LLM landscape, and also raising awareness throughout your organization of the risks of LLM use. Many companies have already run afoul of employees entering confidential data into an unsecured LLM just to try it out.

Your LLMOps function then also needs to be empowered with a dedicated toolset. This means the race is on to develop that toolset as the challenges for LLMOps are mounting.

The Challenges For LLMOps

Large language model operations (LLMOps) is an emerging discipline that is already facing challenges. As we all work to understand the capabilities and use cases for large language models (LLMs), LLMOps already have to deal with a variety of issues:

LLM Rationalization

Determining which LLM applications create value for you and which simply increase complexity will be key. Likewise, LLMOps will need to decide whether having a single LLM with multiple uses or many dedicated LLMs is more beneficial.

LLM Scale

Given the number of potential use cases for LLMs and the security risks, it's essential to determine at what scale to implement artificial intelligence (AI) tools. Should you limit LLMs to specific and siloed use cases or allow it to operate throughout your tech stack?

LLM Cost

Many LLMs have been, thus far, released as free services with premium options for enterprise. As the costs of running an LLM grow, however, the price of subscription is likely to grow significantly.

LLM Load

Running an LLM takes an extraordinary amount of computing power, and organizations looking to leverage them will likely need to invest in increased capacity. This only serves to make scaling LLM use more challenging.

LLM Security

Securing your LLM to prevent leakage of confidential data is a vital function of LLMOps. In addition, regulators are promising closer scrutiny of generative AI tools.