Skip to content

What does the LLM Stack offer?

What problems does the LLM Stack solve?

Large Language Models (LLMs) have transformed the way we approach artificial intelligence. These general-purpose models have proven to be adept at many different applications.

We are just beginning a new age of innovation centered around LLM. To accelerate this innovation, we present a platform that takes care of the undifferentiated heavy lifting activities that are better managed centrally.

It is easy for a person with development experience to build a prototype with LLM using a library such as Langchain or LLM-index. However, there are many challenges to producing a production-ready application:

  • Procuring managed services and GCC accounts for a proper production setup is painful. In addition, there are additional challenges when legal requirements come into play. For example, we need to clear some legal agreements before we can use OpenAI for government work.
  • Deploying such an app to production also requires a stringent setup of infrastructure and network security, with compliance to IM8 a key overhead.
  • While building LLM-powered apps might be easy for someone with development experience, it could be a major hurdle for non-developers. Our platform aims to enable anyone, especially problem owners in government, with or without development experience, to prototype new LLM-powered applications and even deploy them to production quickly.
  • The above means that the speed from idea to production is limited. Our platform aims to shorten this drastically. We will use Launchpad as a sharing and discovery platform. This enables officers to find similar use cases or to innovate upon existing ones.
  • Besides access to managed models such as OpenAI’s GPT or Google’s PaLM, there might be requirements to use only models that are hosted in Singapore. Achieving this for each new product is difficult and costly. However, it is much more easily attainable with a central platform achieving economies of scale.
  • For a use case such as an internal knowledge base question and answer bot, document management is time-consuming. Documents usually get updated over time, so re-indexing of these documents is necessary. It’s difficult to manage an optimal setup for this to reduce cost and ensure frequent updates. In addition, sometimes there is a need to manage different versions of the internal knowledge base, which adds overhead to self-management.

What does the Stack provide?

  • App Builder: A no-code/low-code App Builder in Launchpad for building LLM-powered applications or APIs that can be integrated into new or existing products. This tool uses composable building blocks to provide a versatile and extensible method that can satisfy existing methods and adapt to new innovative methods that are constantly being released.
  • Discovery Platform: A central location for sharing and discovering LLM-powered use cases in Launchpad. In addition, an innersource platform allows officers to co-create and maintain components across WOG, accelerating the pace of innovation.
  • Data Hosting: IM8-compliant hosting for Vector Stores and Models that are necessary for running LLM Apps.
  • Secure Infrastructure: Robust, scalable, and secure APIs and applications that accelerate the speed from idea to production. Access control, logging, and monitoring are provided by default.
  • Model Access: Access to both SG-based and overseas-managed models and open-source models hosted in GCC. They can be easily swapped via a config to enable officers to experiment and find the most suitable model.

Additional Benefits of the Stack

  • Experimentation: Building AI applications involves experimentation. Different models, model settings, and prompts can all affect the output. Our platform enables app creators to easily create different versions with different setups and perform data analytics to understand what works best for their use case.
  • Introspection: Traces and logging are enabled by default, allowing app creators to deeply analyze the performance of their application in terms of both speed and quality of output.
  • Stability: Given the need to use multiple external services, the platform allows users to easily maintain caching and retries mechanisms to ensure high workflow SLA.