Skip to content

LLM Stack Documentation for App Creators

Overview

Large Language Models (LLMs) are AI models that understand and generate human language. Using deep learning and transformer architectures, LLMs process and generate text. They excel in language tasks like question answering, search, summarization, text generation tasks. Trained on vast text datasets, LLMs learn language patterns and context, producing coherent and contextually relevant responses.

The LLM Stack (Jun 2023 version) takes mission to expedite solution prototyping and go-live using LLM for DSAID Hackathon. This is based on a collaborative effort of the following development teams:

  • DSAID LaunchPad team provides an intuitive platform for practitioners (data scientists and developers) to create AI applications
  • GDS ACE team and DASID GovText team which work together on the tech stack backend flow and computing resources

What does the LLM Stack offer?

The LLM Stack aims to expedite LLM prototyping with the following features:

  • Studio: A GUI interface to link workflow in a simpler manner to ship and build AI applications. This tool uses composable building blocks to provide a versatile and extensible method that can satisfy existing methods and adapt to new innovative methods that are constantly being released. Studio available include:
    • Vector Store Management Flow
    • Q&A Engine Flow
  • Data Hosting: GCC hosting for Vector Stores and Models that are necessary for running LLM Apps.
  • Model Access: Access to both SG-based and overseas-managed models and open-source models hosted in GCC. They can be easily swapped via a config to enable officers to experiment and find the most suitable model. These models include OpenAI models hosted on Azure.

For more details, visit this page.

How to use

  1. Go to LaunchPad Studio, and start with the available Flow templates, or build from scratch.
  2. Once done, you can publish the flow and choose whether to use the flow as an API endpoint, or as an application built by us.
  3. The published flow will return an API endpoint and an application URL if applicable.

For example, in order to build a Q&A app, the user shall proceed with the following steps:

  1. Create a Vector Store Management flow and publish the flow from LaunchPad Studio. Upon successful publishing the flow, an URL will be provided for the Vector Store web app interface.
  2. Use the Vector Store app from the given URL to convert your data to vector and store it in vector store.
  3. Create a Q&A flow (linking to the vector store created in step 2 and publish the flow from LaunchPad Studio. Upon successful publishing the flow, an URL will be provided for the Q&A web app interface.
  4. Use the Q&A app from the given URL to ask questions about your document.

Step-by-Step Guide

Quick Start

Refer to LaunchPad Studio - Quick Start

Building a Q&A app with LaunchPad Studio

Refer to LaunchPad Studio - E2E Q&A app

Help

Frequently Asked Questions

Is it mandatory to use LLM Stack for DSAID Hackathon?

It is not mandatory to use LLM Stack for DSAID Hackathon. However, Hackathon participants are encouraged to try out LLM Stack as it is expected to help expediting the development process. In addition, participants tried LLM Stach can provide us with feedback at the end of the Hackathon period so that we can validate the value of building the LLM Stack and identify areas for improvement.

Are the app user interface, for example the Q&A bot user interface and URL, accessible from Intranet?

The current version of LLM Stack produces apps and API endpoints which are internet accessible only. GSIB allowed with internet surfing shall be able to access the apps. In the future, we will consider to add accessibility via Intranet.

Will the LLM Stack, flows/apps and API endpoints produced during DSAID Hackathon be made available after Hackathon?

Hackathon participants can inform the LLM Stack technical support team about your expected period that the Stack is expected to stay live for your applications. The team will discuss with you about the feasibility for longer-term support. For applications which are expected to scale up, we will try our best to support the scale up.

Contact us

Contact us at govtext@tech.gov.sg for any questions and technical support.