Skip to content

Model Abstraction Layer

Model Abstraction Layer

The Model Abstraction Layer talks to Large Language Model APIs directly, abstracting away the differences between models so the business logic inside your Sessions do not need to change when switching models or model providers.


Sessions access the MAL by executing Programs. A Program is an object that contains both a ModelSpecification that describes how to talk to the LLM and a list of Steps that will be executed synchronously. Each Step can be an Instruction or a Prompt.


Prompts provide callbacks that allow developers to handle LLM output. After a Program executes a Prompt and handles its output callback it gets turned into an Instruction.


Instructions are Prompts that have run before. Instructions allow you to create Many/Multi Shot LLM requests easily by demonstrating to the LLM how previous queries were answered.