Skip to content

Model Registry

Models

The Model Registry keeps track of all the ModelSpecifications you want your GenAI Gateway project to use along with metadata (RBAC roles, etc.) that you can use in your selection logic to find the right model for a given request.

You manipulate the Model Registry’s registrations using the Roli SDK (CLI) or the Admin Console (GUI).

ModelSpecification

A ModelSpecification is configuration information about how to call a specifc Large Language Model API. ModelSpecifications are registered with the Model Registry using a string key that can later be passed to the getModel(key: string) function to get the ModelSpecification at runtime. You can re-define the ModelSpecification at any time by updating the registration, thus changing what gets returned by the getModel function.

Conclusion

Now that you understand the core concepts of Roli, you’re ready to complete the first tutorial.