Create a Conversational Chatbot

Who is this for?

This guide is suitable for entry-level software developers who want to learn how to use the Roli platform to build powerful backends.

What you’re building

This tutorial walks you through building a chatbot on Roli that consists of a public internet-facing WebSocket API routing conversational calls to an LLM registered with the Model Registry. Completing with a code-generate, documented SDK you can distribute (npm module).

Prerequisites

• Comfortable running terminal commands.
• Node Package Manager (NPM) installed.
• Basic understanding of TypeScript or JavaScript.
• (Optional) Join the Roli Community Discord.

Code Conventions

// This is what it looks like when you're supposed to add a line...
console.log("Hey there");
// and when you're supposed to delete a line...
console.log("Woops!");
// or insert something mid-line:
console.log("Hey!");
// or delete an item mid-line:
console.log("Oh no...");

Get the tutorial scaffolding

1. Download the tutorial scaffolding

npx tiged roliai/tutorial tutorial

2. The remainder of the commands will be run from the tutorial directory. Change to it.

cd tutorial

Setup Roli Tools

1. Run the command to install the roli-tools package

npm install -g roli-tools

2. Set Roli’s connection information so it can talk to the backend platform installation.

roli set-connection-info admin=https://admin.roli.app api=https://api.roli.app login=https://admin.roli.app/login --enterprise

3. Login to Roli. This will open a browser window.

roli login

Initialize the service

1. Init the service code directory

roli init-service chatbot -d chatbot/service

Write an endpoint class

1. Edit chatbot/service/config.ts to import the Endpoint class from the roli-runtime package.

chatbot/service/config.ts

import {Endpoint} from "./roli-runtime";

2. Write a class named ChatbotApi that extends the Endpoint class.

chatbot/service/config.ts

import {Endpoint} from "./roli-runtime";

export class ChatbotApi extends Endpoint {
    constructor(key: string) {
        super(key);
    }
}

At this point we have an Endpoint but it’s not very useful because there’s no methods for clients to call. We’ll get back to this after we create a Session.

Write a session class

1. Import the Session class from roli-runtime.

chatbot/service/config.ts

import {Endpoint, Session} from "./roli-runtime";

2. Write a class named ChatbotSession that extends the Session class.

chatbot/service/config.ts

import {Endpoint, Session} from "./roli-runtime";

export class ChatbotApi extends Endpoint {
    constructor(primaryKey: string) {
        super(primaryKey);
    }
}

export class ChatbotSession extends Session {
    userName: string | null;
    constructor(sessionId: string) {
        super(sessionId);
        this.userName = null;
    }
}

Add an endpoint method that returns a session

1. Add createSession to the list of imports

chatbot/service/config.ts

import {Endpoint, Session, createSession} from "./roli-runtime";

2. Add a method to ChatbotApi that creates the session, sets userName and returns the session.

chatbot/service/config.ts

import {Endpoint, Session, createSession} from "./roli-runtime";

export class ChatbotApi extends Endpoint {
    constructor(primaryKey: string) {
        super(primaryKey);
    }
    getSession(userName: string) : ChatbotSession {
        const session = createSession(ChatbotSession);
        session.userName = userName;
        return session;
    }
}

export class ChatbotSession extends Session {
    userName: string | null;
    constructor(sessionId: string) {
        super(sessionId);
        this.userName = null;
    }
}

Call the Model Registry

1. Add getModel to the list of imports

chatbot/service/config.ts

import {Endpoint, Session, createSession, getModel} from "./roli-runtime";

2. Add a method to the session that clients will call to interact with the model

chatbot/service/config.ts

export class ChatbotSession extends Session {
    userName: string | null;
    constructor(sessionId: string) {
        super(sessionId);
        this.userName = null;
    }

    async tell(message: string) : Promise<string | null> {

    }
}

3. Call the Model Registry to get the model

chatbot/service/config.ts

export class ChatbotSession extends Session {
    userName: string | null;
    constructor(sessionId: string) {
        super(sessionId);
        this.userName = null;
    }

    async tell(message: string) : Promise<string | null> {
        const model = getModel("my-model");
    }
}

Construct a basic executable prompt

1. Add ChatModelResponse, Prompt, and Program to the list of imports

chatbot/service/config.ts

import {Endpoint, Session, createSession,
    getModel, ChatModelResponse, Prompt, Program} from "./roli-runtime";

2. Create a prompt with a fun system message, a user message that comes from the message argument, and a callback to handle the output

chatbot/service/config.ts

export class ChatbotSession extends Session {
    userName: string | null;
    constructor(sessionId: string) {
        super(sessionId);
        this.userName = null;
    }

    async tell(message: string) : Promise<string | null> {
        const model = getModel("my-model");

        let result: string | null = null;
        const prompt = {
            system: `You are a member of an elite social club in London England. You are having a fun and interesting discussion with a long time friend named ${this.userName}. You must always be friendly, courtious, and respectful to ${this.userName}.`,
            user: message,
            assistant: (response: ChatModelResponse)=> {
                result = response.message;
                return result;
            }
        } as Prompt;
    }
}

3. Create a program passing the model and the prompt, execute it, and return the results.

chatbot/service/config.ts

export class ChatbotSession extends Session {
    userName: string | null;
    constructor(sessionId: string) {
        super(sessionId);
        this.userName = null;
    }

    async tell(message: string) : Promise<string | null> {
        const model = getModel("my-model");

        let result: string | null = null;
        const prompt = {
            system: `You are a member of an elite social club in London England. You are having a fun and interesting discussion with a long time friend named ${this.userName}. You must always be friendly, courtious, and respectful to ${this.userName}.`,
            user: message,
            assistant: (response: ChatModelResponse)=> {
                result = response.message;
                return result;
            }
        } as Prompt;

        const program = new Program(model, prompt);
        await this.execute(program);
        return result;
    }
}

Almost there…

At this point your chatbot backend is nearly functional. However, if you were to use it you’d find it isn’t able to carry on a converation- each thing you say would be replied to as if it were the first time you said anything. To remedy this we will make the chatbot keep track of what’s said and send that to the model with each request.

Make the chatbot conversational

1. Add Step and Instruction to the list of imports

chatbot/service/config.ts

import {Endpoint, Session, createSession,
    getModel, ChatModelResponse, Prompt, Program, Step, Instruction} from "./roli-runtime";

2. Add a property to ChatbotSession to hold the conversation history

chatbot/service/config.ts

export class ChatbotSession extends Session {
    private _history: Instruction[];
    userName: string | null;
    constructor(sessionId: string) {
        super(sessionId);
        this.userName = null;
        this._history = [];
    }

3. Modify the tell method to construct a list of steps for the program starting with this._history if it exists

chatbot/service/config.ts

export class ChatbotSession extends Session {
    private _history: Instruction[];
    userName: string | null;
    constructor(sessionId: string) {
        super(sessionId);
        this.userName = null;
        this._history = [];
    }

    async tell(message: string) : Promise<string | null> {
        const model = getModel("my-model");

        let steps: Step[];
        if(this._history) {
            steps = Array.from(this._history);
        } else {
            steps = [];
        }

        let result: string | null = null;
        const prompt = {
            system: `You are a member of an elite social club in London England. You are having a fun and interesting discussion with a long time friend named ${this.userName}. You must always be friendly, courtious, and respectful to ${this.userName}.`,
            user: message,
            assistant: (response: ChatModelResponse)=> {
                result = response.message;
                return result;
            }
        } as Prompt;

        const program = new Program(model, prompt);
        await this.execute(program);
        return result;
    }
}

4. Make it so the system message is only added to the first prompt

chatbot/service/config.ts

    async tell(message: string) : Promise<string | null> {
        const model = getModel("my-model");

        let steps: Step[];
        if(this._history) {
            steps = Array.from(this._history);
        } else {
            steps = [];
        }

        let result: string | null = null;
        const prompt = {
            system: `You are a member of an elite social club in London England. You are having a fun and interesting discussion with a long time friend named ${this.userName}. You must always be friendly, courtious, and respectful to ${this.userName}.`,
            user: message,
            assistant: (response: ChatModelResponse)=> {
                result = response.message;
                return result;
            }
        } as Prompt;

        if(this._history.length === 0) {
            prompt.system = `You are a member of an elite social club in London England. You are having a fun and interesting discussion with a long time friend named ${this.userName}. You must always be friendly, courtious, and respectful to ${this.userName}.`;
        }

        const program = new Program(model, prompt);
        await this.execute(program);
        return result;
    }

5. Pass the whole list of steps instead of just the single prompt to the program and add the newly created Instruction to the history after the Program’s execution.

chatbot/service/config.ts

    async tell(message: string) : Promise<string | null> {
        const model = getModel("my-model");

        let steps: Step[];
        if(this._history) {
            steps = Array.from(this._history);
        } else {
            steps = [];
        }

        let result: string | null = null;
        const prompt = {
            user: message,
            assistant: (response: ChatModelResponse)=> {
                result = response.message;
                return result;
            }
        } as Prompt;

        if(this._history.length === 0) {
            prompt.system = `You are a member of an elite social club in London England. You are having a fun and interesting discussion with a long time friend named ${this.userName}. You must always be friendly, courtious, and respectful to ${this.userName}.`;
        }

        steps.push(prompt);
        const program = new Program(model, steps);
        await this.execute(program);
        this._history.push(program.steps.peek() as Instruction);

        return result;
    }

Deploy the service

1. Use the Roli SDK to deploy the service from the service directory

# Deploys a new service version
roli deploy-service -d chatbot/service

Code generate a client package

1. Generate a client package so our client code can talk to the service we just deployed.

roli generate-client chatbot --project chatbot/client

2. Answer npm when asked what package manager to use.

Integrate service with the client code

1. In the file chatbot/client/src/index.ts import createRoliClient, ServiceOptions, and the roli client key.

chatbot/client/src/index.ts

#!/usr/bin/env node
import inquirer from 'inquirer';
import chalk from "chalk";
import { createRoliClient, ServiceOptions } from "chatbot-service";
import { key } from "chatbot-service/key";

(async () => {
  await inquirer
    .prompt([
      {
        type: 'input',
        name: 'userName',
        message: 'Login: ',
      },
    ])
    .then(async ({ userName }) => {

The default for Roli clients is to log lots of things we don’t want to see when using a CLI so we need to turn off verbose logging.

2. Create an instance of the roli client by calling createRoliClient passing the key and ServiceOptions.

chatbot/client/src/index.ts

#!/usr/bin/env node
import inquirer from 'inquirer';
import chalk from "chalk";
import { createRoliClient, ServiceOptions } from "chatbot-service";
import { key } from "chatbot-service/key";

const roli = createRoliClient(key, new ServiceOptions(false, false));

(async () => {
  await inquirer
    .prompt([
      {
        type: 'input',
        name: 'userName',
        message: 'Login: ',
      },
    ])
    .then(async ({ userName }) => {

3. Use the Roli client to get a reference to the ChatbotApi endpoint

chatbot/client/src/index.ts

#!/usr/bin/env node
import inquirer from 'inquirer';
import chalk from "chalk";
import { createRoliClient, ServiceOptions, ChatbotApi } from "chatbot-service";
import { key } from "chatbot-service/key";

const roli = createRoliClient(key, new ServiceOptions(false, false));
const chatbotApi = roli.getEndpoint(ChatbotApi, "default");

(async () => {
  await inquirer
    .prompt([
      {
        type: 'input',
        name: 'userName',
        message: 'Login: ',
      },
    ])
    .then(async ({ userName }) => {

4. Get the session from the endpoint, output the sessionId, and send the typed chat text to its tell method.

chatbot/client/src/index.ts

#!/usr/bin/env node
import inquirer from 'inquirer';
import chalk from "chalk";
import { createRoliClient, ServiceOptions, ChatbotApi } from "chatbot-service";
import { key } from "chatbot-service/key";

const roli = createRoliClient(key, new ServiceOptions(false, false));
const chatbotApi = roli.getEndpoint(ChatbotApi, "default");

(async () => {
  await inquirer
    .prompt([
      {
        type: 'input',
        name: 'userName',
        message: 'Login: ',
      },
    ])
    .then(async ({ userName }) => {
      console.log('Username: ' + userName);

      const session = await chatbotApi.getSession(userName);
      console.log(`You are speaking with a chatbot. Your session ID is ${session.sessionId}. Use /quit to exit chat.`);

      while (true) {
        await inquirer
          .prompt([
            {
              type: 'input',
              name: 'text',
              message: `${userName}: `,
            },
          ])
          .then(async ({ text }) => {
            if (text === '/quit') {
              process.exit(0);
              return;
            } else {
              const response = await session.tell(text);
              const timestamp = chalk.grey(
                `[${new Date().toLocaleTimeString()}]`
              );
              const userName = chalk.whiteBright(`[chatbot]:`);
              const t = chalk.greenBright(response);
              console.log(`${timestamp} ${userName} ${t}`);
            }
          });
      }
    });
})().catch((e: any) => {
  console.error('Unexpected failure: ' + e);
});

At this point both the service and client are close but there’s still one step left before it’s usable. We need to register an LLM with the Model Registry.

Register a ModelSpecification

Tip

You can find your OpenAI API key here

Caution

You may need to add credits to your OpenAI account.

1. Edit the file chatbot/model.json, add your API key.

chatbot/model.json

{
    "name": "gpt-3.5-turbo-1106",
    "url": "https://api.openai.com/v1/chat/completions",
    "settings": {
        "kind": "openai"
    },
    "apiKey": "YOUR API KEY"
}

2. Register the ModelSpecification with the Model Registry

roli register-model chatbot -k my-model -m chatbot/model.json

# expected output
OK: Model registered

Test it all out

1. Build and run the chatbot example client.

cd chatbot/client
npm run build
npm run chat

2. Login with your name and then say hello

? Login:  Scott

# expected output

Username: Scott
You are speaking with a chatbot. Your session ID is f88b0842-71c6-4dc0-b48c-d09570656d99. Use /quit to exit chat.
? Scott:  Hello
[8:42:00 AM] [chatbot]: Hello! How are you today, Scott?

Conclusion

You’ve created a conversational AI chatbot and deployed it to a service running on the Roli platform. This service is versatile, and could be used from any web UI or microservice using similar client code.