Skip to content

Create a Chatbot

What you’re building

This tutorial prescribes how to build a versatile enterprise grade chatbot backend. This includes a CLI client Node.js application but could equally be integrated with a web site UI or a microservice.

Prerequisites

Create the directory structure

  1. Create two directories.
Terminal window
# create the directories we'll need for this tutorial
mkdir chatbot
mkdir chatbot/service

Set Roli’s connection information

  1. Run the command to set Roli’s connection information so it can talk to the backend platform installation.
Terminal window
roli set-connection-info admin=https://admin.roli.app api=https://api.roli.app --enterprise

Login to the Roli SDK

  1. Run the command to login to the Roli SDK
Terminal window
roli login

Initialize the service

  1. Init the service code directory using roli init-service
Terminal window
# initialize the service directory
roli init-service chatbot -d chatbot/service
# example output
Success: The "chatbot" service has been initialized in the directory "/home/scott/chatbot/service".

Open your text editor and create the index.ts file

  1. Open the chatbot folder in your editor (e.g. VSCode):
Terminal window
# Open the chatbot folder if you're using VSCode
code chatbot
  1. Create a new file in the service directory named index.ts
  • Your editor window should look like this: index-screenshot

Write an endpoint class

  1. Edit chatbot/service/index.ts to import the Endpoint class from the roli-runtime package.
chatbot/service/index.ts
import {Endpoint} from "./roli-runtime";
  1. Write a class named ChatbotApi that extends the Endpoint class.
chatbot/service/index.ts
import {Endpoint} from "./roli-runtime";
export class ChatbotApi extends Endpoint {
constructor(key: string) {
super(key);
}
}

At this point we have an Endpoint but it’s not very useful because there’s no methods for clients to call. We’ll get back to this after we create a Session.

Write a session class

  1. Import the Session class from roli-runtime.
chatbot/service/index.ts
import {Endpoint, Session} from "./roli-runtime";
  1. Write a class named ChatbotSession that extends the Session class.
chatbot/service/index.ts
import {Endpoint, Session} from "./roli-runtime";
export class ChatbotApi extends Endpoint {
constructor(primaryKey: string) {
super(primaryKey);
}
}
export class ChatbotSession extends Session {
userName: string | null;
constructor(sessionId: string) {
super(sessionId);
this.userName = null;
}
}

Add an endpoint method that returns a session

  1. Add createSession to the list of imports
chatbot/service/index.ts
import {Endpoint, Session, createSession} from "./roli-runtime";
  1. Add a method to ChatbotApi that creates the session, sets userName and returns the session.
chatbot/service/index.ts
import {Endpoint, Session, createSession} from "./roli-runtime";
export class ChatbotApi extends Endpoint {
constructor(primaryKey: string) {
super(primaryKey);
}
getSession(userName: string) : ChatbotSession {
const session = createSession(ChatbotSession);
session.userName = userName;
return session;
}
}
export class ChatbotSession extends Session {
userName: string | null;
constructor(sessionId: string) {
super(sessionId);
this.userName = null;
}
}

Call the Model Registry

  1. Add getModel to the list of imports
chatbot/service/index.ts
import {Endpoint, Session, createSession, getModel} from "./roli-runtime";
  1. Add a method to the session that clients will call to interact with the model
chatbot/service/index.ts
export class ChatbotSession extends Session {
userName: string | null;
constructor(sessionId: string) {
super(sessionId);
this.userName = null;
}
async tell(message: string) : Promise<string | null> {
}
}
  1. Call the Model Registry to get the model
chatbot/service/index.ts
export class ChatbotSession extends Session {
userName: string | null;
constructor(sessionId: string) {
super(sessionId);
this.userName = null;
}
async tell(message: string) : Promise<string | null> {
const model = getModel("my-model");
}
}

Construct a basic executable prompt

  1. Add ChatModelResponse, Prompt, and Program to the list of imports
chatbot/service/index.ts
import {Endpoint, Session, createSession,
getModel, ChatModelResponse, Prompt, Program} from "./roli-runtime";
  1. Create a prompt with a fun system message, a user message that comes from the message argument, and a callback to handle the output
chatbot/service/index.ts
export class ChatbotSession extends Session {
userName: string | null;
constructor(sessionId: string) {
super(sessionId);
this.userName = null;
}
async tell(message: string) : Promise<string | null> {
const model = getModel("my-model");
let result: string | null = null;
const prompt = {
system: `You are a member of an elite social club in London England. You are having a fun and interesting discussion with a long time friend named ${this.userName}. You must always be friendly, courtious, and respectful to ${this.userName}.`,
user: message,
assistant: (response: ChatModelResponse)=> {
result = response.choices[0].message.content;
return result;
}
} as Prompt;
}
}
  1. Create a program passing the model and the prompt, execute it, and return the results.
chatbot/service/index.ts
export class ChatbotSession extends Session {
userName: string | null;
constructor(sessionId: string) {
super(sessionId);
this.userName = null;
}
async tell(message: string) : Promise<string | null> {
const model = getModel("my-model");
let result: string | null = null;
const prompt = {
system: `You are a member of an elite social club in London England. You are having a fun and interesting discussion with a long time friend named ${this.userName}. You must always be friendly, courtious, and respectful to ${this.userName}.`,
user: message,
assistant: (response: ChatModelResponse)=> {
result = response.choices[0].message.content;
return result;
}
} as Prompt;
const program = new Program(model, prompt);
await this.execute(program);
return result;
}
}

Make the chatbot conversational

  1. Add Step and Instruction to the list of imports
chatbot/service/index.ts
import {Endpoint, Session, createSession,
getModel, ChatModelResponse, Prompt, Program, Step, Instruction} from "./roli-runtime";
  1. Add a property to ChatbotSession to hold the conversation history
chatbot/service/index.ts
export class ChatbotSession extends Session {
private _history: Instruction[];
userName: string | null;
constructor(sessionId: string) {
super(sessionId);
this.userName = null;
this._history = [];
}
  1. Modify the tell method to construct a list of steps for the program starting with this._history if it exists
chatbot/service/index.ts
export class ChatbotSession extends Session {
private _history: Instruction[];
userName: string | null;
constructor(sessionId: string) {
super(sessionId);
this.userName = null;
this._history = [];
}
async tell(message: string) : Promise<string | null> {
const model = getModel("my-model");
let steps: Step[];
if(this._history) {
steps = Array.from(this._history);
} else {
steps = [];
}
let result: string | null = null;
const prompt = {
system: `You are a member of an elite social club in London England. You are having a fun and interesting discussion with a long time friend named ${this.userName}. You must always be friendly, courtious, and respectful to ${this.userName}.`,
user: message,
assistant: (response: ChatModelResponse)=> {
result = response.choices[0].message.content;
return result;
}
} as Prompt;
const program = new Program(model, prompt);
await this.execute(program);
return result;
}
}
  1. Make it so the system message is only added to the first prompt
chatbot/service/index.ts
async tell(message: string) : Promise<string | null> {
const model = getModel("my-model");
let steps: Step[];
if(this._history) {
steps = Array.from(this._history);
} else {
steps = [];
}
let result: string | null = null;
const prompt = {
system: `You are a member of an elite social club in London England. You are having a fun and interesting discussion with a long time friend named ${this.userName}. You must always be friendly, courtious, and respectful to ${this.userName}.`,
user: message,
assistant: (response: ChatModelResponse)=> {
result = response.choices[0].message.content;
return result;
}
} as Prompt;
if(this._history.length === 0) {
prompt.system = `You are a member of an elite social club in London England. You are having a fun and interesting discussion with a long time friend named ${this.userName}. You must always be friendly, courtious, and respectful to ${this.userName}.`;
}
const program = new Program(model, prompt);
await this.execute(program);
return result;
}
  1. Pass the whole list of steps instead of just the single prompt to the program and add the newly created Instruction to the history after the Program’s execution.
chatbot/service/index.ts
async tell(message: string) : Promise<string | null> {
const model = getModel("my-model");
let steps: Step[];
if(this._history) {
steps = Array.from(this._history);
} else {
steps = [];
}
let result: string | null = null;
const prompt = {
user: message,
assistant: (response: ChatModelResponse)=> {
result = response.choices[0].message.content;
return result;
}
} as Prompt;
if(this._history.length === 0) {
prompt.system = `You are a member of an elite social club in London England. You are having a fun and interesting discussion with a long time friend named ${this.userName}. You must always be friendly, courtious, and respectful to ${this.userName}.`;
}
steps.push(prompt);
const program = new Program(model, steps);
await this.execute(program);
this._history.push(program.steps.peek() as Instruction);
return result;
}

Deploy the service

  1. From a terminal, use the Roli SDK to deploy the service from the service directory
Terminal window
# Deploys a new service version
roli deploy-service -d chatbot/service
# example output
OK: The "chatbot" service is now live at version 1.

Download the client example code

  1. From a terminal, download the client code from GitHub and make sure the download worked
Terminal window
# Copies the example client code into the chatbot/client directory.
npx tiged roliai/tutorial/chatbot/client chatbot/client
  1. Verify the download worked
Terminal window
# Check to make sure it's there
dir chatbot/client
# this is what you should see
LICENSE package.json src tsconfig.json

Code generate a client package

  1. From a terminal, generate a client package so our client code can talk to the service we just deployed.
Terminal window
# code generate a package that lets the client talk to the service
roli generate-client chatbot -d chatbot/client
# example output
OK: A new code generated client was created for the chatbot service version 1 in the project located at chatbot/client.
  1. Answer npm when asked what package manager to use.

Integrate service with the client code

  1. In the file chatbot/client/src/index.mts import the createRoliClient function and call it to get a client object.
chatbot/client/src/index.mts
#!/usr/bin/env node
import inquirer from 'inquirer';
import chalk from "chalk";
import { createRoliClient } from "chatbot-service";
const roli = createRoliClient();
await inquirer.prompt([{
type: 'input',
name: 'userName',
message: "Login: "
}]).then(async ({userName}) => {

The default for Roli clients is to log lots of things we don’t want to see when using a CLI so we need to turn off verbose logging.

  1. Import ServiceOptions and pass it to createRoliClient
chatbot/client/src/index.mts
#!/usr/bin/env node
import inquirer from 'inquirer';
import chalk from "chalk";
import { ServiceOptions } from 'roli-client';
import { createRoliClient } from "chatbot-service";
const roli = createRoliClient(new ServiceOptions(false, false));
await inquirer.prompt([{
type: 'input',
name: 'userName',
message: "Login: "
}]).then(async ({userName}) => {
  1. Use the Roli client to get a reference to the ChatbotApi endpoint
chatbot/client/src/index.mts
#!/usr/bin/env node
import inquirer from 'inquirer';
import chalk from "chalk";
import { ServiceOptions } from 'roli-client';
import { createRoliClient, ChatbotApi } from "chatbot-service";
const roli = createRoliClient(new ServiceOptions(false, false));
const chatbotApi = roli.getEndpoint(ChatbotApi, "default");
await inquirer.prompt([{
type: 'input',
name: 'userName',
message: "Login: "
}]).then(async ({userName}) => {
  1. Get the session from the endpoint, output the sessionId, and send the typed chat text to its tell method.
chatbot/client/src/index.mts
#!/usr/bin/env node
import inquirer from 'inquirer';
import chalk from "chalk";
import {ServiceOptions} from 'roli-client';
import {createRoliClient, ChatbotApi } from "chatbot-service";
const roli = createRoliClient(new ServiceOptions(false, false));
const chatbotApi = roli.getEndpoint(ChatbotApi, "default");
await inquirer.prompt([{
type: 'input',
name: 'userName',
message: "Login: "
}]).then(async ({userName}) => {
console.log("Username: " + userName);
const session = await chatbotApi.getSession(userName);
console.log(`You are speaking with a chatbot. Your session ID is ${session.sessionId}. Use /quit to exit chat.`);
while (true) {
await inquirer.prompt([{
type: 'input',
name: 'text',
message: `${userName}: `
}]).then(async ({text}) => {
if (text === "/quit") {
process.exit(0);
return;
} else {
const response = await session.tell(text);
const timestamp = chalk.grey(`[${new Date().toLocaleTimeString()}]`);
const userName = chalk.whiteBright(`[chatbot]:`);
const t = chalk.greenBright(response);
console.log(`${timestamp} ${userName} ${t}`);
}
})
}
})

At this point both the service and client are close but there’s still one step left before it’s usable.

Register a ModelSpecification

  1. Create a new file named chatbot/model.json with this content.
chatbot/model.json
{
"name": "gpt-3.5-turbo-1106",
"url": "https://api.openai.com/v1/chat/completions",
"settings": {
"kind": "chat-model"
},
"apiKey": "YOUR API KEY"
}
  1. Register the ModelSpecification with the Model Registry
Terminal window
roli register-model chatbot -k my-model -m chatbot/model.json
# expected output
OK: Model registered

Chatting with a new friend

  1. Build and run the chatbot client
Terminal window
cd chatbot/client
npm run build
npm run chat
# expected output
> roli-chatbot@1.0.0 chat
> node dist/index.mjs
? Login:
  1. Login with your name and then say hello
Terminal window
? Login: Scott
# expected output
Username: Scott
You are speaking with a chatbot. Your session ID is f88b0842-71c6-4dc0-b48c-d09570656d99. Use /quit to exit chat.
? Scott: Hello
[8:42:00 AM] [chatbot]: Hello! How are you today, Scott?

Conclusion

You’ve built an enterprise-grade chatbot backend with a CLI front-end. This backend service is versatile, and could be used from any web or microservice using similar client code.