How To Build An AI Chatbot: A Comprehensive Guide
AI chatbots took the world by storm. After the spectacular success of ChatGPT, Google, Microsoft, and other tech giants rushed to release their own versions of conversational AI bots.
Yet, all AI chatbots have one limitation when it comes to business use – they can’t provide specific and technical information that is either niche or not available to the public.
AI knowledge bases are designed to solve this problem. An AI knowledge base allows developers to integrate the capacities of AI models with a custom knowledge base – a database that contains business-specific documents or other types of unique data.
In this article, we share our experience in AI chatbot development and explain how to build an AI chatbot with custom knowledge base by using a real-life case study.
Let’s get started.
What are AI chatbots?
AI chatbot assistants are primarily used to provide faster and cheaper customer service or easily retrieve internal information from company databases. AI chatbots can also be used to simplify internal processes such as onboarding and knowledge-sharing.
Types of chatbots
There are several main types of chatbots that can be used for business purposes.
Rule-based chatbots
Rule-based chatbots are the simplest type of chatbot. As the name implies, their responses are predefined based on a set of rules. These bots can only respond to specific commands and they do not possess the ability to understand context or intent.
A rule-based chatbot can provide a response when the word or a phrase in the query triggers it. Additionally, the word or phrase should be set within specific language parameters.
An example would be a pizza ordering bot that responds to specific input like "Order a large pepperoni pizza". Luckily, there are plenty of chatbot builders available on the market which makes creating rule-based chatbots an easy and pleasant experience.
Machine learning-based AI chatbots
ML-based AI chatbots use machine learning and natural language processing to understand, learn and respond to user queries. They do not rely on hardcoded responses but instead learn from previous interactions.
The main difference is the capacity to respond to a larger diversity of queries – ML-based bots can generate a variety of specific responses by recognizing an intent and context of the input. ML-based chatbots are also more convenient, since they can save data from past conversations, such as customer’s shipping address or billing info. This enables these chatbots to engage in more natural, human-like conversations over time.
Hybrid chatbots
Hybrid chatbots combine elements from both rule-based and conversational AI chatbots. These bots can follow predefined rules for simple queries while using ML for more complex tasks.
Voice-based chatbots
Voice-based chatbots use speech-to-text and text-to-speech functions together with AI and Natural Language Processing. Voice-based chatbots allow the user to get a response by speaking out the request instead of typing. They work similarly to ML-based bots – but with an added technology that allows the bot to convert the speech into textual segments to understand user intent and convert the textual response to speech to produce an answer.
As the AI chatbots become smarter, more affordable, and easier to develop, rule-based chatbots are gradually becoming a thing of the past. Here are the key technologies that enable quick, efficient, and affordable AI chatbot development.
Architectural components of AI chatbots
AI-based chatbots are based on several technologies and components that enable them to recognize user queries and maintain dialogue with the user.
Natural Language Processing
Natural Language Processing is a technology that uses the principles of computational linguistics together with machine learning to understand and generate text and speech. NLP allows chatbot AI agents to understand user intent, summarize large volumes of text, and generate text or speech on demand.
The key processes underlying NLP are focused on simplification of text – since texts consist of unstructured data, machines struggle to decipher the intended meaning behind the text.
The key NLP processes used to make the text understandable for chatbots include:
- Text classification, which enables the chatbot to learn associations between different pieces of text and understand that a particular textual output is expected from a particular input. In order to do that, the NLP engine transforms the text into a numerical representation called a vector, and, after being trained on pieces of data, learns to classify text into predetermined categories.
- Tokenization is used to further simplify the process for the chatbot. Tokenization breaks down large chunks of text into smaller units, known as tokens. Each token typically represents a word or phrase, which makes it easier for the bot to analyze each piece of information in isolation.
- Part-of-speech tagging is used in NLP to understand the grammatical links between words used in user inputs. POS allows the system to identify whether a word is being used as a noun, verb, adjective, etc. This enables the chatbot to comprehend the whole sentence instead of the meaning of a single word.
- Named entity recognition. Often users' inputs contain key information such as names, dates, locations or specific products/services. Named entity recognition categorizes words into semantic groups. It helps the bot to understand that “Estonia” is a country, and “Dan” is a name.
- Lemmatization and stemming reduce words to their base or root form (e.g., 'smiling' to 'smile'), which makes it easier for the chatbot to process user input.
- Sentiment analysis allows the bot to understand the emotional undertone of the message – whether it’s positive, negative, or neutral. Advanced types of sentiment analysis can also detect particular feelings and emotions (angry, happy, sad) as well as the level of urgency and intentions.
- Word sense disambiguation allows the bot to understand words with multiple meanings based on what makes sense in a particular context. For example, it enables the chatbot to distinguish whether the word “star” refers to an entity in the sky or to a famous personality.
NLP is a core component of AI chatbots that enables them to decipher user intents and respond quickly and accurately.
Large Language Models
Large Language Model is a type of AI that allows the chatbot to recognize and generate text.
LLMs are built through machine learning – specifically, by training a type of a neural network called a transformer model. These models use self-attention mechanisms to weigh the importance of different words in a sentence, which allows them to understand context better and generate useful text. Most LLMs are trained on an extremely large amount of data that was gathered from the internet.
LLMs are capable of performing all NLP tasks, such as tokenization and part-of-speech tagging, and are considered an advanced type of an NLP model. They are also adaptable and can be used for a variety of apps that previously relied on specialized NLP models.
This makes LLMs an ideal solution for AI-based enterprise chatbots. The main benefit of using LLM for chatbots is time and money – an LLM provides an already-trained language model that can be fine-tuned for particular tasks and integrated with a custom knowledge base to retrieve company-specific data.
This eliminates an initial step of training a language model from scratch – a costly and lengthy process that is unnecessary for the level of complexity of most chatbots.
Some examples of LLM include:
- GPT. The most well-known LLM is GPT, which underlies the famous ChatGPT – a chatbot that kickstarted the AI adoption by businesses and individual users. GPT also underlies Microsoft’s chatbot Bing.
- LaMDA. LaMDA powers Google’s Bard chatbot – a competitor to ChatGPT launched in 2023.
- LLaMA. This LLM was developed by Meta AI and released in February 2023.
The choice of LLM for development of a specific chatbot depends on multiple factors, such as data privacy, chat format, and others. A software development team will test various LLM models to determine which one fits your use case the best.
Chatbot knowledge base
An AI knowledge base is a base of information the chatbot needs to use to provide the user with detailed answers.
While all LLMs are trained on extremely large sets of data, they are not omniscient. For instance, GPT-3.5 is only aware of information that was accessible on the Internet until September 2021. If you’ve ever used ChatGPT, you might have noticed that it struggles with answering highly specific or detailed queries – that is a problem of all LLMs.
Therefore, if you need your chatbot to have access to specific data of your company or other technical information (for instance, internal proposals, project reports, blueprints, etc) you will need to create a custom AI chatbot knowledge base, connect the LLM to it, and train the chatbot to retrieve information from it.
One effective technology that enables developers to connect LLMs to custom knowledge bases is called RAG – Retrieval-Augmented Generation.
Essentially, RAG allows users to have conversations with data repositories. For instance, one can connect an LLM to a medical index to build an AI assistant for doctors and nurses. Or connect the LLM to market data to create a financial chatbot assistant. The advantages of this are crystal clear – a user will be able to get industry-specific or internal data with a solid guarantee of accuracy in a few seconds.
In the next section, we’ll describe how you can develop an AI chatbot with a custom knowledge base by effectively using RAG technology.
How to build an AI chatbot with RAG: a case study
Our client is a full life cycle construction service provider. The client reached out to Apiko to develop a chatbot that would facilitate effective access to historical data within their services, including proposals, project reports, blueprints, invoices, and more.
The team has chosen to use RAG technology to create a custom knowledge base that would contain the company’s data, and integrate it with an LLM to produce a fast and efficient chatbot capable of responding to company-specific inquiries. Here is a graph that describes the architecture of the solution.
The process of development consisted of several steps: selection of an LLM model, selection of a model for embeddings, and development of a knowledge base that would be integrated with a selected LLM.
LLM model selection
Since the company’s requirement was development on their own infrastructure, it was not possible to use models like OpenAI's GPT. OpenAI stores all data for 30 days, which was not acceptable for our client due to privacy concerns. Indeed, if the model is working with data that should not be publicly available, you need to choose a model that allows data to be hosted on private servers.
Accordingly, after analyzing several modern LLMs, the team chose Mistral, an LLM that is fine-tuned for the question-answer chat format. It is important to choose a model that is already fine-tuned for specific tasks (question-answer chats, voice-based chats, etc).
On top of that, the best model for your specific solution will be determined by experimental testing. For instance, our team also tested LLaMA 2 model, but the results did not meet our standards.
Selection of an embeddings model
As already mentioned, neural networks do not understand text, and only know how to work with numbers. Therefore, in order for the LLM to be able to work with your documents, they should be converted into numeric vectors of a certain dimension that refer to a particular text. For instance, HELLO -> [0.23, 0.001, 0.707].
When words are converted into vectors, they become comprehensible and can be processed by a neural network. It is not known in advance which word will correspond to a certain vector – the neural network “learns” the relationships between words during the training.
To enable this, developers need a sentence-transformers model. For instance, all-mini-16 model maps sentences and paragraphs to a 384 dimensional dense vector space and is often also used for tasks like clustering or semantic search.
Custom knowledge base development
After the models were chosen, the team created a custom knowledge base for an AI chatbot. Here are the steps that were taken by the team:
- Collection and preparation of the knowledge base. First, documents were collected and prepared. In our case, these were PDF documents, but, in general, you can use various types of documents, including pdf, docx, txt, and others.
- Chunking. Since some documents can be large and contain a lot of text, each document was divided into sectors (chunks) with a fixed number of characters. Typically, the sector contains 1024 characters. However, the exact number depends on the task: if the sector size is very small, the model will not be able to get enough information. If the sector size is large, the model may struggle with low speed of retrieval and low accuracy of answers. In general, the size should be neither too small nor too large.
- Creation and storage of embeddings. Next, you need to create embeddings and store the obtained numerical vectors in a specialized vector database. These databases are intended specifically for storing embeddings. Popular solutions include FAISS, Chroma, and Pinecone.
Let’s keep in touch
Programming a chatbot
After that, everything is combined into a chatbot by using Python, HuggingFace Transformers, LLama.cpp, and other frameworks.
In the end, the chatbot works in the following manner:
- User request. First, the user types in a request to the chatbot.
- Converting. Since the question is a text, the system uses an embedding model to convert the request into a numeric vector.
- Finding relevant documents. Next, using the similarity search method, the system searches for similar documents in the knowledge base and selects a number of relevant documents.
- Making a request to LLM. After finding the documents, the system asks the same question to the LLM model in a prompt, and places the relevant documents within the prompt.
- A response. The system responds to the user inquiry based on the information in the relevant documents.
The user experience remains identical to a regular chatbot – the user types in a question, and immediately gets an answer.
If you want to learn more about how our team managed to increase the accuracy of AI assistants working with custom databases, check out this article on AI SQL query optimization.
How to build an AI chatbot: conclusion
AI-based chatbots with custom knowledge bases are an excellent alternative to both outdated rule-based chatbots and costly and time-consuming custom neural networks. They enable businesses to leverage capacities of the strongest Large Language Models on the market while integrating them with custom knowledge bases that contain company-specific information. This makes enterprise AI chatbot solutions affordable enough for medium- and small-sized businesses.
Apiko offers professional AI chatbot development services. As an AI software development company, we are ready to use AI technologies to optimize your internal processes or create an AI chatbot for your customers.
Also, we offer mobile app development services, marketplace development, and more – book a consultation to find out whether our expertise can help to turn your idea into reality!