How To Build An AI Chatbot: A Comprehensive Guide
AI chatbots took the world by storm. After ChatGPT was released in 2022 and reached 180 million users in 2024, Google, Microsoft, and other tech giants struggled to release their own versions of conversational AI bots.
Yet, ChatGPT and similar bots have at least one limitation when it comes to business use – they can’t provide specific and technical information that is either niche or not available to the public.
This is where AI-powered knowledge bases come in. After the grand successes of ChatGPT and other Large Language Models, a new technology called RAG was created. It allows developers to integrate the outstanding capacities of these models with a custom knowledge base that contains the documents a business needs without compromising the confidentiality of data.
In this article, we explain how to build an AI chatbot with custom knowledge base by using a case study of a construction company.
Let’s get started.
What are AI chatbots?
Types of chatbots
There are several main types of chatbots one can develop and use effectively.
Rule-based chatbots
Rule-based chatbots are the simplest type of chatbot. As the name implies, their responses are predefined based on a set of rules. These bots can only respond to specific commands and they do not possess the ability to understand context or intent. A rule-based chatbot can provide a response when the word or a phrase in the query triggers it. The bot will only provide an answer if there is a word or phrase within specific language parameters. An example would be a pizza ordering bot that responds to specific input like "Order a large pepperoni pizza". Luckily, there are plenty of chatbot builders available on the market which makes creating rule-based chatbots an easy and pleasant experience.
Machine learning-based AI chatbots
Machine Learning-Based AI Chatbots use Machine Learning (ML) and Natural Language Processing to understand, learn and respond to user queries. They do not rely on hardcoded responses but instead learn from previous interactions. The main difference is the capacity to respond to a larger diversity of queries – ML-based bots can generate a variety of specific responses by recognizing an intent and context of the input. ML-based chatbots are also more convenient, since they can save data from past conversations, such as customer’s shipping address or billing info. This enables these chatbots to engage in more natural, human-like conversations over time.
Hybrid chatbots
Hybrid chatbots combine elements from both rule-based and conversational AI chatbots. These bots can follow predefined rules for simple queries while using ML for more complex tasks.
Voice-based chatbots
Voice-based chatbots are a type of AI-based chatbots that use speech-to-text and text-to-speech functions together with AI and Natural Language Processing. They are convenient since they don’t require typing anything – you can just speak out your request. They work in an essentially similar way – but with an added technology that allows the bot to convert the speech into textual segments to understand user intent and convert the textual response to speech to produce an answer.
As the AI chatbots become smarter, more affordable, and easier to develop, rule-based chatbots are gradually becoming a thing of the past. Here are the key technologies that enable quick and efficient AI chatbot development.
Architectural components of AI chatbots
AI-based chatbots are based on several key technologies and components that enable them to recognize user queries and maintain dialogue with the user.
Natural Language Processing
Natural Language Processing is a technology that uses the principles of computational linguistics together with machine learning to understand and generate text and speech. NLP allows AI chatbots to understand user intent, summarize large volumes of text, and generate text or speech on demand.
The key processes underlying NLP are focused on simplification of text – since texts consist of unstructured data, machines struggle to decipher the intended meaning behind the text. The key NLP processes used to make the text understandable for chatbots include:
- Text Classification. Text classification enables the chatbot to learn associations between different pieces of text and understand that a particular textual output is expected from a particular input. In order to do that, the NLP engine transforms the text into a numerical representation called a vector, and, after being trained on pieces of data, learns to classify text into predetermined categories.
- Tokenization. To further simplify the process for the chatbot, tokenization is used. Tokenization breaks down large chunks of text into smaller units, known as tokens. Each token typically represents a word or phrase, which makes it easier for the bot to analyze each piece of information in isolation.
- Part-of-Speech Tagging. In order to understand the grammatical links between words used in user inputs, NLP uses part-of-speech tagging. POS allows the system to identify whether a word is being used as a noun, verb, adjective, etc. This enables the chatbot to comprehend the whole sentence instead of the meaning of a single word.
- Named Entity Recognition. Often users' inputs contain key information such as names, dates, locations or specific products/services. Named entity recognition categorizes words into semantic groups. It helps the bot to understand that “Estonia” is a country, and “Dan” is a name.
- Lemmatization and Stemming. Lemmatization and stemming reduce words to their base or root form (e.g., 'smiling' to 'smile'), which makes it easier for the chatbot to process user input.
- Sentiment Analysis. Sentiment analysis allows the bot to understand the emotional undertone of the message – whether it’s positive, negative, or neutral. Advanced types of sentiment analysis can also detect particular feelings and emotions (angry, happy, sad) as well as the level of urgency and intentions.
- Word sense disambiguation. This technology allows the bot to understand words with multiple meanings based on what makes sense in a particular context. For example, it enables the chatbot to distinguish whether the word “star” refers to an entity in the sky or to a famous personality.
NLP is a core component of AI chatbots that enables them to decipher user intents and respond quickly and accurately.
Large Language Models
Large Language model is a type of AI program that allows your chatbot to recognize and generate text. They are built on Machine Learning – specifically, on a type of a neural network called a transformer model. These models use self-attention mechanisms to weigh the importance of different words in a sentence, which allows them to understand context better and generate useful text. Most LLMs are trained on an extremely large amount of data that was gathered from the internet.
Large Language Models are capable of performing all NLP tasks, such as tokenization and part-of-speech tagging, and are considered an advanced type of an NLP model.
LLMs are adaptable and can be used for a variety of apps that previously relied on specialized NLP models – which is why they are now commonly used for AI-based enterprise chatbots. The main benefit of using LLM for chatbots is time and money – an LLM provides an already-trained language model that can be fine-tuned for particular tasks and integrated with a custom knowledge base to retrieve company-specific data.
This eliminates an initial step of training a language model from scratch – a costly and lengthy process that is unnecessary for the level of complexity of most chatbots.
Some examples of LLM include:
- GPT. The most well-known LLM is GPT, which underlies the famous ChatGPT – a chatbot that has already transformed the AI landscape with its unique capabilities of fast and human-like responses. GЗЕ also underlies Microsoft’s chatbot Bing.
- LaMDA. LaMDA powers Google’s Bard chatbot – a competitor to ChatGPT launched in 2023.
- LLaMA. This is an LLM that was developed by Meta AI and released in February 2023.
The choice of LLM for development of a specific chatbot depends on multiple factors, such as data privacy, chat format, and others. Your software development team will test various LLM models to determine which one fits your use case the best.
Chatbot knowledge base
An AI knowledge base is a base of information that the chatbot will need to use to provide you with detailed answers. While LLMs are trained on extremely large sets of data, they are not omniscient. For instance, GPT-3.5 is only aware of information that was accessible on the Internet until September 2021. If you’ve ever used ChatGPT, you might have noticed that it struggles with answering highly specific or detailed queries – that is a problem of all LLMs.
Therefore, if you need your chatbot to have access to specific data of your company or other technical information (for instance, internal proposals, project reports, blueprints, etc) you will need to create a custom AI chatbot knowledge base, connect the LLM to it, and train the chatbot to retrieve information from it.
One effective technology that enables developers to connect LLMs to custom knowledge bases is called RAG – Retrieval-Augmented Generation.
Essentially, RAG allows users to have conversations with data repositories. For instance, one can connect an LLM to a medical index to build an AI assistant for doctors and nurses. Or connect the LLM to market data to create a financial chatbot assistant. The advantages of this are crystal clear – a user will be able to get industry-specific or internal data with a solid guarantee of accuracy in a few seconds.
In the next section, we’ll describe how you can develop an AI chatbot with a custom knowledge base with RAG technology.
How to build an AI chatbot with RAG: a case study
Our client is a full life cycle construction service provider. From initial consultations to post-occupancy and ongoing maintenance, our client tailors their services to meet the needs of their clients and ensure successful completion of the project. The client reached out to Apiko to develop a chatbot that would facilitate effective access to historical data within their services, including proposals, project reports, blueprints, invoices, and more.
The team has chosen to use RAG technology to create a custom knowledge base that would contain the company’s data, and integrate it with an LLM to produce a fast and efficient chatbot capable of responding to company-specific inquiries. Here is a graph that describes the architecture of the solution.
The process of development consisted of several steps: selection of an LLM model, selection of a model for embeddings, and development of a knowledge base that would be integrated with a selected LLM.
LLM model selection
Since the company’s requirement was development on their own infrastructure, it was not possible to use models like OpenAI's GPT. Indeed, it is recommended to host everything on private servers, if the model will be working with data that should not be publicly available. OpenAI, for example, stores all data for 30 days, which was not acceptable for our client.
Accordingly, after analyzing several modern LLMs, Mistral was chosen, which is particularly fine-tuned for the question-answer chat format. It is important to choose a model for specific tasks, which should be a decision made by your development team. For instance, our team has also tested LLaMA 2 model, however the results did not meet our standards.
Selection of an embeddings model
As already mentioned, neural networks do not understand text, and only know how to work with numbers. Therefore, in order for the LLM to be able to work with your documents, they should be converted into numeric vectors of a certain dimension that refer to a particular text. For instance, HELLO -> [0.23, 0.001, 0.707].
When words are converted into vectors, they become comprehensible and can be processed by a neural network. It is not known in advance which word will correspond to a certain vector – the neural network “learns” the relationships between words during the training.
To enable this, developers need a sentence-transformers model. For instance, the all-mini-16 model maps sentences and paragraphs to a 384 dimensional dense vector space and is often also used for tasks like clustering or semantic search.
Custom knowledge base development
After the models have been chosen, the team created a custom knowledge base for an AI chatbot. Here are the steps that were taken by the team:
- Collection and preparation of the knowledge base. First, documents were collected and prepared. In our case, these were PDF documents, but in general these can be various types of documents - pdf, docx, txt, and many others.
- Chunking. Since some documents can have a lot of text and a large size, each document is divided into sectors (chunks) with a fixed number of characters. Most often, 1024 is used. But this parameter should be selected for a specific task. If the size is very small, then the model will not be able to get enough information. Similarly, a very large sector size negatively affects the speed and accuracy of answers. In general, the size should be neither too small nor too large.
- Creation and storage of embeddings. Next, embeddings are created, and the obtained numerical vectors are stored in a specialized vector database, which is intended specifically for storing embeddings. Popular solutions include FAISS, Chroma, and Pinecone.
Let’s keep in touch
Programming a chatbot
After that, everything is combined into a chatbot by using Python, HuggingFace Transformers, LLama.cpp, and other frameworks. In the end, the chatbot works in the following manner:
- User request. First, the user types in a request to the chatbot.
- Converting. Since the question is a text, the system uses an embedding model to convert the request into a numeric vector.
- Finding relevant documents. Next, using the similarity search method, the system searches for similar documents in the knowledge base. It selects a number of relevant documents from the knowledge base.
- Making a request to LLM. After finding the documents, the system asks the same question to the LLM model in a prompt, and also places the relevant documents within the prompt.
- A response. The system responds to the user inquiry based on the information in the relevant documents.
The user experience, however, is identical to a regular chat bot – the user types in a question, and immediately gets an answer.
How to build an AI chatbot: conclusion
AI-based chatbots with custom knowledge bases are an excellent alternative to both outdated rule-based chatbots and costly and time-consuming custom neural networks. They enable businesses to leverage capacities of the strongest Large Language Models on the market while integrating them with custom knowledge bases that contain company-specific information. An enterprise AI chatbot solution is not a dream anymore – it’s a reality.
Apiko offers professional AI chatbot development services – reach out to us if you want to optimize your internal processes or create an AI chatbot for your customers.
Also, we offer mobile app development services, marketplace development, and more – book a consultation to find out whether our expertise can help to turn your idea into reality!