Building a Disaster Manual Chatbot with RAG: The Basics

To build a disaster manual chatbot with RAG, you first need to understand the parts of retrieval-augmented generation and how to cite the source documents.
In short
- RAG is a structure in which the LLM first finds documents and uses them as evidence before it answers.
- Manuals and ordinances depend on their source and how up to date they are, so RAG suits them especially well.
- Vector search, chunk splitting and source citation decide how far the chatbot can be trusted.
The phone rings in a disaster operations room. "When the river level goes above the danger level, who gives the evacuation order?" The person on duty flips through a manual of dozens of pages and a collection of ordinances. Everyone has wished at some point that a chatbot could answer right away.
The problem is that if you simply ask a general generative AI, it can give a plausible but wrong answer with confidence. It may invent content that is not in the manual, or answer with old criteria without knowing that an ordinance was revised last year. In disaster work, a wrong answer like this is not just an inconvenience; it can lead to danger. The method that appeared to deal with this is retrieval-augmented generation, or RAG for short.

What is RAG, and how is it different from an LLM?
RAG stands for Retrieval-Augmented Generation, meaning "retrieve first, then generate." A general large language model (LLM, an AI trained on huge amounts of text to produce sentences) answers only from what it remembered during training. RAG, on the other hand, first finds the related pieces of documents when a question arrives, attaches that content as reference material and then generates the answer. As an analogy, a general LLM is a student taking an exam from memory, and RAG is a student taking an open-book exam. With the book open to look up the answer, the chance of a mistake falls. For documents like disaster manuals, where every clause can be a legal basis, the open-book approach is much safer than memorization.
What parts is RAG made of?
RAG has four main parts. First, chunk splitting (dividing a document into units of meaning) cuts the document into small pieces. Second, embedding (a technique that expresses the meaning of text as coordinates in a vector space) turns those pieces into numeric vectors. Third, a vector database search finds the pieces whose meaning is close to the question. Fourth, the LLM builds an answer using the retrieved pieces as evidence. These four steps have to run in order to find exactly "the manual clause whose meaning is close to the question." In a flood response manual chatbot, picking out only the relevant clause from hundreds of pages when someone asks about "the difference between a watch and a warning" is the job of this search step.
Why does RAG suit documents like manuals and ordinances so well?
Manuals and ordinances are "evidence-type documents" in which clause numbers, issuing criteria and responsibilities are clearly written. For such documents, an exact quotation of the original text matters more than a creative answer. RAG makes the LLM answer with real pieces of the document instead of inventing knowledge, so it considerably reduces hallucination (the phenomenon in which an AI makes up untrue content that sounds plausible). Also, when a document changes, you only need to rebuild the vector database, and you do not have to retrain the LLM itself. For an organization that handles frequently revised documents, such as a practical handbook for landslide crisis response, a RAG structure that only updates the document store, rather than training a new model, is also realistic in terms of running costs.
How does the chatbot show "where an answer came from"?
One of RAG's biggest strengths is that citing sources comes naturally. The search step already brings back information such as "which document, which page, which clause," so it is easy to add a source at the end of the answer, for example "Basis: Chapter 3, Section 2 of the Disaster Response Manual of City OO." This citation is not just a courtesy; it is a safeguard that lets the user verify the answer. It is important that staff build the habit of not trusting the chatbot's answer as it is, and of opening the original clause to check once more. For information that carries responsibility, such as shelter operation standards, it is good to make comparing the source text by following the source link an internal work rule.
How does the chatbot keep up with the latest information when a manual is revised?
The weakness of RAG is that if the document store being searched, that is, the vector database, is out of date, it gives old answers no matter how well it searches. When an ordinance is revised or a manual is replaced with a new version, you must run an update: remove the old document, then split the new document into chunks and embed it again. If you do not automate this process, "current clauses" and "old clauses" can be mixed together in the search. In real operation, you find that things quickly get tangled unless you decide the revision cycle and document version rules in advance. This is something you learn only by trying, so I recommend testing the update cycle and procedure first with a few small documents and then widening the scope.
Step by step on screen

- Prepare the documents
Collect the manual and ordinance PDFs or HWP (Korean word processor) files you want to use in one folder. - Split into chunks
Split the text with a tool that divides documents by paragraph or clause (for example, LangChain or LlamaIndex). - Create embeddings
Feed the split text into an embedding model to convert it into vectors. - Build the vector store
Save the converted vectors in a vector database such as Chroma or FAISS. - Test questions and answers
Enter real questions and check that the retrieved pieces, the answers and the sources are correct. - Check the update procedure
Replace a document with a revised edition, rebuild the vector store and check that the change is reflected.
Menu names may differ slightly between versions. The figure above is a diagram of the sequence.
Prompts you can copy
Have an LLM design the criteria for splitting a manual PDF for RAG
You are an assistant who helps document managers in the disaster safety field. I want to put a document called [disaster manual name] into a RAG chatbot. This document is structured as [chapters, sections, clauses]. Please suggest chunk splitting criteria (length, overlap range, base unit) to improve search accuracy. Also list the metadata items that each chunk should include (for example, chapter title, clause number, revision date). Mark any figure you are not sure of as "needs adjustment depending on the situation."
Replace the bracketed parts for your situation. Do not use the answer as it is; check the facts first.
Try it yourself
- Run a simple RAG example yourself using a public LangChain or LlamaIndex tutorial.
- Split one disaster manual you have into clauses by hand to get a feel for chunk splitting.
- Compare with a colleague how much trust differs when the chatbot's answer has the source clause attached and when it does not.
Common questions
If I use RAG, will the chatbot never give a wrong answer?
No. If the search brings back the wrong pieces, or the document itself is out of date, a wrong answer can come out. You need a procedure in which the source is always shown and a person verifies it.
Do I have to train an LLM myself to use RAG?
No. The structure keeps an existing LLM as it is and only adds a search step, so in many cases no separate model retraining is needed. Instead, you need preparation work to turn the documents into vectors.
Do I have to use a paid service for the vector database?
Not necessarily. There are open-source tools you can use for free, such as Chroma and FAISS, and you can choose according to the amount of documents and your operating environment.
What comes next
We have looked at the four parts of RAG and the importance of source citation and keeping information current. A later post in this series covers the process of building a disaster manual chatbot from scratch with real tools.
This is an English adaptation of a post in the Korean GeoAI basics series. The original is here, and more learning materials are on GeoAI for Disaster Safety.
This post was drafted with the help of AI tools and reviewed by the author. Tool features and data portals change over time, so check the official documentation when you practice.
Comments
Post a Comment