PROJECT 03Natural Language Processing & LLMsVector Search & RAG

RAG DOCUMENT ASSISTANT

Retrieval-Augmented Generation assistant with semantic chunking and grounded citations.

Source Repository

RAG ASSISTANT // DETERMINISTIC RETRIEVAL & GROUNDED CITATION PIPELINE

ZERO HALLUCINATIONS

STEP 01

PDF Normalization

Recursive chunking with 15% overlap preserving boundary clauses.

STEP 02

Vector Embeddings

Sentence Transformers generate high-density 384-dim semantic vectors.

STEP 03

ChromaDB Index

Persistent collection cosine search retrieving top-k relevant passages.

STEP 04

Ollama Synthesis

Local LLM outputs strictly citation-backed response with page hashes.

TECH STACK • CORE RUNTIME

PythonRAGLLMsChromaDBSentence TransformersOllamaFastAPIStreamlit

DOMAIN DISCIPLINE

AI / Retrieval-Augmented Generation

Author: Martin Wassim

VERIFIED INTERFACE ARTIFACTS & ARCHITECTURAL DIAGRAMS

4 visual assets

Figure 1: RAG DOCUMENT ASSISTANT — Visual EvidenceVerified Production Asset

Figure 2: RAG DOCUMENT ASSISTANT — Visual EvidenceVerified Production Asset

Figure 3: RAG DOCUMENT ASSISTANT — Visual EvidenceVerified Production Asset

Figure 4: RAG DOCUMENT ASSISTANT — Visual EvidenceVerified Production Asset

01 / THE PROBLEM BEING SOLVED

Operational & Technical Friction

General LLMs frequently hallucinate facts when queried on proprietary manuals, technical specs, enterprise reports, and dense domain texts.

02 / THE ARCHITECTED SOLUTION

Systemized Implementation

Constructed a deterministic RAG retrieval pipeline where the LLM is constrained to synthesize exclusively from top-k retrieved vectors.

03 / ACTUAL IMPLEMENTATION & WORKFLOW

How It Operates in Production

Built with Python, FastAPI, and ChromaDB. Documents are ingested, normalized, and split using recursive character chunking with 15% overlap. Embeddings are generated using Hugging Face Sentence Transformers and queried via approximate nearest neighbor search.

Engineered chunk overlap strategies to preserve semantic boundaries, optimized embedding dimensionality for rapid cosine similarity queries, and implemented citation mapping.

04 / SYSTEM ARCHITECTURE EXPLANATION

Data Flow & Infrastructure

Unstructured documents flow into an ingestion worker that parses text and generates vector embeddings. Vectors are indexed in persistent ChromaDB collections. Query requests trigger semantic vector searches, which feed retrieved text passages into the prompt context of an Ollama LLM.

Architectural Highlight: Pipeline guarantees zero hallucinations by enforcing strict context prompting and returning source page references for every assertion.

05 / KEY ENGINEERING DECISIONS

Recursive semantic chunking: Maintained paragraph integrity and header context to avoid splitting critical clauses across chunk boundaries.

Privacy-preserving local inference: Integrated Ollama (Llama 3 / Mistral) to allow enterprises to query confidential documents without sending data to external APIs.

Strict verification prompt templates: Constrained LLM output to state 'Information not found in provided documents' if cosine similarity falls below safety threshold.

06 / CHALLENGES & MITIGATIONS

•Preserving context across complex multi-column PDF layouts and nested tables.

•Balancing chunk size between semantic granularity and LLM context window efficiency.

•Eliminating hallucinated page numbers by appending metadata hashes to each retrieved chunk.

07 / VERIFIED CAPABILITIES

Automated PDF and text ingestion pipeline

Semantic text chunking with overlap preservation

High-density vector embeddings using Sentence Transformers

ChromaDB persistent collection indexing with cosine distance

Local LLM inference via Ollama integration (privacy-preserving)

Strict citation-backed answers with clickable document references

FastAPI backend with a reactive Streamlit research interface

Return to Project Index Inquire About Similar Architecture