CrewAI RAG Chatbot

A citation-first RAG chatbot built on a 4-agent CrewAI pipeline. Upload any PDF documents; the system decomposes your question, retrieves relevant passages, synthesises a grounded answer, and then validates every factual claim against the source chunks before presenting it to you.

LLM inference runs on Groq (Llama 3.3 70B). Embeddings are computed locally (no API cost). The full pipeline runs in Docker and can be deployed to AWS EC2 in one command.

Architecture

flowchart LR
    subgraph Ingest["📄 Ingestion"]
        PDF["PDF Upload"] --> Parse["pypdf parser"]
        Parse --> Chunk["Sentence chunker\n800 chars, 100 overlap"]
        Chunk --> Embed["all-MiniLM-L6-v2\n(local embeddings)"]
        Embed --> Store["ChromaDB\nPersistent vector store"]
    end

    subgraph Query["💬 Query Pipeline (CrewAI)"]
        Q["User question"] --> P["🔍 Planner Agent\nDecomposes into 2-3 sub-queries"]
        P --> R["📚 Retriever Agent\nSearches ChromaDB per sub-query"]
        R --> S["✍️ Synthesizer Agent\nBuilds cited answer (markdown)"]
        S --> V["✅ Validator Agent\nFact-checks every claim"]
        V --> A["Final answer\nwith PASS / FLAG labels"]
    end

    Store --> R

Agent Responsibilities

| Agent | Role | Output | |-------|------|--------| | Planner | Decomposes the user query into 2–3 focused sub-queries | Plain-text sub-query list | | Retriever | Runs each sub-query against ChromaDB; tags every chunk with [source:file, page:N, chunk:N] | Raw chunk bundle | | Synthesizer | Writes a structured markdown answer grounded solely in retrieved chunks | Cited answer | | Validator | Cross-checks every factual claim against the raw chunks; marks each PASS or FLAG | Validation report |

All agents share a single LLM (Groq Llama 3.3 70B, temperature = 0) for fully deterministic, auditable output.

Features

• 4-agent validation pipeline — answers are fact-checked before delivery, surfacing hallucinations proactively
• Semantic chunking — sentence-boundary splitting with 100-character overlap preserves context
• Idempotent ingestion — deterministic chunk IDs (SHA-256 hash) mean re-uploading the same PDF is a no-op
• Local embeddings — all-MiniLM-L6-v2 runs on-device; no embedding API costs
• Source attribution — every claim is linked back to [Doc: filename, Chunk N]
• Zero-friction uploads — drag-and-drop PDFs via sidebar; auto-ingested at app startup from ./docs/
• Docker-ready — embedding model pre-baked into image to avoid cold-start delays
• One-command EC2 deploy — infra/deploy.sh rsyncs, rebuilds, and restarts the container

Quick Start

Prerequisites

• Python 3.12+
• Groq API key (free tier available)
• uv package manager (recommended) or pip

Local development

git clone https://github.com/ashish-code/crewai-rag-chatbot
cd crewai-rag-chatbot

# Install dependencies
uv sync           # or: pip install -e .

# Configure environment
cp .env.example .env
# Edit .env and set GROQ_API_KEY

# Configure Streamlit password (optional — comment out login gate in app.py to skip)
mkdir -p .streamlit
echo 'password = "changeme"' > .streamlit/secrets.toml

# (Optional) pre-load documents
cp your_docs/*.pdf docs/

# Run
streamlit run app.py

The app opens at http://localhost:8501.

Docker

cp .env.example .env          # set GROQ_API_KEY
echo 'password = "changeme"' > .streamlit/secrets.toml

docker compose up --build

Chroma data persists in the chroma_data Docker volume between restarts.

Environment Variables

| Variable | Required | Description | |----------|----------|-------------| | GROQ_API_KEY | ✅ | Your Groq API key — obtain from console.groq.com |

Streamlit secrets (.streamlit/secrets.toml):

password = "your-login-password"

Project Layout

crewai-rag-chatbot/
├── app.py                      # Streamlit entry point + login gate
├── src/
│   ├── agents/                 # CrewAI agent definitions (planner, retriever, synthesizer, validator)
│   ├── crew/                   # Pipeline orchestration (sequential crew)
│   ├── ingestion/              # PDF parsing + sentence-level chunker
│   ├── models/                 # LLM factory (Groq via LiteLLM)
│   ├── ui/                     # Streamlit UI components
│   └── vector_store/           # ChromaDB client + search helpers
├── docs/                       # Drop PDFs here for auto-ingestion on startup
├── infra/
│   ├── setup_ec2.sh            # One-time EC2 bootstrap (Docker + swap)
│   └── deploy.sh               # Rsync + rebuild + restart on EC2
├── .env.example                # Environment template
├── docker-compose.yml
├── Dockerfile
└── pyproject.toml

Deploying to AWS EC2

1. Launch an EC2 instance (t2.small or larger recommended; t2.micro works with 1 GB swap).

2. Bootstrap the instance (first time only):

   ssh -i key.pem ec2-user@<EC2_IP> 'bash -s' < infra/setup_ec2.sh
   

3. Deploy or re-deploy:

   bash infra/deploy.sh <EC2_IP> key.pem
   

   This rsyncs the project (excluding chroma_db, __pycache__, .git), rebuilds the Docker image, and restarts the service. The Chroma volume is preserved across deploys.

4. Access the app at http://<EC2_IP>:8501.

How the RAG Pipeline Works

User question
    │
    ▼
[Planner]  → "What are the maintenance intervals?"
             "What failure modes exist?"
    │
    ▼
[Retriever] → queries ChromaDB with each sub-query
              → returns chunks tagged [source:manual.pdf, page:12, chunk:3]
    │
    ▼
[Synthesizer] → "According to [Doc: manual.pdf, Chunk 3], the interval is…"
    │
    ▼
[Validator] → Claim: "interval is 6 months" → PASS (found in chunk 3)
              Claim: "costs $500" → FLAG: not found in retrieved chunks
    │
    ▼
Final answer with validation report

Dependencies

| Package | Purpose | |---------|---------| | crewai | Multi-agent orchestration | | litellm | Unified LLM gateway (Groq, OpenAI, Anthropic, …) | | chromadb | Persistent vector store | | sentence-transformers | Local embedding model | | torch | Backend for sentence-transformers | | pypdf | PDF text extraction | | streamlit | Web UI |

License

MIT — see LICENSE.