Local-Rag-Application

Offline Developer Documentation Assistant

An offline Retrieval-Augmented Generation (RAG) app for developer documentation. It runs locally with Foundry Local, SQLite, and JavaScript, so it can answer questions from your own docs without cloud APIs or outbound network calls.

What It Does

• Answers questions from local markdown documents
• Retrieves the most relevant chunks before generation
• Streams responses in the browser
• Lets you upload new .md or .txt files at runtime
• Works fully offline after the model is available locally

How It Works

1. npm run ingest reads the markdown files in docs/
2. Each document is split into chunks and stored in data/rag.db
3. When a question is asked, the app searches for matching chunks
4. The local model uses that context to generate the answer

Requirements

• Node.js 20 or newer
• Foundry Local
• The phi-3.5-mini model, which is downloaded on first run if needed
• First run may take several minutes while the model downloads and loads

Quick Start

git clone https://github.com/leestott/local-rag.git
cd local-rag
npm install
npm run ingest
npm start

Open http://127.0.0.1:3000 in your browser.

If the app starts slowly the first time, that is usually the model download. Later runs are much faster because the model is cached locally.

Screenshots

Landing Page

Chat Response

Upload Document

Architecture Diagram

RAG Flow Sequence

Project Structure

local-rag/
├── docs/       # Source documents used by the RAG pipeline
├── public/     # Single-file web UI
├── src/        # Server, ingestion, vector store, and chat engine
├── data/       # SQLite database created at runtime
├── test/       # Node.js test files
└── package.json

Main Scripts

Script	Command	Description
Ingest	npm run ingest	Chunk and index all docs into SQLite
Start	npm start	Start the server
Dev	npm run dev	Start with file watching
Test	npm test	Run the test suite

API Endpoints

Method	Endpoint	Description
POST	/api/chat	Non-streaming chat response
POST	/api/chat/stream	Streaming chat response via SSE
POST	/api/upload	Upload and index a document
GET	/api/docs	List indexed documents
GET	/api/health	Check model and database status

License

MIT