A Retrieval-Augmented Generation (RAG) application built with FastAPI, Streamlit, and Google Gemini that allows users to upload documents, ask questions grounded in the document's content, and generate concise summaries.

The application extracts text from supported documents, splits it into meaningful chunks, retrieves the most relevant chunks using semantic search, and uses an LLM to generate accurate, context-aware responses.

• 📄 Upload PDF, TXT, or DOCX documents
• ⛔ Maximum file size is 30MB.
• 🔍 Semantic search using sentence embeddings
• 💬 Ask questions and response is grounded only in the uploaded document
• 📝 Generate concise document summaries
• 🚫 Detect and reject out-of-context questions
• ⚡ FastAPI backend with Streamlit frontend
• 🧩 Modular and easily extensible codebase

                Upload Document
                      │
                      ▼
            Extract Text from File
                      │
                      ▼
           Recursive Text Chunking
                      │
                      ▼
          Generate Chunk Embeddings
                      │
         Store Chunks + Embeddings
                      │
         ┌────────────┴────────────┐
         │                         │
         ▼                         ▼
   Ask Question             Summarize Document
         │
         ▼
 Generate Query Embedding
         │
         ▼
 Cosine Similarity Search
         │
         ▼
 Retrieve Top 15 Chunks
         │
         ▼
 Reranker model
         │
         ▼
 Top 3 chunks along with their 
 neighbours with no repetitive chunks
         │
         ▼
      Google Gemini
         │
         ▼
 Grounded Response

Intern_Project/

│
├── app/
│   ├── routes/
|   |     ├── ask.py
|   |     ├── summarize.py
│   |     └── upload.py
│   ├── steps/
|   |     ├── chunking.py
|   |     ├── document_store.py
│   |     ├── embedding.py
│   |     ├── file_reader.py
│   |     ├── LLM_endpoint.py
│   |     └── retrieval.py
│   └── main.py
|
├── using_chroma/
│   ├── routes/
|   |     ├── ask.py
|   |     ├── summarize.py
│   |     └── upload.py
│   ├── steps/
|   |     ├── chunking.py
|   |     ├── document_store.py
│   |     ├──embedding.py
│   |     ├── file_reader.py
│   |     ├── LLM_endpoint.py
│   |     ├── retrieval.py
│   |     └── save_to_db.py
│   └── main.py
│
├── frontend/
|     └── app.py
│
├── notes.md
├── test_results.md
├── requirements.txt
├── docker-compose.yml
├── Dockerfile.backend
├── Dockerfile.frontend
├── dockerignore
├── README.md
└── .env.example

The user uploads a supported document.

Currently supported:

• PDF
• TXT
• DOCX

The application extracts all readable text from the uploaded file.

Instead of splitting text at fixed character counts, the application recursively splits the document using progressively smaller separators (paragraphs → lines → sentences → words).

This preserves semantic meaning while keeping chunks within the desired size.

Each chunk is converted into a dense vector using:

multi-qa-MiniLM-L6-cos-v1

These embeddings are generated only once per uploaded document.

When a user asks a question:

• The query is embedded using the same embedding model.
• Cosine similarity is computed between the query and document chunks.
• The top 15 chunks are selected.
• If similarity is below a threshold, the question is considered outside the document.
• The top 15 chunks are then sent to reranker model which then reranks them and sends top 3 chunks
• Top 3 chunks along with their neighbours are sent to Gemini. No duplicate chunks are sent to the api.

Only the retrieved chunks and the user's question are sent to Google Gemini.

The prompt instructs Gemini to answer only using the provided document context, preventing hallucinations.

POST /upload

Uploads a document and generates embeddings.

POST /ask

Returns an answer based only on the uploaded document.

GET /summarize

Returns a concise summary of the uploaded document.

1. Clone the repository

git clone https://github.com/PramishSapkota/Intern_Project.git

cd Intern_Project

1. Create a virtual environment

python -m venv .venv

Activate it on:

• Windows

.venv\Scripts\activate

• Linux / macOS

source .venv/bin/activate

1. Install dependencies

pip install -r requirements.txt

Create from .env.example file and add the api keys there.

Save the file as .env

Example:

export GEMINI_API_KEY="AQ..."

OR: in terminal

cp .env.example .env  #then paste your own api keys in .env file which got created

1. Start the FastAPI server

uvicorn app.main:app --reload

1. Launch the Streamlit frontend

streamlit run frontend/app.py

1. Build and run the docker image

docker compose up --build

• backend docker image is of 3.07GB while frontend docker image is of 781.29MB in my machine.

1. Then open http://localhost:8501 for the Streamlit UI (backend API is at http://localhost:8000). Or follow the links shown in terminal window.

For subsequent runs after docker images are already built

docker compose up

• Lightweight
• High performance
• Automatic API documentation
• Excellent for ML applications

Chosen because it provides:

• High-quality responses
• Large context window
• Free API tier
• Easy Python integration

Selected because it offers:

• Fast inference
• Small model size
• Also trained in question answer pairs
• Suitable for CPU execution

Selected because it offers:

• Fast inference
• Small model size
• Good semantic retrieval performance
• Suitable for CPU execution

• Persistent storage out of the box
• Very easy API
• Handles low to moderate amount of data which is what needed in this project
• Perfect for local RAG applications
• Scales comfortably to thousands of chunks, which is far beyond the needs of this assignment.

• Used semantic search instead of keyword matching to improve retrieval accuracy.
• Implemented recursive chunking to preserve context and avoid breaking sentences unnecessarily.
• Stored embeddings in memory to eliminate repeated computation for each query.
• Restricted the LLM to retrieved document chunks to reduce hallucinations and ensure grounded responses.
• Added similarity thresholding to identify questions unrelated to the uploaded document.
• Used chromadb over FAISS because main use case of FAISS is in huge amount of data while chroma is for low to medium amount of data

• Supports one uploaded document at a time.
• Does not perform OCR on documents(supports OCR for pdf when using chroma db)
• Embeddings are regenerated after server restart.
• Retrieval accuracy depends on user query(More detailed query gives better answers)

• OCR support for docx
• Multi-document querying
• Streaming responses

Each module was tested using if __name__ == "__main__" to verify it is working as intended.

The application was tested using:

• Short text document
• Long PDF
• Invalid/edge-case document

Most of the testing files were taken from sample-files.com

Detailed testing results are available in test_results.md.