IITM IEEE HACKSAGON | Team Binary Brain | Amity University

Extracting structured data (Dealer Name, Model, Horse Power, Amount, Signature, etc.) from unstructured invoices is labor-intensive and error-prone. Traditional rule-based approaches fail to handle diverse invoices of varying layouts and languages (English, Hindi, Gujarati).

Manual data entry is:

• \u274c Slow
• \u274c Error-prone
• \u274c Expensive
• \u274c Delays credit decisioning

A Hybrid Document AI Pipeline combining OCR + Vision + NLP + AI Engine to automatically extract structured data from tractor loan quotation invoices with \u226595% Document Level Accuracy.

PDF Input \u2192 Preprocessing \u2192 OCR \u2192 Layout & Object Detection \u2192 Field Extraction \u2192 Validation \u2192 JSON Output

• \ud83c\udf10 Multilingual Support \u2014 English, Hindi, Gujarati
• \ud83d\udcd0 Layout-Independent Extraction \u2014 Works with any invoice format
• \u270d\ufe0f Signature & Stamp Detection \u2014 Using object detection with bounding boxes
• \ud83d\udcb0 Cost-Efficient Inference \u2014 < \u20b91 per document
• \ud83d\udda5\ufe0f CPU Compatible Pipeline \u2014 No GPU required
• \ud83d\udcc8 Scalable \u2014 Works for any invoice type (retail/industrial)

• Hybrid rule + AI system for higher accuracy
• Pseudo-labeling for no ground truth scenario
• Self-consistency validation mechanism
• Confidence-based rejection system

• Python 3.10+

# Clone the repository
git clone <repo-url>
cd tanuhackathon

# Create virtual environment
python -m venv .venv
.venv\Scripts\activate    # Windows
# source .venv/bin/activate  # Linux/Mac

# Install dependencies
pip install -r requirements.txt

streamlit run app.py --server.port 8501

Open http://localhost:8501 in your browser.

tanuhackathon/
\u251c\u2500\u2500 app.py                    # Streamlit Web Application
\u251c\u2500\u2500 requirements.txt          # Python dependencies
\u251c\u2500\u2500 README.md                 # This file
\u251c\u2500\u2500 src/
\u2502   \u251c\u2500\u2500 __init__.py
\u2502   \u251c\u2500\u2500 preprocessing.py      # PDF to Image, Deskew, Denoise, Enhance
\u2502   \u251c\u2500\u2500 ocr_engine.py         # EasyOCR multilingual text extraction
\u2502   \u251c\u2500\u2500 detector.py           # Signature & Stamp detection
\u2502   \u251c\u2500\u2500 field_extractor.py    # Dealer Name, Model, HP, Cost extraction
\u2502   \u251c\u2500\u2500 validation.py         # Confidence scoring & JSON output
\u2502   \u2514\u2500\u2500 pipeline.py           # Main orchestration pipeline
\u251c\u2500\u2500 sample_invoices/          # Sample test invoices
\u251c\u2500\u2500 uploads/                  # User uploaded files
\u2514\u2500\u2500 outputs/                  # Extraction results (JSON + annotated images)

{
  "dealerName": "Agri Machinery",
  "modelName": "Sonali 550",
  "horsePower": "50",
  "assetCost": "650000",
  "dealerSignature": [238, 678, 345, 721],
  "dealerStamp": [452, 676, 523, 725],
  "metadata": {
    "processingTime": "3.42s",
    "confidence": {
      "dealerName": 0.92,
      "modelName": 0.95,
      "horsePower": 0.85,
      "assetCost": 0.90,
      "dealerSignature": 0.78,
      "dealerStamp": 0.72
    },
    "documentLevelAccuracy": "85.3%",
    "costEstimate": "\u20b90.67"
  }
}

This project was built for IITM IEEE HACKSAGON hackathon.

Built with \u2764\ufe0f by Team Binary Brain