Verdict ⚖️

A multi-agent hypothesis testing framework that systematically evaluates claims in academic papers using AI-powered evidence analysis.

Overview

Verdict automates the scientific review process by deploying a pipeline of specialized AI agents that extract claims, decompose hypotheses, hunt for evidence, and synthesize verdicts using mathematical frameworks like Dempster-Shafer theory.

Key Features

• Multi-Agent Pipeline: Six specialized agents handle different aspects of claim evaluation
• PDF Processing: Automated extraction and parsing of academic papers
• Evidence-Based Analysis: RAG (Retrieval-Augmented Generation) system for evidence gathering
• Mathematical Rigor: Implements Dempster-Shafer theory and Sequential Probability Ratio Testing (SPRT)
• Interactive Web UI: Streamlit-based interface for paper analysis and results visualization
• Dependency Tracking: Graphs relationships between claims and evidence

Architecture

Agent Pipeline

1. Claim Extractor → Identifies testable claims
2. Hypothesis Decomposer → Breaks down complex hypotheses  
3. Evidence Hunter → Searches for supporting/contradicting evidence
4. Evidence Judge → Evaluates evidence quality and relevance
5. Devil's Advocate → Challenges findings with counterarguments
6. Verdict Synthesizer → Produces final assessment

Core Components

• Database: SQLite-based storage with pipeline state tracking
• Embeddings: Sentence transformers for semantic search
• LLM Integration: Google Gemini API for agent reasoning
• RAG System: ChromaDB for efficient evidence retrieval

Tech Stack

• Backend: Python, LangGraph, Pydantic
• AI/ML: Google Generative AI, Sentence Transformers, ChromaDB
• Frontend: Streamlit with custom CSS styling
• Data: SQLite, NetworkX for dependency graphs
• Processing: PyMuPDF for document parsing

Getting Started

1. Install dependencies: pip install -r requirements.txt
2. Set up environment variables (Google API key)
3. Run the application: python run.py
4. Access the web interface and upload academic papers for analysis

Mathematical Framework

Verdict employs rigorous mathematical approaches:

• Dempster-Shafer Theory: Handles uncertainty and conflicting evidence
• SPRT: Sequential hypothesis testing for statistical significance
• Dependency Graphs: Models relationships between claims and evidence sources