SEMDRIFT

Measuring Semantic Drift in Multilingual LLM-Generated Clinical Trial Explanations: A Reliability Framework

A production-quality research platform for systematically evaluating semantic drift in multilingual clinical trial explanations generated by large language models.

---

Overview

SEMDRIFT is a full-stack AI research platform built to investigate semantic drift — the degradation of medical meaning — when LLMs translate and explain clinical trial eligibility criteria across multiple languages (English, French, Arabic, Urdu). The framework combines a 5-stage NLP pipeline, cosine-similarity drift scoring via LaBSE embeddings, and a data visualization dashboard, producing results in IEEE-ready export formats.

---

Screenshots

Authentication

Login Page

Register Page

Dashboard

Dashboard Overview	Drift Statistics
Activity Feed

Experiments

Experiment List

Experiment Details

Statistical Analysis

Statistical Analysis	Correlation Analysis
Bootstrap Confidence Intervals

Visualizations

Drift Scores by Language	Reliability Radar Chart
Medical Term Preservation	Round-Trip Analysis
Cross-Language Comparison	Benchmark Comparison

---

Key Features

Feature	Description
Multilingual Pipeline	5-stage pipeline across EN / FR / AR / UR
Semantic Drift Scoring	LaBSE cosine-similarity with composite reliability scores
Statistical Analysis	Bootstrap CIs, correlation analysis, significance testing
Benchmark Comparison	Cross-experiment comparison with statistical significance
Export Engine	CSV, Excel, JSON, Markdown, LaTeX, PDF (IEEE-ready)
Interactive Dashboard	Real-time experiment progress and drift visualization
Dataset Management	ClinicalTrials.gov integration with quality indicators
Data Integrity	Immutable experiment configs, full audit trails

---

Research Findings (Preliminary)

Language	Resource Level	Avg. Drift Score	Drift Category
English (EN)	High	0.92	✅ Low
French (FR)	High	0.81	✅ Low
Arabic (AR)	Medium	0.72	⚠️ Medium
Urdu (UR)	Low	0.58	🔴 High

Key finding: Semantic drift correlates significantly with language resource level (r = 0.94, p < 0.01). Numeric eligibility criteria (e.g., lab values, BMI thresholds) exhibit 16% greater drift than qualitative criteria across all low-resource languages.

---

Tech Stack

Layer	Technologies
Backend	Python 3.12, FastAPI, SQLAlchemy, Alembic, Pydantic, SQLite
Frontend	Next.js 15, TypeScript, Tailwind CSS, shadcn/ui, Recharts
NLP / ML	LaBSE (sentence-transformers), Llama-3-8B via Ollama
DevOps	Docker, Nginx, GitHub Actions

---

Quick Start

Prerequisites

• Python 3.12+
• Node.js 18+
• Ollama (for LLM inference)

Backend

cd backend
python -m venv venv && source venv/bin/activate   # Windows: venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env
alembic upgrade head
python main.py                                     # API at http://localhost:8000

Frontend

cd frontend
npm install
cp .env.example .env.local
npm run dev                                        # UI at http://localhost:3000

---

Project Structure

SEMDRIFT/
├── backend/
│   ├── app/
│   │   ├── api/            # Router aggregation (v1 prefix)
│   │   ├── core/           # Logging, exceptions
│   │   ├── models/         # SQLAlchemy ORM models
│   │   ├── schemas/        # Pydantic schemas
│   │   ├── repositories/   # Data-access layer
│   │   ├── services/       # Business logic
│   │   └── routes/         # FastAPI route handlers
│   ├── alembic/            # Database migrations
│   └── main.py             # Entry point
├── frontend/
│   ├── app/                # Next.js app directory
│   ├── components/         # React components
│   ├── services/           # API client functions
│   └── types/              # TypeScript types
├── docs/images/            # Screenshots
└── docker-compose.yml

---

API Reference (Selected Endpoints)

Method	Endpoint	Description
GET	/api/v1/health	Health check
GET	/api/v1/dashboard	Dashboard statistics
GET	/api/v1/datasets	List datasets
POST	/api/v1/experiments	Create experiment
GET	/api/v1/experiments/{id}/results	Fetch drift results
GET	/api/v1/statistics	Aggregated statistics
POST	/api/v1/exports	Generate export

Full interactive API docs: http://localhost:8000/docs

---

Testing

# Backend tests
cd backend && pytest tests/

# Specific modules
pytest tests/test_ollama_client.py
pytest tests/test_inference_queue.py
pytest tests/test_inference_logging.py

---

License

This project is developed for academic research purposes.

---

Asad Raza · SEMDRIFT v0.1 · 2026