OwnDrive ☁️

A self-hosted private cloud storage engine with multi-modal semantic search, automatic virus gates, and storage deduplication insights.

---

---

🏗️ Architecture Flow

graph TD
    Client[Browser: React + Vite]
    Backend[Backend API: Fastify Node.js]
    Db[(Postgres + pgvector)]
    Storage[(Object Storage: MinIO/S3)]
    Queue[Queue: Redis]
    Worker[AI Worker: Python FastAPI]
    Scanner[Antivirus: ClamAV Daemon]

    Client -->|1. Presigned Upload| Backend
    Client -->|2. Direct Upload| Storage
    Backend -->|3. Enqueue Job| Queue
    Queue -->|4. Pull Job| Worker
    Worker -->|5. Download & Scan| Scanner
    Worker -->|6. If clean, vectorize| Db
    Backend -->|7. Query Hybrid Search| Db

Loading

---

✨ Core Features

• 📁 Organized Drive — Multi-folder nested trees with drag-and-drop uploads.
• 🔍 Hybrid Search — Combines lexical keyword indexing (ILIKE) with vector database similarity (pgvector) using Reciprocal Rank Fusion (RRF).
• 🛡️ Virus Gatekeeper — Asynchronous post-upload scanning via a decoupled ClamAV daemon to prevent infected file distribution.
• 📊 Deduplication Dashboard — Identifies exact duplicates (SHA256) and visual/textual similarity duplicates (vector distance) to save S3 storage costs.
• 🔄 Real-time Sync — Event-driven client state invalidation over WebSockets via Socket.IO.
• 🔐 Secure Auth — JWT-based authentication with refresh token rotation and bcrypt hashing.

---

🚀 Quick Start

1. Configure Environments

Copy the environment template and set your credentials:

cp .env.example .env

2. Start the Stack (Docker)

Build and boot up the database, storage, Redis queue, backend, frontend, worker, and virus scanner:

# Linux/Mac
docker compose up -d --build

# Windows
.\up.ps1

3. Service Endpoints

• Frontend Web App: http://localhost:5180
• Backend API Health: http://localhost:3010/health
• Storage Console: http://localhost:9090 (MinIO)

---

💡 Production Scaling & Roadmap (Interview Ready)

OwnDrive is designed as a single-server monolith for development simplicity. To scale this system to production-grade workloads, we analyzed these key architectural trade-offs:

1. Inference Offloading (Client-Side AI):
   • The Problem: Generating vectors for every file upload consumes massive server CPU/GPU resources.
   • The Solution: Migrate to client-side embeddings using Transformers.js (ONNX) in the browser. File vectorization occurs on the client's local device, dropping S3/EC2 server costs to virtually zero.
2. Zero-Knowledge Security:
   • The Problem: Exposing raw documents and files on S3 compromises user privacy.
   • The Solution: Encrypt files locally in the browser using the Web Crypto API (AES-GCM) before upload. The server only sees encrypted blobs and mathematical vectors.
3. Decoupled Serverless Infrastructure:
   • The Solution: In production, migrate from local Docker services to AWS-native managed services:
     • Postgres ➔ Amazon RDS (PostgreSQL)
     • Redis ➔ Amazon ElastiCache
     • S3 API ➔ Amazon S3
     • AI Worker / ClamAV ➔ AWS Lambda (triggered asynchronously via S3 Event Notifications).