MLDock

MLDock is a self-hosted model serving platform designed to simplify the deployment of machine learning models. It provides a lightweight infrastructure for ML engineers to expose multiple model frameworks as secure REST APIs without writing boilerplate backend code.

Core Features

• Automated API Generation: Upload a serialized model and metadata schema, and MLDock dynamically provisions a validation-backed REST endpoint.
• In-Memory Caching: Models are lazy-loaded into memory upon first request to minimize startup latency and optimize memory footprint.
• Access Control: Includes built-in JWT authentication for the dashboard and HMAC-hashed API keys for endpoint access.
• Observability: Tracks endpoint latency, memory usage, and request status codes.
• Interactive Playground: Test predictions and input schemas directly from the UI.
• Multi-Framework Drivers: Supports scikit-learn, PyTorch, TensorFlow/Keras, and ONNX models when the relevant runtime is installed.

Architecture

• Backend: FastAPI, SQLAlchemy, PostgreSQL
• Frontend: React, Redux Toolkit, Tailwind CSS, Vite
• ML Stack: scikit-learn, joblib, torch, tensorflow, onnxruntime
• Security: passlib (bcrypt), python-jose

Getting Started

Using Docker Compose (Recommended)

1. Clone the repository
2. Spin up the containers:

   docker compose up --build -d

3. Access the dashboard at http://localhost:5173 (or your mapped port). The initial run will prompt you to configure the admin account.

Local Development Setup

Backend:

cd backend
python -m venv venv
source venv/bin/activate  # or venv\Scripts\activate on Windows
pip install -r requirements.txt
export DATABASE_URL="postgresql://user:pass@localhost:5432/mldock"
uvicorn app.main:app --reload

Frontend:

cd frontend
npm install
npm run dev

Usage: Deploying a Model

Models require a metadata.json file defining the expected schema. MLDock uses this to automatically generate Pydantic validation for the endpoint.

Supported model file extensions depend on the selected framework:

• scikit-learn: .pkl, .joblib
• PyTorch: .pt
• TensorFlow/Keras: .h5, .keras
• ONNX: .onnx

Example metadata.json:

{
  "name": "car-price",
  "display_name": "Car Price Predictor",
  "description": "Predicts used car prices based on input features",
  "framework": "sklearn",
  "version": "1.0.0",
  "input_schema": {
    "year": "integer",
    "km_driven": "integer",
    "fuel": "string",
    "transmission": "string"
  },
  "output_schema": {
    "prediction": "float"
  }
}

Supported schema types: integer, float, string, boolean.

The framework field should match the model type you upload. MLDock uses that value to select the appropriate runtime driver and validate the uploaded model file.