Velesio AI Server

A high-performance, microservice-based AI inference server designed for scalable LLM and Stable Diffusion workloads.

Overview

Velesio AI Server is a production-ready AI inference platform that provides:

• LLM Text Generation via llama.cpp or Ollama
• Stable Diffusion Image Generation with WebUI support
• Redis Queue Architecture for scalable job processing
• Docker-based Deployment with GPU acceleration
• Built-in Monitoring with Grafana and Prometheus
• Unity Integration ready endpoints

Architecture

1
2
3
4
5
6
7
8
9
┌─────────────┐    ┌─────────┐    ┌─────────────┐
│    API      │────│  Redis  │────│ GPU Workers │
│  (FastAPI)  │    │ Queue   │    │ (LLM + SD)  │
└─────────────┘    └─────────┘    └─────────────┘
       │                                  │
       │           ┌─────────────┐        │
       └───────────│ Monitoring  │────────┘
                   │(Grafana+Prom)│
                   └─────────────┘

Key Features

🚀 High Performance

• Standard llama.cpp server with CUDA support or Ollama for flexible LLM deployment
• GPU acceleration with CUDA support
• Asynchronous job processing via Redis Queue

🔧 Easy Setup

• Docker Compose deployment
• Automatic model downloading
• Ollama for simplified model management
• Pre-configured monitoring stack

🎯 Unity Ready

• Compatible with “LLM for Unity” asset
• Base64 image encoding for seamless integration
• Standardized API endpoints

📊 Production Monitoring

• Real-time metrics with Prometheus
• Visual dashboards in Grafana
• Redis queue monitoring
• GPU utilization tracking

Services

Next Steps

• Quickstart Cloud Infra - Cloud deployment guide
• Quickstart Self hosted - Self-hosted setup
• Ollama Integration - Use Ollama for LLM inference
• Architecture - System design deep dive
• Deployment Guide - Production deployment strategies
• Model Templates - Model configurations

Need help? Check our troubleshooting guide or open an issue on GitHub.