Comprehensive, scalable ML inference architecture using Amazon EKS, leveraging Graviton processors for cost-effective CPU-based inference and GPU instances for accelerated inference. Guidance provides a complete end-to-end platform for deploying LLMs with agentic AI capabilities, including RAG and MCP

High-performance LLM Gateway built in Go - OpenAI compatible proxy with multi-provider support, adaptive routing, and enterprise features

Connect any LLM-powered client app, such as a coding agent, to any supported inference backend/model.

This repo presents resilience patterns for scaling inference for Generative AI workloads on AWS: Bedrock cross-Region inference, AWS account sharding, and intelligent routing with LLM gateways.

LLMCallGateway 是一个基于 LiteLLM 构建的专业 LLM API 网关服务。将所有模型（包括非 OpenAI 模型）的请求格式统一为 OpenAI 格式，并提供详细的日志跟踪和性能监控，帮助开发者直观了解与下游 LLM API 交互的细节与成本。

Master’s thesis developed as part of the Master’s degree in Telecommunications Engineering.