DeepSeek-V3, R1 671B on 8xH100 Throughput Benchmarks

Z-AI V1.4 发行版本

NVIDIA HPC Benchmarks

From-scratch, heavily-annotated CUDA inference runtime for Qwen2.5-Coder-7B on H100 (sm_90). Custom INT4 packer, fused GEMV, paged KV, split-KV attention, CUDA graph decode — every hot path commented for the why. Educational, not a llama.cpp replacement.

rvLLM — Gemma 4 inference in Rust

Bunch of explanations and tutorials around confidential computing

CLI tool to check Oracle Cloud compute shape availability - find capacity across regions

Dashboard for AI Studio, Open Source Continuous Inference | Deepseek-R1, Qwen2.5, Llama3.1 | 4xRTX-5090 inside PRU2500, 2xH100 inside PRU2500, 8xMI210 in SuperMicro

Faster attention kernels for serving TML's Inkling model on vLLM. 2.7x over the shipping path on H100, and the only implementation that runs on A100.

In the recent competition, we were challenged to finetune a model that can convert a LaTeX expressions into Python code effectively. My team, which I led, secured 6th place overall.

Demo for installing ComfyUI on Azure VM powered by Nvidia H100 to run Text to Image and to Video models like Z-Image, Qwen and Wan

A real-time speech translation web interface that combines Automatic Speech Recognition (ASR) and Neural Machine Translation (NMT) services to provide instant translations in multiple languages. Powered by DigitalOcean GPUs.

Movement Mapper (formerly Matter Activity Logger) is a local-first activity logging and visualization tool designed for smart home motion sensors (specifically Tapo P100/T100 series). It allows you to track, visualize, and analyze movement patterns within your home without relying on cloud dashboards.

Reference Flash Attention implementation in PyTorch with V1/V2, GQA/MQA, Triton kernels, benchmark and docs.

Benchmarking QLC vs TLC NVMe for LLM inference: TPS/TCO comparison of Solidigm D5-P5336 & D7-P5520 on 8×H100 with MiniMax-M2.5

LLM Inference Benchmark (vLLM vs SGLang vs Ollama) - Tested on NVIDIA A10G (Ampere) and H100 (Hopper) with Llama 3.1 8B. SGLang wins 3.4x over vLLM at scale.

MCP server for ServPrivacy - deploy KYC-free, crypto-paid offshore VPS, dedicated, Windows RDP and GPU AI servers from any AI agent.

Thermal-aware batch controller for vLLM/TensorRT-LLM. Prevents HBM thermal throttling from killing p99 latency on H100/H200. Monitors nvidia-smi, auto-cuts batch size at 85°C, migrates cold KV to DRAM. Prometheus + Grafana included. 4.2s -> 2.1s p99 at 128K context.

Monitor low-utilization time, idle-state episodes, and workload starvation signals on NVIDIA datacenter GPUs.

Cog Single GPU Quantized Implementation of Step-Video-T2V