Higher RAII, and the Seven Arcane Uses of Linear Types Languages and Archit Programming LanguagesVale Programming Language 2 years ago
Layer-wise inferencing + batching: Small VRAM doesn't limit LLM throughput anymore Languages and Archit Artificial IntelligenceLarge Language Models 2 years ago