Introduction to LLM Inference Engines

What you'll learn

Understand where an inference engine sits between model weights and the application
Compare the architectures of vLLM, TensorRT-LLM, SGLang, llama.cpp and others
Learn how KV cache management, continuous batching, speculative decoding, tensor parallelism and quantization work
Choose and configure the right engine, from a single local GPU to high-throughput production clusters

Why you should attend

Inference engines are the systems that turn a trained large language model into a usable service. They sit between the model weights and the application, handling token generation, memory allocation, batching, scheduling, caching, quantization, and hardware execution. Their job is not to make the model smarter, but to make it faster, cheaper, more scalable, and more reliable when serving real users.

In this session, we will look at how modern inference engines such as vLLM, TensorRT-LLM, SGLang, llama.cpp, and others work under the hood. We will compare their architectures, explore concepts such as KV cache management, continuous batching, speculative decoding, tensor parallelism, and quantization, and learn how to choose and configure an engine for different deployment scenarios, from a single local GPU to high-throughput production clusters.

More about hosts & guests

Aya Nasser Salama

Founder of MLOps MENA Community and Senior MLOps Engineer

I'm a Senior MLOps and LLMOps Engineer with 6+ years in AI, holding a master's in Informatics from Nile University. I've worked at Unifonic, Valeo, Aiactive Technologies, and Advanced Programs Co. I'm an MLOps instructor at ITI Cairo and founder of the MLOps MENA Community.

Mohamed Rashad

Co-Founder and CTO

Hosting organization

MLOps MENA Community

Free MLOps and AI learning for engineers across the Middle East and North Africa. Live sessions, structured roadmaps, open courses, and mentorship — free for the community. For companies, we also train, staff, and build.