AdvancedИнженерия

LLM Engineer: From Local Setup to Production

A hands-on course for ML engineers: local inference with Ollama, RAG pipelines, LoRA/QLoRA fine-tuning, and production deployment with vLLM, guardrails, and LLMOps. Based on mlabonne/llm-course (Apache 2.0).

3modules

9lessons

320 mintotal time

ML engineers building production LLM systemsaudience

Module 1

Running LLMs Locally

Ollama, quantization, embeddings, RAG pipeline, and advanced hybrid retrieval.

Running LLMs Locally with Ollama

Setting up Ollama, choosing a model (Llama 3, Mistral), Q4/Q8 quantization, and local vs API tradeoffs.

35 min

Embeddings and RAG Pipeline

Building RAG with sentence-transformers, FAISS/ChromaDB, chunking strategies, and a 15-line Python example.

40 min

Advanced Retrieval: Hybrid Search and Reranking

Hybrid search (BM25 + semantic), cross-encoder reranking, and HyDE for improved RAG precision.

35 min

Module 2

Fine-Tuning

When to fine-tune, dataset preparation, LoRA/QLoRA adapters, and model export.

When to Fine-Tune and Dataset Preparation

Decision criteria between prompting, RAG, and fine-tuning; preparing a JSONL dataset with quality filters.

30 min

LoRA and QLoRA: Adapters for Fine-Tuning

How LoRA adapters work, rank selection, peft library, training script, and QLoRA for 4-bit training.

40 min

Model Export: GGUF, Ollama, Hugging Face Hub

Merging LoRA adapters, converting to GGUF, creating an Ollama Modelfile, and uploading to Hugging Face Hub.

35 min

Module 3

Production and Security

vLLM, Flash Attention, LLM security, and LLMOps with RAGAS + W&B.

Inference Optimization: vLLM, Batching, Flash Attention

vLLM PagedAttention, continuous batching, Flash Attention 2, and GPTQ/AWQ quantization for production inference.

35 min

LLM Security: Injections, Leakage, PII

Prompt injection in RAG, output leakage, PII filtering, and NeMo Guardrails for production protection.

35 min

LLMOps: Evaluation, Monitoring, and Cost Tracking

RAGAS eval pipeline, drift monitoring, cost tracking, and Weights & Biases integration for production LLM.

35 min