Guided Decoding for Training-Free Alignment of LLMs

A hands-on tutorial covering standard decoding strategies and guided decoding with logits processors.

Authors: Son The Nguyen, Tommy Cheng, Theja Tulabandhula

Tutorial resources Watch the tutorial segment or explore the slide deck.

Guided Decoding · starts at 1:30:56

Open on YouTube

This hands-on tutorial introduces inference-time methods for aligning language model behavior without modifying model weights. It begins with standard decoding strategies, including greedy decoding, beam search, temperature sampling, and top-k/top-p sampling, and then develops guided decoding with custom logits processors.

View source on GitHub Open in Colab
Son The Nguyen

Author & curator

Son The Nguyen

AI/ML Researcher
PhD Candidate in MIS at UIC

Son The Nguyen is a PhD candidate in Management Information Systems (MIS) at the University of Illinois Chicago, advised by Professor Theja Tulabandhula. His research applies optimization and machine-learning techniques to enhance large language models and agentic AI, with a focus on data quality, alignment, controllability, and reasoning.

Discussion

Comments & questions

Join the conversation with GitHub.