AI & Machine Learning · Intermediate

Reinforcement Learning from Human Feedback (RLHF)

Aligning a model by training a reward model on human preference rankings and optimizing the model against it.

Definition

Humans compare model outputs, a reward model learns to predict their preferences, and the LLM policy is optimized with reinforcement learning (classically PPO) to maximize reward while staying close to the original model. RLHF makes models more helpful and harmless.

Related terms