S02-R02 — On the Limited Generalization Capability of DPO¶
Summary¶
| Title | On the Limited Generalization Capability of the Implicit Reward Model Induced by Direct Preference Optimization |
| URL | https://machinelearning.apple.com/research/reward-generalization |
| Date accessed | 2026-03-29 |
| Publication date | 2025 |
| Authors | Apple Machine Learning Research |
| Publication | Apple ML Research |
Selection Decision¶
Selected as an important counterpoint to DPO claims. Demonstrates DPO's out-of-distribution limitations with quantified accuracy drops.