🖼 Improve Vision Language Model Chain-of-thought Reasoning

This research paper investigates how to improve the chain-of-thought (CoT) reasoning capabilities of vision language models (VLMs). The authors address the lack of high-quality CoT data for training VLMs and propose two key methods: first, distilling rationales from a powerful language model (GPT-4o) to enrich the training data and fine-tune VLMs, leading to significant improvements in CoT performance. Second, they leverage reinforcement learning (RL) through the Direct Preference Optimization (DPO) algorithm to further calibrate reasoning quality, utilizing positive and negative pairs of model-generated reasoning chains. The authors demonstrate that their approach effectively enhances reasoning capabilities, paving the way for more robust and interpretable multimodal models.

📎 Link to paper