🆕 New Study Alert: LLMs can find the right answer even before finishing! 🎉 On GSM8K, they nailed up to 97% of problems, and on MMLU, a whopping 99% of answers are spot-on halfway through! 🚀 Enter the Prophet method: it speeds things up by 3.4x without sacrificing quality! ⏩✨ 💡 Here’s how Prophet works: 1. It checks the confidence gap between the top-1 and top-2 tokens at each step 2. If the gap is wide → model's already "sure" 3. Decoding stops early, locking in the remaining tokens right away! 🔒👀
🆕 New Study Alert: LLMs can find the right answer even before finishing! 🎉
On GSM8K, they nailed up to 97% of problems, and on MMLU, a whopping 99% of answers are spot-on halfway through! 🚀
Enter the Prophet method: it speeds things up by 3.4x without sacrificing quality! ⏩✨
💡 Here’s how Prophet works:
1. It checks the confidence gap between the top-1 and top-2 tokens at each step
2. If the gap is wide → model's already "sure"
3. Decoding stops early, locking in the remaining tokens right away! 🔒👀
