30 подписчиков

⚙️ 2024 AI Inference Infrastructure Survey: 71% of companies are using open-source models

27 сентября 202527 сен 2025

1 мин

⚙️ 2024 AI Inference Infrastructure Survey: 71% of companies are using open-source models! 🎉 BentoML just dropped some fascinating findings from their survey of 250+ organizations diving into AI solutions. Here’s the scoop on how businesses are adapting their models: 1/ Surprise, surprise! Companies are juggling multiple models like pros! Their responses show a mix with options pulling in 30%-35% votes for more complex models. 💡📊 44% are rocking multi-modal embedded models, while 50% are all about those LLMs! 2/ And here’s some eye-opening usage stats: ▪️43% handle deployment in-house (compared to 59% using AI APIs and another 50% relying on vendor APIs). ▪️A whopping 71% go for open-source models, and 48% use fine-tuned versions—while only 47% stick to closed ones. ▪️Why the love for open-source? 66% crave control over data, 58% say it’s cheaper for specific tasks, and another 48% find better performance! 📈 ▪️What about the pain points? A hefty 49% struggle with creation and ma

⚙️ 2024 AI Inference Infrastructure Survey: 71% of companies are using open-source models! 🎉

BentoML just dropped some fascinating findings from their survey of 250+ organizations diving into AI solutions. Here’s the scoop on how businesses are adapting their models:

1/ Surprise, surprise! Companies are juggling multiple models like pros! Their responses show a mix with options pulling in 30%-35% votes for more complex models. 💡📊 44% are rocking multi-modal embedded models, while 50% are all about those LLMs!

2/ And here’s some eye-opening usage stats:

▪️43% handle deployment in-house (compared to 59% using AI APIs and another 50% relying on vendor APIs).

▪️A whopping 71% go for open-source models, and 48% use fine-tuned versions—while only 47% stick to closed ones.

▪️Why the love for open-source? 66% crave control over data, 58% say it’s cheaper for specific tasks, and another 48% find better performance! 📈

▪️What about the pain points? A hefty 49% struggle with creation and maintenance, while 45% face GPU challenges and 43% worry about security. Eek! 😬

▪️Where's the action happening? On-premises deployment is at 31%, but Microsoft Azure leads with a solid 42%, followed by AWS at 31%, and Google Cloud close behind at 30%.

3/ This data highlights that most companies are testing their tools (71% using open source), while others still need robust on-prem solutions (31%). But building these alone can be tough (49%)—and good luck finding GPUs (45%)! So, big corporate vendors still dominate… rumor has it OpenAI plans to spend $17B on compute by '26! 💸

4/ Question time! 🤔 When the market stabilizes, will companies stick with external vendors for convenience, or shift back to their own tools due to security concerns and on-prem needs? There will likely be a split—certain sectors can’t risk sending sensitive data out due to regulations. For now, that data stays put until self-built models become smoother and more efficient. Less sensitive info? That might stay with external vendors!

👉 Dive deeper into the report