Leliuga’s Post

🤔 OpenAI's latest structured outputs feature is definitely a life-saver for building LLM-based apps, but it could come with some performance trade-offs. A new paper titled "Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models" highlights this concern. 📖 Some insights: ⛳ The paper examines the effects of structured generation, where LLMs produce content in standardized formats like JSON or XML, which is common in real-world applications for extracting key information. ⛳ It shows that while structured generation simplifies parsing and integration into applications, it also has a significant downside. Specifically, LLMs exhibit a notable decline in reasoning abilities when restricted to these formats, with stricter format constraints leading to greater performance degradation. ⛳ Looser format restrictions generally improve performance and reduce variance in reasoning tasks. Parsing errors, while not the primary cause of performance differences, can be mitigated through corrective prompting. 👉 I’m not sure if this applies to the latest OpenAI models since the authors only tested it on GPT-3.5 and a few other models that might not be fully optimized for structured outputs. But it’s definitely something to keep in mind and check if you’re planning to use this feature a lot. Link: https://lnkd.in/eHRURmSH

Checkout key highlights from Gauri about new Open AI structured output feature in the API. #llm #openai #genai

💥💥💥 Qwen2.5 Technical Report Abstract In this report, we introduce Qwen2.5, a comprehensive series of large language models (LLMs) designed to meet diverse needs. Compared to previous iterations, Qwen 2.5 has been significantly improved during both the pre-training and post-training stages. In terms of pre-training, we have scaled the high-quality pre-training datasets from the previous 7 trillion tokens to 18 trillion tokens. This provides a strong foundation for common sense, expert knowledge, and reasoning capabilities. In terms of post-training, we implement intricate supervised finetuning with over 1 million samples, as well as multistage reinforcement learning. Post-training techniques enhance human preference, and notably improve long text generation, structural data analysis, and instruction following. To handle diverse and varied use cases effectively, we present Qwen2.5 LLM series in rich sizes. Open-weight offerings include base and instruction-tuned models, with quantized versions available. In addition, for hosted solutions, the proprietary models currently include two mixture-of-experts (MoE) variants: Qwen2.5-Turbo and Qwen2.5-Plus, both available from Alibaba Cloud Model Studio. Qwen2.5 has demonstrated top-tier performance on a wide range of benchmarks evaluating language understanding, reasoning, mathematics, coding, human preference alignment, etc. Specifically, the open-weight flagship Qwen2.5-72B-Instruct outperforms a number of open and proprietary models and demonstrates competitive performance to the state-of-the-art open-weight model, Llama-3-405B-Instruct, which is around 5 times larger. Qwen2.5-Turbo and Qwen2.5-Plus offer superior cost-effectiveness while performing competitively against GPT-4o-mini and GPT-4o respectively. Additionally, as the foundation, Qwen2.5 models have been instrumental in training specialized models such as Qwen2.5-Math, Qwen2.5-Coder, QwQ, and multimodal models. https://lnkd.in/d9uATxTM #machinelearning

Cohere launches Command R+, a powerful enterprise LLM that beats GPT-4 Turbo: Cohere launches Command R+, a powerful, enterprise-ready large language model optimized for real-world business applications, now available on ...

Hugging Face has done something great! OpenAI's o1 is a significant advancement in language models, but it's not yet reliable for planning tasks. PlanBench is a challenging benchmark for testing planning abilities. While o1 excels at simple planning tasks, it struggles with longer, more complex ones. Additionally, o1 often confidently gives wrong answers to unsolvable problems. Furthermore, o1 is significantly more expensive to use than specialized planning algorithms. Hybrid approaches combining language models with specialized planners may be more promising for real-world planning tasks. #AI #LLMs #Benchmark #Reasoning #HuggingFace #OpenAI #ChatGPT #Planning

With the rapid growth of artificial intelligence technology, converting spoken language into text has become an incredibly useful skill. OpenAI’s Whisper API is a powerful tool for doing just this—it can accurately turn your spoken words into written text. https://lnkd.in/gUTQB3Ah

OpenAI has introduced a significant upgrade to their Moderation API with a new model called 'omni-moderation-latest'. This advanced model, built on GPT-4o technology, offers enhanced capabilities in detecting harmful content across both text and images. The improved system is more accurate than its predecessor, particularly in non-English languages, and can assess content for various categories of harm including hate, violence, and self-harm. It also provides more nuanced control over moderation decisions by offering probability scores that reflect the likelihood of content matching specific harmful categories1 .The new moderation model brings several key improvements to the table. It can now perform multimodal harm classification across six categories, evaluating both images and text for potential harmful content. The model has also expanded its text-only harm detection to include two new categories: 'illicit' and 'illicit/violent'. Furthermore, it demonstrates significantly improved accuracy, especially for non-English content, with substantial enhancements in low-resource languages. The scores provided by the model are now better calibrated, offering a more precise representation of the probability that content violates relevant policies

🆕 OpenAI yesterday introduced GPT-4o mini, a new and affordable small model that is not only significantly smarter but also much cheaper. It is now available in the API. 🚀 🔍 A quick overview: 🧠 Intelligence: GPT-4o mini outperforms GPT-3.5 Turbo in textual intelligence, scoring 82% on MMLU compared to ~70%, and excels in multimodal reasoning. 💲 Price: GPT-4o mini is over 60% cheaper than GPT-3.5 Turbo, priced at $0.15 per 1M input tokens and $0.60 per 1M output tokens (~the equivalent of 2,500 pages in a standard book). 🔄 Modalities: GPT-4o mini currently supports text and vision capabilities, with plans to add support for audio/video inputs and outputs in the future. 🌐 Languages: GPT-4o mini has improved multilingual understanding over GPT-3.5 Turbo across a wide range of non-English languages. ⚡ Performance: GPT-4o mini is ideal for high-volume tasks, cost-sensitive tasks, and tasks requiring fast responses. It has a knowledge cut-off date of October 2023. 🌟

LLM and now SLM. Phi-3 models are the most capable and cost-effective small language models (SLMs). https://lnkd.in/gdXS4tka