blocksml

AI is like a winnower that separates the chaff from the grain. You need to decide whether you are the chaff or the grain. Build solid skills and AI won't replace you, else prepare to be chaffed and dusted away.

Securing your Mongodb. https://lnkd.in/g3EFdhJ3

Design Pattern:- If you want to grow as software engineer writing modular, scalable clean code then design patterns are the most important components of it. Some people get traumatized with Gang of Four design pattern book and doesn't want such exhaustive list. Here are the list of few important design patterns which goes a long way to make you a solid coder. Very easy to follow and can be made a go to quick reference. https://lnkd.in/gQXMBkze

Late Chunking:- A new method for long-context retrieval in large-scale Retrieval-Augmented Generation (RAG) applications. Traditional methods either embed each document chunk independently or use advanced techniques like ColBERT with high storage and cost. Late chunking improves upon these by embedding the entire document first and then chunking the embeddings, preserving contextual information across chunks. Please follow these two articles for more details. https://lnkd.in/eXcgBzuj https://lnkd.in/dc68YzFs

Last week, there was a discussion about a new LLM that was said to be "the world's top open-source model." The name of the model was "Reflection Llama 3.1 70B". The model didn't turn out to be as good as advertised (there are currently many community discussions on whether this was an honest mistake or a scam) Either way, this got me curious to read up on the "Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning" paper (https://lnkd.in/gFE2fcyQ) -- not affiliated with the developers of the Reflection model -- which appears to be a legit way to improve LLMs through improving the dataset quality. So, what's Reflection-Tuning? In essence, it's a method that uses GPT-4 to improve the instructions and responses in a given instruction-finetuning dataset. The improved instruction-finetuning dataset can then be used to improve the LLM you are interested in finetuning (for example, a Llama 3.1 model). One of the fundamental concepts is the "garbage in / garbage out" principle in classic machine learning: the model can only be as good as your data. This is no different for LLMs: better datasets will produce better LLMs. I just implemented the dataset improvement methodology from the Reflection-Tuning paper in a standalone notebook here if you are curious about additional details and want to give it a try: https://lnkd.in/gesFpg8t PS: When I recently reviewed the recent Llama 3.1, Gemma 2, Phi-3, and Qwen 2 papers, improving (instead of just growing) the dataset was one of the biggest themes. For example, the Qwen 2 team used LLMs to generate instruction-response pairs specifically tailored for "high-quality literary data" to create high-quality Q&A pairs for training. Or, in Gemma 2, the instruction data involved using English-only prompt pairs, which were a mix of human-generated and synthetic-generated content. Specifically, and interestingly, the responses were primarily generated by teacher models, and knowledge distillation was also applied during the SFT phase.

We are excited to announce that our researchers' paper, "DualSpeech: Enhancing Speaker-Fidelity and Text-Intelligibility Through Dual Classifier-Free Guidance," has been accepted to INTERSPEECH 2024 and will be presented tomorrow at the conference. Text-to-Speech (TTS) models have advanced significantly, aiming to accurately replicate human speech's diversity, including unique speaker identities and linguistic nuances. Despite these advancements, achieving an optimal balance between speaker-fidelity—capturing the unique qualities of different speakers—and text-intelligibility—ensuring that speech remains clear and understandable—remains a challenge, particularly when diverse control demands are considered. Addressing this challenge, our researchers have introduced "DualSpeech," a TTS model that integrates phoneme-level latent diffusion with dual classifier-free guidance. This innovative approach offers precise control over both speaker-fidelity and text-intelligibility, demonstrating strong performance and competitive results compared to existing state-of-the-art TTS models. 📄 Our team will present this work at INTERSPEECH 2024, which will take place from September 1-5, 2024, in Kos, Greece. We invite you to attend our presentation during session 3, [Speech Synthesis: Paradigms and Methods 3], from 10:00 AM to 12:00 PM (EEST) on Sep 5th ▶ Research demos are available at https://lnkd.in/eVgQqk2B ▶ Read the paper at https://lnkd.in/eQr9yEnr ▶ For those interested in learning more or scheduling a meeting with our team at the conference, please contact Jinhyeok Yang or Hyeongju Kim. The contributors are Jinhyeok Yang, Hyeongju Kim, Lee Juheon, Seunghun Ji, Junhyeok Lee, and Hyeong-Seok Choi. #TTS #SpeechSynthesis #AI #INTERSPEECH2024 #Supertone

Building LLMs from the Ground Up: A 3-hour Coding Workshop https://lnkd.in/gyEvwh9n