Argilla

Today is a huge day for open source AI: Argilla is joining Hugging Face 🤗 🚀 It's time to double down on community, good data for AI, product features, and open collaboration. We're thrilled to continue our path with the wonderful Argilla team and a broader team and vision, with shared values and culture! Thanks to our investors Zetta Venture Partners (James Alcorn), Criteria Venture Tech (Roma Jelinskaite, Albert Morro, Aleix Pérez), Eniac Ventures (Hadley Harris, Dan Jaeck, Monica Lim), and many others, so lucky to have worked with you! https://lnkd.in/dfxvgpsT

Fine-tuning ModernBERT for text classification using synthetic data generation From prompt to model in 3 steps. 1 simple datasets 20 minutes of generating 60 minutes of fine-tuning on my Macbook Pro Tutorial:

Today, I annotated 42 examples from the Najdi Arabic dataset on Argilla by Hugging Face on https://lnkd.in/eUUSyabE #Annotation #Text #Dataset

Today, I took a step further into my AI exploration journey by experimenting with dataset generation using Argilla and fine-tuning a model using Unsloth AI. I generated a small dataset of 200 rows (just for experimental purposes), fine-tuned the model, and even created an app leveraging the fine-tuned model! Here's a quick overview of my process: 1️⃣ Dataset Generation: I utilized Argilla's Synthetic Data Generator to create a dataset. It was an intuitive experience, and the generated dataset was tailored to the specific task I was working on. 2️⃣ Model Fine-Tuning: Using Unsloth with its advanced fine-tuning capabilities, I trained the Llama-3.2-3B-Instruct model. The process was smooth and efficient, with the training loss starting at 1.0129 and steadily decreasing to 0.6909 by the final step, reflecting significant improvements in the model's performance. 3️⃣ Application Creation: Post fine-tuning, I built a functional app utilizing the fine-tuned model. It was an exhilarating experience to see the results of my efforts come to life. 📝 Reflection: I am not a data scientist or an AI engineer, just an AI enthusiast trying to learn and explore this fascinating domain. This journey of experimenting with datasets, fine-tuning models, and building applications continues to deepen my appreciation for the field. This experience reminded me that you don't have to be an expert to start experimenting and creating. Every small step contributes to the bigger picture. Big thanks to Argilla, Unsloth AI, Hugging Face, and Gradio I’d love to hear your thoughts on dataset generation, model fine-tuning, or anything AI-related! Also, if you’ve experimented with Unsloth or Argilla, share your experiences in the comments below. Let’s learn together. Dataset: https://lnkd.in/dAAr-vrU Model: https://lnkd.in/dUtwfGH3 App: https://lnkd.in/d7_VZfXN #AI #MachineLearning #LLM #Argilla #Unsloth #AIEnthusiast #LearningByDoing #FineTuning #SyntheticData #AIApplied

I've just contributed 77 examples to the Arabic Najdi dataset on Argilla by Hugging Face: https://lnkd.in/eUUSyabE Noticed that the examples are not fixed but change every time I refresh the page, now wondering how the website generates the examples? #LLM #Dataset #Annotations #Educational_content

The quality of a Large Language Model depends heavily on its training data. This includes the pre-training dataset which is used to train an initial base model before doing SFT/RLHF etc. While the open-source community made great progress with building and sharing open, high-quality English datasets, many other languages lack high-quality training data. Daniel Vila Suero Natalia E. Guilherme Penedo Hynek Kydlíček Thomas Wolf, and the Hugging Face community, are trying to improve this! How? Most large pre-training datasets include some quality filtering, including: 🔗 Applying URL filtering using a blocklist to remove adult content and low-quality web pages 📏 Rule-based filters which remove very repetitive or machine-generated text patterns 🌍 Language filters to ensure texts match the target language and remove mixed-language content Refining by Educational Quality? aka FineWeb-edu for all languages? Recently, the authors of FineWeb demonstrated that filtering a pretraining dataset to high educational quality could improve the resulting downstream models. This was done using a classifier trained on synthetically labelled data using Llama-3-70B-Instruct. This approach works well for English but may not work for other languages. This is where the community can help build better datasets and models for more languages. The FineWeb2-C initiative aims to create large, high-quality datasets for pretraining language models in many languages. We're doing this by building educational-quality classifiers through a community-driven effort to rate the quality of texts in many languages. Additionally, these datasets can be useful for other applications, such as providing high-quality reference data in each language, benchmarking, and improving model (synthetic) annotation capabilities. What has been done so far? After around two weeks, the community has already greatly impacted this effort. We've already released the first version of the dataset (https://lnkd.in/ep9PXZ8H), covering 12 languages, reaching the 1,000 annotations threshold. We've already seen: 34,571 total annotations submitted: 95 Languages with annotations: 321 total contributors One of my holiday plans is to train some educational quality classifiers on the current data! How to start annotating? - Create a Hugging Face Account (if you don't have one) - Visit our Argilla Space (https://lnkd.in/ereX_QmG) and login with your Hugging Face account - Select the language you'd like to annotate - Read the annotation guidelines carefully before starting - Start Annotating! Let me know if you have any questions :)

Making AI work for all of us takes us working together for open-source AI. I am joining the efforts to create a multi-lingual open-source AI training dataset, contributing to the Nepali language dataset. Join me in this collaborative annotation sprint! No experience is needed—simply follow the link to start annotating! It's easy enough to get started and to contribute. Join through the link or reach out. Cheers to the team at HuggingFace, Argilla, and FineWeb for the initiative! Argilla Daniel Vila Suero Amélie Viallet https://lnkd.in/efnRUh8U

I recently generated a chain of thought & reflection dataset using Llama 3.3 70B model. The dataset was created using the data from argilla/magpie-ultra-v1.0. The dataset is available at: https://lnkd.in/gZy-9C6N I also fine tuned (SFT) a Qwen 2.5 1.5B Instruct Model on this dataset using unsloth. The model is available here with all the details for usage and the prompt: https://lnkd.in/gpyDTWhb Future Work: Currently working on a larger dataset and on fine tuning the same Qwen 2.5 1.5B Instruct Model using DPO (Direct Policy Optimization). #opensource #transformers #reflection #chainofthought Argilla Unsloth AI Meta

Releasing the v0.1 of Gunny a fine tuned Llama-3.2-3B-Instruct I am working on for veteran support and guidance. 🔥 Fine tuned on the gunny_x and solo dolo datasets I put together a few months back using Argilla 's magpie generator. Super early version. This needs a ton of work beyond an SFT fine tune, expect better future versions over time. See it loaded up in Homebrew Research's Jan: https://lnkd.in/dN-tSWeB https://lnkd.in/dVKyz2qc

🥏 You can now push your annotated dataset directly to the Hub. 𝗔𝗹𝗹 𝘁𝗵𝗶𝘀 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝗮 𝗹𝗶𝗻𝗲 𝗼𝗳 𝗰𝗼𝗱𝗲! 🪼🪼🪼 Kudos to the whole Argilla team for this smooth release. 👉 𝗥𝗲𝗮𝗱𝘆 𝘁𝗼 𝘁𝗿𝘆 𝗶𝘁 𝗼𝘂𝘁? Get started here: https://lnkd.in/dhA-swR5 Release highlights: https://lnkd.in/dbdQXG-W