Superlinked

Just before the Holidays, we're celebrating 800 Github stars on our open-source. It's been a great year for all of us here at Superlinked, we're really excited for what's coming in 2025. Thanks to everyone who has supported, said hello at events and checked out our Git repo! If you haven't checked out our repo yet, you can find it here: https://lnkd.in/gwh8j5qp See you in the New Year ⭐ ⭐ ⭐

🦈 BE A SHARK.🦈 It's one of our company values. 🩸 Impatiently move in the direction of maximum 𝚋̶𝚕̶𝚘̶𝚘̶𝚍̶ value. As we now know, sharks can be quite a friendly bunch until they mistake a hapless surfer for some kind of sea-turtle. Our Superlinked Shark has been fed to the gills on added-value, and turned into this cute sticker. If you see us at an event, you might just get one. Don't worry, we won't bite! 😁

🦈 BE A SHARK.🦈 It's one of our company values. 🩸 Impatiently move in the direction of maximum 𝚋̶𝚕̶𝚘̶𝚘̶𝚍̶ value. As we now know, sharks can be quite a friendly bunch until they mistake a hapless surfer for some kind of sea-turtle. Our Superlinked Shark has been fed to the gills on added-value, and turned into this cute sticker. If you see us at an event, you might just get one. Don't worry, we won't bite! 😁

Tokenization – The Root of All Evils? 🔢👿 Tokenization is the first step in how language models process text. The translation layer between human-readable text and numbers that computers can process. Why can't we just convert each letter to a number❓ Two critical problems: ❌ Wasting massive computing power relearning basic character patterns like "th" or "ing" repeatedly. ❌ Long sequences of individual characters make it nearly impossible for the model to learn meaningful language patterns. This is why modern LLMs use Byte-Pair Encoding (BPE). 💡 BPE combines common character patterns into single tokens. Instead of seeing: "i" "c" "e" "space" "c" "r" "e" "a" "m", the AI sees: "ice" " cream" (note the space at the beginning of the token). This unlocks modern LLMs capabilities, but creates three major flaws: 🔢 Arithmetic ▪️ Historically significant numbers (like years 1930-2019) might get single tokens, while other numbers don't. This makes it very hard for models to learn basic arithmetic. 🍓 Word-Counting ▪️ Since words are broken into chunks rather than individual letters, LLMs struggle with counting 'r' in "strawberry". 🌍 Languages ▪️ Less common languages often get suboptimal tokenization since their patterns appear less in training data. Tokenization is the great paradox of LLMs. ⚖️ Their fundamental enabler also limits what they can achieve. Current research seems to find higher ROI in other improvements, this limitation remains. The future might lie in a routing approach - different processing methods for different tasks, rather than forcing everything through the same tokenization pipeline. 💡 At least, that’s how I see it—what about you?

🚀 Making Vector Search Work with Complex Data At the AI User Group’s weekly meetup, Daniel Svonava, Co-Founder & CEO of Superlinked, tackled a challenge many face: vector search in production. 🔍 While vector search is widely discussed, few have successfully implemented it at scale. Daniel introduced Superlinked, our open-source library that simplifies building vector search-based applications. Here’s what he covered: 📊 Handling metadata efficiently. 🧩 Extracting text from complex documents. 🛠️ Managing the infrastructure complexity that comes with scaling vector search. 🎯 Why high-quality vector creation is key for relevant search results. The talk was packed with real-world insights, collaboration opportunities, and a look at how Superlinked makes vector search accessible to developers. If you’re exploring vector search solutions, we’d love for you to check out Superlinked and the talk here: https://lnkd.in/ebNvZgqj #VectorSearch #AIUserGroup #Superlinked #OpenSource #AI

How has your online shopping experience been this year? The Superlinked Team went on the search for Christmas jumpers this week, the hunt led us to the website of a well-known UK clothing retailer. It looks like their systems might need a little help from our vector search technology? The top recommendation here is a Halloween jumper for pets. We weren't expecting to have to specify "clothing for humans"... 🐶 🤯 Vector search (powered by Superlinked) can help customers find what they're REALLY looking for in a matter of seconds. This is a common story across the board when it comes to e-commerce sites, their search systems simply don’t understand the semantics or relevancy of what customers are searching for. Admittedly, some are better than others, but searching with adjectives like “goofy” or “fun” was generally unsuccessful at refining the results on most of the sites we looked at. We're awarding a prize for "Search Guff of the Year" so please send us any spectacular search failures for us to add to our list. Who knows, you might win a glow in the dark dog jumper or something equally naff 👀 🐕

Time to level up your recommendations game! ⭐ These VectorHub articles will help you surface exactly what your audience wants: ⤷ RecSys for Beginner [RecSys for Beginners | VectorHub by Superlinked](https://lnkd.in/e8gqjtEM) ⤷ Movie Recommendations using a Vector Database https://lnkd.in/e_FTYY4N ⤷ How to Build a Recommender System From Clickstream Data https://lnkd.in/edfFen-W ⤷ Finding better movies using semantic search, built with Superlinked https://lnkd.in/eETF9vPH ⤷ A Recommender System: Collaborative Filtering with Sparse Metadata https://lnkd.in/eVhD58TN ⤷ Creating personalized, real-time recommendation systems with Superlinked https://lnkd.in/eEySyxhK ⤷ Step-by-Step Tutorial on Vector-Powered Video Game Search for Beginners https://lnkd.in/ewwMauTj Dive into these resources and start building. Or, if you have any related questions, feel free to reach out. What would you like to learn next? 🫵💭

Another day, another helpful video on the benefits of #vectorsearch-powered software from our CEO! 😎 Dive right in 📺 https://lnkd.in/ej7Tg_ER Comment if you've encountered the same challenges 👇

In an ever changing cycle of news reporting from across the world, relevance and recency have never been so important. In our latest Vectorhub notebook article, we break down how to build a semantic-search-powered news recommendation app, using parts of Superlinked’s library: https://lnkd.in/eCQ3rEKy Personalised content from multiple sources, without the effort of filtering through outdated and irrelevant articles. Cut through the noise using Superlinked!

A comprehensive, hands-on ML course... for free? Say no more! 💎💎💎