Apache Doris

A thorough comparison between Apache Doris and Elasticsearch by Kang, Apache Doris PMC Member and key developer https://lnkd.in/g3yGM28r

Revisiting Pengfei Zhang's informative talk at the AWS × Apache Doris Meetup: 💡 "While many focus on models, the data foundation is key to deriving value from generative AI applications." 🌟 Data architecture for GenAI applications: ⬆️ Core GenAI features Amazon Bedrock ⬆️ Lakehouse & vector database VeloDB Zilliz ⬆️ Data integration & orchestration WhaleOps Technology ⬆️ Open data lake #S3 #Iceberg #Hudi #DeltaLake 🌟 AWS provides GenAI tools on 3 levels: 1️⃣ Infrastructure to build & train AI models 🔍 SageMaker AI: a one stop place to build, train and deploy your machine learning models at scale. 🔍Trainium & Inferentia: purpose-built chips for AI training and inference 2️⃣ Models & tools to build GenAI APPs 🔍 Amazon Bedrock: make it easier to access various LLMs and customize them while maintaining security & privacy. 🔍 Amazon Bedrock Guardrails: evaluate the prompt and the output from the model to avoid potential harmful content 🔍 Amazon Nova: SOTA foundation models that deliver frontier intelligence & industry-leading price performance 3️⃣ Applications to boost productivity 🔍 Amazon Q Business: connect your organization data (from Salesforce, Sharepoint, Google Drive, etc.) and combine it with LLM to implement a RAG system 🔍 Amazon Q in QuickSight: use natural language to complete common BI tasks 🔍 Amazon Q Developer: coding assistants that helps throughout the full software development life cycle, while making sure your code is secure and compliant 🔍 Amazon Q Connect: for specialized cases Watch full replay on YouTube: https://lnkd.in/g-cwWydP #AI #AWS #BigData #Database #DataScience

I've uploaded material files that mentioned in the "Demo: Building an End to End Data Warehouse Solution with Apache Doris" by VeloDB This repository showcases the complete process of building a Data Warehouse (DWH) and Business Intelligence (BI) dashboard using Apache Doris, Apache Hop, and Metabase. https://lnkd.in/gqbqcHb8 Feel free to share feedback!👏

Matt Yi, Apache Doris PMC Member is presenting on "Real-Time Warehouse: Powering Modern AI & Analytics": https://lnkd.in/gXZPqWK6

Check out the Apache Doris 2025 Roadmap and join the discussion 🙌 https://lnkd.in/gnrEVT2u 🌟 Focus: lakehouse & semi-structured data analysis 🌊 Continuous efforts: optimize query execution, storage, and query optimizer to further improve performance, stability, and ecosystem compatibility To list a few: 🌱 Open table format 🌱 Inverted index enhancement 🌱 VARIANT data type enhancement 🌱 Vector search ... #opensource #apachedoris #database #GitHub #analytics #bigdata #dataengineering

Many members of the Apache Doris community have developed their own ecosystem tools to extend the capabilities of Apache Doris and address various use case needs. We’d like to recommend these tools to users who may find them helpful, and we applaud the developers for their excellent work and dedication to the open-source spirit! 🙌 1️⃣ Doris Loader by Ray Lin https://lnkd.in/gtGNsAeQ Different from the Apache Doris Stream Loader (https://lnkd.in/gKCiAS5C), the above is a library designed to be directly embedded in user applications, making it easier for developers to integrate data loading functionality into their own applications. 2️⃣ Metabase-Doris-Driver by @ihadoop(GitHub) https://lnkd.in/gHVMiYKi It helps better sync Doris data into #Metabase and it is found helpful by some Doris users.  (P.S. if you're interested in building a solution using Apache Doris as the OLAP engine on Metabase, join this live webinar: https://lnkd.in/g4T9f_Jp) 💡 Please note that these tools are independently developed by individual developers and have not been officially included in the Doris project. They have not undergone rigorous testing by the Doris team, so we encourage users to evaluate them based on their specific needs.

In February, we have a few Apache Doris community events coming up, including both live webinars and in-person meetups. 👨👩👧👦 Live webinar: Building an End-to-End Data Warehouse Solution with Apache Doris The speaker Wandhana Kurnia is an experienced data warehouse engineer. He will showcase a demo of using Apache Doris as the OLAP engine on Metabase. Register here: https://lnkd.in/g4T9f_Jp 👨👩👧👦 AWS × Apache Doris Meetup: Building a Global Big Data and AI Ecosystem with AWS and Apache Doris in the GenAI Era ⭐ Sessions 1️⃣ Generative AI with AWS Amazon Web Services (AWS) 2️⃣ Real-Time Warehouse: Powering Modern AI & Analytics Apache Doris 3️⃣ Transforming Observability Platforms for the GenAI Era TrueWatch 4️⃣ WhaleStudio Empowers Real-Time Data Pipelines in the Era of Large Models WhaleOps Technology 5️⃣ Zilliz & Vector Databases: Driving Innovation in the Big Data & AI Ecosystem Zilliz The Apache Doris speaker, Matt Yi, has been leading and contributing to many core features of Apache Doris. He is one of the key driving forces behind Apache Doris' breakthroughs in query performance across various data analytics scenarios, helping establish it as an industry-leading tool. Register here: https://lnkd.in/gGUjzTcn Come join us to engage with the community, exchange technical insights, and connect with more Doris users and big data professionals. We look forward to meeting you! #opensource #meetup #webinar #dataengineering #database #AI

1️⃣ What's new about the #Amazon #S3 Tables? "For database professionals, the emergence of it signifies that the era of modular data analytics has arrived." "f we consider standard S3 Buckets as rough, unfinished houses, then Table Buckets are like fully furnished, move-in-ready homes." 2️⃣ How to build a simple data lakehouse with Apache Doris + Amazon S3 Tables? "S3 Tables are compatible with the Iceberg API, and Apache Doris already has robust support for the Iceberg table format." ... (a hands-on guide) 3️⃣ What's the big deal about lakehouse? "In big data processing, users can use #Spark / #Hive to perform batch data processing on #Iceberg; use #Flink / #RisingWave to stream #OLTP database changes into Iceberg through CDC or conduct stream processing; and use Apache Doris / #Trino to perform interactive query analysis on Iceberg. On the other hand, users can also use # DuckDB / #Daft to access Iceberg on their Macs at any time and explore data they are interested in. They can also process AI training data on Iceberg through the #PyIceberg project. Excitingly, all of this happens on the same dataset. We no longer need to rebuild clusters, create tables, and import data for temporary data analysis needs. Nor do we need to worry about data credibility and inconsistent data quality. Everything becomes quite natural and flexible." This incredibly informative and insightful article by Mingyu Chen is totally worth a read! https://lnkd.in/gpkvpikS

The tech world has thrown its spotlight on #DeepSeek for the past months. As AI thrives on big data and in turn, empowers it, how does the data world reacts to this continuous wave of AI innovation that consistently challenges and expands our perspectives? The #AWS × Apache Doris Meetup is set to bring together the brightest minds in #BigData to explore how we can build a robust, global ecosystem in the Generative AI era. Prepare for an exciting lineup of sessions and discussions from Amazon Web Services (AWS), Apache Doris, TrueWatch, WhaleOps Technology, and Zilliz RSVP now to secure your spot and be part of the movement that’s setting the stage for the next wave of technological innovation! https://lnkd.in/gGUjzTcn

Announcing AWS × Apache Doris Meetup in Singapore! 🚀 Join us on February 24 for the topic of Building a Global Big Data and AI Ecosystem with AWS and Apache Doris in the GenAI Era. Prepare for an exciting lineup of sessions, including keynote speeches from Amazon Web Services (AWS) and Apache Doris, as well as in-depth talks from TrueWatch, WhaleOps Technology, and Zilliz. Connect with fellow data professionals and discover how these technologies are shaping the future of data processing and AI. See full agenda and save your spot 🙌 : https://lnkd.in/gGUjzTcn