DataExpert.io

Hey folks! I wanted to share my progress after the first week (Dimensional Data Modeling) of Zach Morris Wilson's DataExpert.io bootcamp. It’s been a challenging but rewarding experience so far. The lectures and labs are packed with valuable knowledge that goes beyond just watching videos. If you want to truly absorb the material, I recommend watching each video 2-3 times. Here’s my approach: First, I watch the video without taking notes, just to grasp the core concepts. Then, I revisit the lecture, take detailed notes, and redo the lab. On average, I’m spending around 18 hours on lectures and labs alone, with another 10 hours dedicated to homework. Though it’s a lot of work, I’m really happy with my progress and excited to continue learning! 💪 PS: Kendrick Lamar dropped a new album today and Zach Morris Wilson will release a new lecture tomorrow. What am I gonna do?! 😅

Yash Bhawsar in crushing it!! #dataengineering

Uma tabela dentro de outra tabela em SQL? fazer consultas históricas sem precisar de Group By? 🤯 É mais ou menos essa a ideia que o Zach Morris Wilson (Ex Airbnb, Meta) nos apresentou na primeira etapa do seu Bootcamp de Engenharia de Dados da DataExpert.io através do conceito de Cumulative Table Design. Os objetivos desse padrão são: 1. Otimizar o armazenamento de dados aproveitando a otimização oferecida pelo Run-Lenght Encoding de formatos de dados colunares. Esse tipo de codificação é comum em arquivos do tipo .parquet e permite valores duplicados dentro da mesma coluna sejam ignorados (como nomes e telefones repetidos) no armazenamento, sem comprometer a leitura dos dados originais depois. 2. Uso de tipos complexos de dados para otimização de consultas analíticas. No exemplo, o tipo STRUCT foi utilizado para modelar os resultados de jogadores da NBA em uma dada temporada. Para cada nova temporada, um novo registro foi criado, acrescentando o histórico da temporada anterior a uma coluna de array de structs, que contém os resultados de todas as temporadas daquele jogador até aquele momento. isso permite por exemplo: calcular em cima de valores históricos dos jogadores sem a necessidade de sequer realizar joins ou agrupamentos, deixando consultas tradicionalmente custosas leves e objetivas. ------------------------------------------------------------------------------ Na primeira imagem, a declaração do tipo STRUCT personalizado para os resultados de jogadores de basquete em uma temporada e o DDL da tabela, declarando a coluna season_stats como um array do tipo personalizado. Na segunda imagem, um exemplo da estrutura de um registro com uma coluna de ARRAY de STRUCTs, contendo os resultados de cada temporada de um jogador. Na terceira imagem, um exemplo de consulta que calcula o aumento do rendimento entre a primeira e a mais recente temporada de cada jogador, criada sem o uso de nenhum join, agrupamento ou window function, apenas com o uso do tipo complexo STRUCT 😯

🚀 Day 2 of the DataExpert.io Free Bootcamp! I streamed this morning, and a few people joined to ask how the start of the bootcamp is treating me. Honestly, today was a challenge! 😅 The lab required me to: ✅ Download PostgreSQL ✅ Set up DBeaver ✅ Connect a local repository to GitHub ✅ Clone a repo These were all new to me, and I struggled to get off the ground at first. But with some determination, reading documentation, and learning from tips shared on Discord, I finally got everything up and running! 🙌 I’ll be doing a surprise stream tonight to make more progress on today’s lab. Looking forward to continuing this journey and growing my skills. A huge thanks to Zach and the team at DataExpert.io for motivating me to get back into both streaming and learning. Your guidance and this community make a huge difference! 💬 For anyone else going through this bootcamp (or something similar), how did you approach these first steps? Let’s share tips and keep each other motivated! Catch me on Twitch: https://lnkd.in/ebWqpRXH #DataEngineering #BootcampJourney #PostgreSQL #GitHub #TwitchStreaming #LearningTogether

I've been following the #DataEngineering Bootcamp from DataExpert.io and Zach Wilson!! It's nice to review and rethink concepts (such as Dimensional data modeling), that if are not well implemented can lead to mess when things scale up in your Cloud Infrastructure for the DW!

Let’s see who wins!!!

What’s the secret to becoming a great data engineer? 💡 Join host John Kutay as he chats with Zach Morris Wilson in this episode of What’s New in Data. Zach, with experience at DataExpert.io, Facebook, Netflix, and more, shares his journey and offers valuable insights into the field. Listen to discover: 🔍 A hearty serving of SQL and backend development insights 🔍 AI’s role in shaping data engineering 🔍 Why the human element is crucial in a landscape driven by algorithms 🔍 The latest innovations transforming the data engineering domain 🔍 The importance of community within the data sphere 🔍 Strategies for advancing in an ever-evolving tech landscape 🔗 Tune in now: Spotify: https://lnkd.in/eWspkAPD Apple: https://lnkd.in/edUV9WYY #whatsnewindata

My 11 year data career in 11 lines: Year 1: Tableau rocks Year 2: Tableau sucks, what is this Hadoop stuff? Year 3: Utah sucks. I need to leave. Move to DC. Move to SF. I MADE IT. FACEBOOK Year 4: Data engineering sucks, what about backend? Year 5: NETFLIX GAVE ME A 130% RAISE Year 6: Cybersecurity backend is a lot. Filing patents with Netflix. Watch my entire management chain get fired Year 7: Pandemic. Quit Netflix. Time to try to be a pro gamer, at least better than Carly at Warzone Year 8: Start making content everyday, Airbnb takes a chance on a 26 year old Year 9: Burn myself into the ground doing full time content and Airbnb job Year 10: I leave Airbnb. Bet on myself and make a million dollars Year 11: Scaling DataExpert.io to thousands of students What about yours?

✅ Week 2: DataExpert.io Analytics Engineering Bootcamp ✅ Another great week and a packed schedule with the highlight being hearing from Carly and her experiences in the industry: 🌟 🔷 Tuesday: TA Support Office Hours 🔷 Wednesday: CDC vs SCD(with Snowflake) & Tech Talk with Carly, who just helped deliver Call of Duty: Black Ops 6! 🤯🎮 🔷 Thursday: Passing the Python 🐍 Interview with Zach 🔷 Friday: Data Modelling with Snowflake & Week 1 Assignment Deadline 📅 🔷 Saturday: TA Support Office Hours My goal for the rest of this week is to wrap up the assignment and get started on the Capstone Project! 💼

👉 You may have heard of Data Analysts, Data Scientists, and Data Engineers, but do you know about Analytics Engineers, ML Engineers, and Decision Scientists? There are so many exciting data roles, but with so many titles and specializations, you might wonder, "Which path is right for me?" Here's a breakdown of each role, with their unique strengths and skillsets: ---------------------------------------------------------------------------------- 🔍 Data Analysts: The Insight Hunters 🔵 Strengths: Transforming raw data into actionable insights, visualizing trends, and communicating findings to stakeholders. 🔵 Skills to Develop: Excel, SQL, Tableau, Power BI, basic statistical modeling. 🔵 Perfect for You If: You love exploring data, spotting trends, and turning complex information into digestible insights for business partners. 🧪 Data Scientists: The Experimenters 🔴 Strengths: Building complex models, predictive analytics, machine learning, diving deep into unstructured data. 🔴 Skills to Develop: Python, R, advanced statistical methods, machine learning algorithms. 🔴 Perfect for You If: You have a curious mind, enjoy experimentation, and love uncovering hidden patterns in data. 🛠️ Data Engineers: The Builders 🟢 Strengths: Designing and maintaining data architectures, ETL processes, ensuring data quality and efficiency. 🟢 Skills to Develop: SQL, Airflow, Spark, Data Warehousing, Pipelines, Cloud Data Warehouse 🟢 Perfect for You If: You have a knack for building and enjoy creating robust foundations that empower others to work with data. 🔧 Analytics Engineers: The Pipeline Specialists 🟣 Strengths: Building modular data transformations, creating reliable data models, implementing data testing and documentation. 🟣 Skills to Develop: dbt, SQL, version control, data modeling, Apache Airflow 🟣 Perfect for You If: You love creating order from chaos and building scalable, maintainable data transformations that others can trust and understand. 🤖 ML Engineers: The Deployers 🟡 Strengths: Productionizing machine learning models, building ML infrastructure, optimizing model performance. 🟡 Skills to Develop: MLflow, Docker, Inc, Kubernetes, Python, TensorFlow/PyTorch. 🟡 Perfect for You If: You enjoy bridging the gap between data science and engineering, making ML models work reliably in production. 🎯 Decision Scientists: The Strategists 🟤 Strengths: Combining data science with business strategy, designing experiments, causal inference, and optimization. 🟤 Skills to Develop: A/B testing, business strategy, R, experimental design, optimization techniques, statistical analysis 🟤 Perfect for You If: You're passionate about using data to drive strategic decisions and love designing experiments to prove causation, not just correlation. ---------------------------------------------------------------------------------- What role interests you the most? Drop a comment below! 👇 #DataScience #DataAnalytics #DataEngineering