The Power of Big Data

Did you know that there is consistent growth of data science applications in businesses of all sizes across an array of industries?   According to research implemented by IBM the roles of data professionals have increased to over 3 million jobs, which is over a 364% increase since 2012? 

More businesses are starting to realize the powerful and necessary benefits of data science applications.  Applying machine learning and AI to your business models will require sufficient data and data professionals who can analyze and build models to solve problems, and ultimately optimize and scale your business systems.  

With modern businesses who decide to apply data science practices and techniques you can make informed decisions and augment your perspective of your business.  Data science is a super power if you know how to wield it.  

Content Summary

• Executive Summary
• The Power of Big Data
• Data Science Applications
• AI/ML + Deep Learning Overview

Executive Summary

Shopping online is normal… but that wasn’t the case many years ago. We are now entering into where we can shop in virtual worlds and realities. Some may still prefer to go outside to do their shopping. If I use myself as an example, I shop online for my groceries, I order an Uber, I shop on Amazon and get deals on certain things that I need. It s a lifestyle… that consumers are now accustomed. Our lives are optimized and changed by the convenience that technology provides.

Those from older generations went to the store, drove miles for shopping they had to purchase magazines and drive by billboards to find solicitations. However it's evident that we’ve been in a shift in lifestyle since the birth of the internet. Online we have billboards and ads displayed on our cellphones, these advertisements pop up on websites and on streaming platforms such as Netflix and Hulu.

Some of you may be wondering, what exactly does this mean and how does it relate to modern and future businesses?

All of what I’ve mentioned proposes and illustrates that we live in a data rich world. According to an IDC report, the global datasphere is expected to grow to 175 zettabytes by 2025, from just 33 zettabytes in 2018. These numbers give an idea of the massive amounts of data that are being generated and collected.

Data science is the key to the lifecycle of your data, how it will be positioned, implemented and scaled including the integrity of your data. Data governance is often left out in many AI discussions with those who all of sudden became an AI expert.

A survey by PwC showed that highly data-driven organizations are three times more likely to report significant improvements in decision-making. With this massive growth in data but also the growth of businesses globally hiring data professionals it is important to augment your business by making decisions through a data driven lens.

According to an Epsilon survey, 80% of customers were more likely to do business with a company if it offered personalized experiences. Data is critical in providing these personalized experiences by building machine learning models such as recommendation engines and systems. Amazon and Netflix are examples of these systems where products, and movies are recommended to personalize your experience.

The International Data Corporation (IDC) predicted that the compound annual growth rate (CAGR) for global spending on AI would be 18.4% by 2024, projected to reach $110 billion. This shows the growing investment in data-powered technologies. Machine learning for context has a lot to do with building many modern AI systems.

A 2020 Gartner survey found that 56% of CEOs said that digital improvements have led to revenue growth. As Data-driven technologies are an essential part of these digital improvements in order to make your business systems more efficient as well as cost reduction.

The predictive analytics market size was valued at $7.2 billion in 2020 and is expected to grow at a CAGR of 21.6% by 2026 2027, this is according to a Grand View Research paper. For context, predictive analytics uses historical data to forecast future events, which is a key tool in many industries. Predictive analytics is not a crystal ball, but what it presents is probability. Meaning you can assess the potential risk in advance in order to prescribe solutions.

Moving into the future, data is only expected to become even more critical for businesses. The proliferation of Internet of Things (IoT) devices is expected to lead to an explosion in data generation. In fact, Statista predicted that by 2025, and as stated earlier, the global data sphere will grow to 175 zettabytes, from since 33 zettabytes in 2018. This massive increase in data will provide businesses with even more opportunities to derive insights and improve their operations, provided they have the tools and skills to effectively analyze the data.

I'll be having a live discussion on AI Ethics - Register Here

Continue reading...

The Power of Data

When we talk about innovation as it relates to the business realm, innovation is a no brainer in order to keep moving forward in this upward hill of ever evolving technologies.  We came from using chariots pushed by horses to electric and self-driving vehicles, from wooden locks to the modern day metal key and even digital keys.  We are in a constant cycle of evolutionary progress of technology.  

As a business it is important to have innovation as part of your strategy,  the ability to conceive new ideas, develop them, deliver the final package, and scale new products, services, optimized processes, and business models for your customers.  Data is a massive part of that evolution.  

There is a growing body of research and case studies that delve deep into the role of data in sparking new ideas, new products, as well as processes. 

According to a report by Pwc (PricewaterhouseCoopers) 56% of companies had adopted big data analytics, which they utilized to innovate in response to an increasingly competitive global environment.  Modern businesses are not only using data but machine learning, deep learning and artificial intelligence to improve business processes but to invent new processes and products.

There is a survey by Dell Technologies that goes into how companies that are using data effectively were 50% more likely to report significant improvements in decision-making, with 36% having made gains in their innovative capabilities. Their data-driven innovations ranged from introducing new business models to improving existing products and services.

There is also a 2020 Startup Genome report that revealed that about 46% of startups are leveraging Big Data analytics to drive innovation and business growth. Many of these startups bring innovative products and services to market, essentially disrupting established industries in the process.

Regarding Artificial intelligence; AI is powered by data, it is a key driver of innovation. According to McKinsey's report, the global adoption of AI was nearly 25% in 2020, up from 18% in 2019, and these AI technologies were used to drive innovations across various sectors, most notably in healthcare to finance and many more industries.

In fact data-driven innovation is particularly notable in healthcare. A report by Stanford Medicine stated that 77% of healthcare executives found that AI applications and data have significantly changed how they manage and operate their organizations, leading to innovative patient care methods and better health outcomes.

As the technologies for capturing and analyzing data continue to advance, businesses will have even more opportunities to leverage data for innovation, improving not just their products and services but also their strategies and ways of working.  Just as I’ve stated earlier, machine learning and AI powered technologies have totally shifted how consumers operate from grocery matching in apps and delivery, how we are able to use machine learning, AI and even heuristic algorithms for Uber or GPS technologies.  

For context- heuristic algorithms are often used in Global Positioning System (GPS) technology, and it specifically functions in the part of the system that determines the optimal route from one location to another.

GPS navigation systems often use variations of the A* (A-Star) algorithm, which is a heuristic search algorithm for pathfinding and graph traversal. The A* algorithm uses a heuristic approach to estimate the cost (usually distance or time) to reach the goal from a certain node. This allows the algorithm to prioritize paths that are likely to be more efficient, and helps in finding the optimal route faster.

For instance, if you are driving and using a GPS for navigation, the system would use an algorithm like A* to calculate the shortest or fastest route to your destination, taking into account various factors such as the distance of different possible routes, current traffic conditions, road works, etc.   I’m actually going to have a room on robotics and autonomous with how heuristic algorithms and AI planning systems.  I’m also going to have an Astro Physics room to show how heuristic algorithms and planning systems can help navigate technology in outer space.  

So Heuristic algorithms can provide a good approximation of the optimal route, they are still based on heuristics, or rules-of-thumb, and so the routes they provide may not always be the best possible route, particularly in situations where the available data (on traffic conditions, for example) is incomplete or it keeps changing, but hybrid machine learning systems with Heuristic and Planning systems may help.

Data Science Applications

If it isn’t clear to you just yet, I hope that all of you realize just how much data there is.  Even as a business starts out there is more data that you either have and are not aware of how to make use of it, or there is data that you can gather and collect for your usage in order to make informed decisions.  I hope all of us are hopefully on the same page when it comes to this thought process.

The words Data Science are becoming increasingly popular and more leadership teams are seeking this type of intelligence, and thus making your competitors even more dangerous.  Machine learning and AI gives businesses the ability to optimize their processes, personalize user experiences and more.  This would mean that businesses that are effectively using machine learning and AI with the proper applications of Data Science are going to move faster and more efficiently. 

Which leads us to a discussion on Big Data…

Big data refers to extremely large data sets that can really only be analyzed computationally to reveal patterns, trends, and associations, especially relating to human behavior and interactions.  Patterns and trends that we as humans may not be able to see or be aware of.  There is also the time factor when it comes to how we manage and position big data.

Now the the concept of BIG DATA isn't just about the volume of data per day, but also its variety and velocity - often referred to as the 3 Vs of Big Data there are other Vs but for high level we can cover this for now:

Volume: The sheer quantity of data generated and stored. This can range from the terabytes of data stored by small businesses to the petabytes (1 million gigabytes) or even exabytes (1 billion gigabytes) of data stored by large tech companies.

Variety: The different types of data available, including structured data (like databases), semi-structured data (like XML files), and unstructured data (like social media posts, images, videos).

Velocity: The speed at which new data is generated and the pace at which it moves. With the rise of the Internet of Things and real-time analytics, data is being generated and processed faster than ever before. So this ties into what we’ve discussed earlier regarding just how much data that we have globally and what we would need to do with that data.  

Now Big data as it relates to data science; plays a crucial role because it provides the raw input that data scientists use to extract the valuable insights required. 

Data Collection and Storage: With the volume, variety, and velocity of big data, specialized methods and tools are required to collect, store, and manage it. This includes distributed storage solutions like Hadoop and cloud-based platforms.

Preprocessing: Big data is often messy and requires preprocessing before it can be effectively used. This can include dealing with missing or erroneous data, normalizing data, and transforming data into a format suitable for analysis.

Analysis: Once the data is ready, data scientists or data professionals would use various statistical and machine learning algorithms to analyze this data.  This can include descriptive analytics (understanding what has happened), diagnostic analytics (understanding why it happened), predictive analytics (predicting what will happen), and prescriptive analytics (suggesting what actions to take).

Insights and Decision-Making: The ultimate goal of data science is to extract insights from data that can drive decision-making. This can range from identifying trends and patterns to building predictive models. With big data, businesses can make more informed decisions, identify new opportunities, and better understand their customers and the market.  Maybe even invent new products based on predictive analytics.

In the future, as data continues to grow in volume, variety, and velocity, the role of big data in data science will become even more significant. We have the rise of technologies like artificial intelligence and machine learning which also means that we'll be able to extract even more value from big data, driving innovation and improvements across all sectors of the economy.

AI, ML & Deep Learning  Overview 

What is artificial intelligence vs machine learning, vs deep learning? 

Artificial Intelligence and Machine Learning are often used interchangeably, but they aren’t the same thing.  They are related fields with overlapping concepts, but they have very distinct differences and uses.  

AI refers to the simulation of human intelligence in machines that are programmed to think like humans and to mimic human action and behavior. The term may also be applied to any machine that exhibits traits associated with a human mind in general such as learning and the ability to solve problems.  

Artificial intelligence essentially should give the ability to computers to imitate human thought and behavior with the ability to act out or perform tasks in the real-world, whereas machine learning would be the technologies and algorithms that would give systems the ability to identify patterns, to make decisions, and improve on their own through experience and the ability to self learn and adapt just like a human would.   

I often say that Machine Learning would be one of the ways that artificial intelligence develops a form of intelligence.  

Machine learning is emphasized and is centered around the development of algorithms… It consists of methods around data analysis and automating the building of analytical models.    It is a branch of artificial intelligence but again it serves a distinct purpose outside of what would be considered artificial intelligence.  

It's based on the concepts of deploying systems that can learn from data, identify patterns, and make decisions with very little human intervention. Essentially, you feed an algorithm a large amount of data, and then the algorithm would use that data to learn and make predictions or decisions without being necessarily specifically programmed to perform whatever it is set out to do.  

Deep Learning (DL) on the other hand is a further subset of ML.  So Deep learning uses neural networks with many layers - hence why it is called 'deep' - it is structured to better model and understand complex patterns in data. So deep learning is especially effective when dealing with large and complex datasets, and it's the technology behind the most human-like artificial intelligence.

When it comes to AI, the goal is to create systems that can perform tasks that would normally require some form of human intelligence, such as understanding language or how we as humans can recognize patterns and respond. Machine Learning provides a way to reach this goal: by training on a large amount of data, an AI system essentially can learn to recognize patterns and make decisions based on those patterns.

For example, if you wanted to create an AI system that could recognize images of what race, nationality, or ethnicity someone may be, you could use Machine Learning to achieve one of a combination of all of the above.   A person may be of a certain race, but be of a different nationality than what could be considered where someone of a particular race may be.  I know there is a question of racism and bias but I am using this specific example for a reason.  We should talk about these things in order to create solutions.  

I'm having a discussion on AI Philosophy to discuss AI ethics and more.

RSVP Here