DataOps Can Bring Certainty to Uncertain Data

In our world data impacts nearly every decision we make. In fact, data just might be the Earth’s most valuable resource, according to a seminal piece from The Economist:   

“Data are to this century what oil was to the last one: a driver of growth and change. Flows of data have created new infrastructure, new businesses, new monopolies, new politics, and—crucially—new economics. 

Digital information is unlike any previous resource; it is extracted, refined, valued, bought, and sold in different ways. It changes the rules for markets, and it demands new approaches from regulators. Many a battle will be fought over who should own, and benefit from, data.” 

Yet, despite 89% of executives recognizing data as the key to success, a staggering 75% of them do not trust their own data. And even fewer execs think their data architecture is adequate. As data becomes more detailed and complex, companies that do not make data management and data governance a priority will struggle to keep up with organizations that are able to leverage their data as a resource. 

Many CEOs and executives have been turning to DataOps as the solution, including tech startup Retina, a data analysis firm focused on consumer data for e-commerce and subscription-based companies. Brad Ito, Co-Founder of Retina, says the “best way to get good data quickly is by using DataOps.”  

And if you haven’t heard of DataOps by now, you soon will. Gartner predicts it will have fully penetrated the market in as little as 2-5 years. Already, 73% of companies have plans to hire in DataOps.  

Let’s explore the principles and concepts behind DataOps and how you can leverage such crucial principles in your company or startup.  

DataOps: Data Management as Philosophy 

Like seemingly every other concept associated with technology today, DataOps does not fit neatly into a standard or even agreed-upon definition. According to Gartner, it is a “collaborative data management practice” that focuses on intercommunication and automation of data flows between users and consumers of that data. Forrester defines it as “the ability to enable solutions, develop data products, and activate data for business value.”  

Ito, mentioned above, defines it simply as “a way of thinking about how an organization deals with data and a set of tools to automate processes and empower individuals.”  

The foundation of DataOps is built on three principles – Agile Development, DevOps, and Lean Manufacturing.  

• Agile Development: An iterative project management principle for software projects. Agile manages collaboration and innovation in DataOps. New or updated analytics are published in short increments or “sprints,” allowing for continual reassessment of priorities and allowing organizations to adapt to evolving requirements based on constant feedback, thus accelerating time to value. 
• DevOps: The precursor to DataOps. The combination of software development and IT operations, DevOps is also a set of practices and principles that help maximize the efficiency and speed of development life cycles through automation. It focuses on continuous integration and continuous delivery of software. The result? Software release cycles reduced from months to seconds, allowing for everything from a reduction in time to market, faster resolution of issues and defects, to accelerated deployment. DevOps revolutionized software development. Indeed, 99% of companies that have leveraged DevOps alone say it has been a positive impact. 
• Lean Manufacturing (specifically Statistical Process Control): A methodology that focuses on the minimization of waste without sacrificing productivity within a given system. Statistical Process Control essentially runs the data pipeline from start to finish, treating the acquisition of data similar to that of a manufacturing pipeline.  

How to Make DataOps Work for You 

Equally as important as data is the time it takes to acquire, process, and productively use that data – known as cycle time. Cycle times being too long is one of the primary reasons executives are finding the promise of data-driven analytics lacking. It is also the primary problem DataOps seeks to solve, and it does so the same way DevOps reduced live cycles for software processes.  

So how do you make it work?  

First, it is important to be clear about what DataOps isn’t. DataOps is not a quick-fix tool to solve data management issues. It isn’t a product or service you can buy, nor is it tied to any particular technology or unique architectural framework. It is a complete overhaul in how an organization approaches data management and data governance. And as such, your organization will need to take steps to implement those changes. The main steps are: 

• Treat data as an asset: Data is more than merely a by-product of business, nor is it just a tool. Data needs to be treated as an organizational asset. This means the data also needs a defined owner, ideally those who know the data best and how that data can be used. The end goal, after all, is to extract maximum value from that data.  
• Make Data Management Collaborative: “At the end of the day, DataOps is all about people,” said Brad Ito. The success of DataOps is tied to how data teams work together, but how data teams are built will be unique to each individual business. Important, though, is to allow the teams to be fluid. Data is constantly changing, and teams must be able to constantly adapt to those changes. Micromanaging or strict divisions of labor will likely compound an organization’s data troubles. 
• Create a Dedicated DataOps Team: Because DataOps is likely an overhaul of existing data management practices, it works best with a resolute team behind it. However, each organization’s data needs will be unique, so teams will need to be tailored to meet those specific needs and goals.  

Conclusion

DataOps is a broad concept and is still in its initial stages, despite having been around for over a decade. But already, companies that have leveraged the technology are already seeing significant improvements. Consider what DataOps can unlock within your organization- and don’t miss out and cultivating your company’s most important asset.  

Author:

Roxanne Dunn

(Staff Writer)