Creativity in Data Science

Most people in the data science space might find it hard to say what they do is creative: writing SQL queries, building models, designing A/B tests, and creating dashboards. That all sounds pretty technical--where does the creativity come in? I wondered the same thing until I took the time to research creativity and understand what it is.

Turns out, our biggest mistake is confounding creativity with the arts (i.e. only a painter, writer, musician, or actor can be creative). Either that, or we think that only the most brilliant minds can be truly creative--those that win Nobel prizes or make breakthroughs in their fields. Well, this isn’t quite the reality...

Businesses all over the world rely on creative people in all fields to move their business forward--to continue to innovate and win against the competition. So, what is creativity then?

In the words of Ken Robinson “creativity is the process of having original ideas that have value”. So, it boils down to three things:

1. First and foremost, it’s a process. It’s not a moment in time, but something that happens over time. As John Doerr said "Ideas are easy. Execution is everything. It takes a team to win". Coming up with the idea is only part of the process--proper execution on that idea will take time.
2. Next, the idea needs to be original. Original ideas don’t have to be new to the whole world, they just have to be new to you or to your community. Take Henry Ford for example: his inspiration for the car assembly line came when he saw the disassembly lines used to process meat! Not a new idea, but new to his industry!
3. Lastly, it has to be something of value. Is this worth-while? Judge the ideas on whether they are useful. 

Now when it comes to creativity, there is one main prerequisite. It’s the necessity of domain knowledge. For example, you can’t be creative on the piano if you don’t have some familiarity with piano. Sure, you can find a way to play some keys, but it won’t be the same as if you had been playing for several years. The same thing applies to other domains like mathematics—imagine being creative in your approach to solving a problem if you didn’t understand its mathematical underpinning. So, domain knowledge here is key. 

So, what is creativity in data science? We say it’s important and seek creative candidates... but how do we define it and cultivate it in ourselves and in our teams? Before saying what creativity in data science is, let’s talk about what it isn’t...

Well, it isn’t brute forcing your favorite algorithms to a dataset.

This approach will lead you to miss much of what is critical to that domain (be it script writing, gaming, finance, etc.). You might end up neglecting to account for an important feature while modeling or limit your analysis resulting in uncovering only the obvious. New and creative ideas will come when you take the time to understand the nuances of the domain and ask the hard questions of your data.

That leads us to our definition of creativity in data science, which is: the marrying of statistical knowledge with domain knowledge and a little bit of mind wandering.

This touches on the importance of both domains (analytics and industry), but also highlights this idea of allowing your mind to wander. This means keeping an open mind while experiencing or learning something new and imagining how you could use that new knowledge and apply it to a problem you are trying to solve.

More tangibly, creativity in data science can be anything from innovative features for modeling, development of new tools, cool new ways to visualize data, or even the types of data that is pulled for analysis. What’s interesting in data is that everyone will do things differently, depending on how they think about the problem. When put that way, almost everything we do in data science can be creative if we think outside the box a little bit...

The best way I could think to describe creativity in a candidate or in an approach is when they give you this moment of "wow, I never thought about it that way." Ideally as a company or team, you want to maximize the number of moments like this--keep good ideas flowing, prioritize, and execute.

How are you being creative in your work today?