Of Ivory Towers, Barbarians and Causality

Fiddles and Battle Axes:

As the flames consumed the great city of Rome, Emperor Nero stood on the rooftop of his palace, transfixed by the destruction unfolding below. The sound of screams and crackling fires filled the air, but amidst it all, Nero heard another sound - the sweet strains of a fiddle. Without hesitation, Nero made his way down to his private quarters and retrieved his prized instrument. As he ascended back up to the rooftop, he began to play, the music mingling with the roar of the fire and the cries of the people.

As the fire raged on and the people of Rome fled in panic, a barbarian warrior from the north continued his work with a fierce determination. Armed with a heavy battle axe, he hacked away at the city's defenses, his eyes fixed on his target as he listened to the haunting melody of Nero's fiddle. Despite the chaos all around him, the barbarian remained focused and resolute, his mind and body consumed by the task at hand.

For a time, Nero lost himself in the music, his fingers dancing across the strings with a wild abandon. The notes rose and fell, a haunting melody that echoed through the streets of Rome. The people below looked up to see Nero on the rooftop, his fiddle in hand. Some were filled with anger and disbelief at his apparent indifference to their suffering, while others were captivated by the beauty of the music.

As the hours passed and the city burned to the ground, the barbarian never once wavered in his duty. He continued to destroy and pillage with a single-mindedness that bordered on madness, the music of Nero driving him forward like a relentless force. Nero continued to play until the flames had died down and the sun began to rise. As the smoke cleared and the city lay in ruins, Nero laid down his fiddle and surveyed the destruction with a sense of awe. “Did I play all the right notes?” pondered Nero as he sadly smiled.

The Emperor vs The Barbarian:

There is of course no evidence that Emperor Nero played the fiddle while Rome was burning and the story is completely fictional. However, it is a story that comes to mind if you ever find yourself debating a purist data scientist. The fiddle playing purists in their ivory towers can sometimes lose precious time debating endlessly on the right solutions to deploy, failing to realize that the best solution to deploy is the one that is the fastest to launch and provides incremental value over what exists today. The pareto principle states that 80% of the value is produced by 20% of the work and prioritizing this work is fundamental to creating a large impact in a relatively short span of time. The unrelenting barbarian scientist wielding his quick and dirty machine learning models and using quasi-experiments (or natural experiments) where pristine randomization isn’t possible, can be just as successful (in terms of impact and not violence) as his barbaric forefathers.

As someone who has worked in the world of data science and analytics for over a decade, I cannot stress the importance of data scientists to corporations. However, too much focus on the methodology and too little focus on the business problem can result in an analysis paralysis.  In his book “paradox of choice”, psychologist Barry Schwartz argues that an overabundance of choices while helping achieve better results, also leads to an inescapable purgatory of analysis paralysis and dissatisfaction which leads to getting nowhere on important projects. No wonder that a simple search on analysis paralysis pulls up over 47M results. With a myriad of experimental design choices, thousands of open-source packages, trade-offs between interpretability and accuracy and the importance of establishing causality, a scientist can become quickly become overwhelmed by the sheer number of available choices. This leads to long unending debates on the right algorithms to use and data models to deploy which can be counter-productive.

Ensuring that the right scientific choices are made while prioritizing product features can sometimes eclipse what matters the most to corporations: Dollar impact created. In his book “Range”, David Epstein discusses how Garry Kasparov lost to IBM deep blue which was at the time dubbed as the battle between natural and artificial intelligence. Today, simple phone applications can beat seasoned grandmasters at chess. This led to the development of “centaur” tournaments (where a human partners with a super computer to face other “centaurs”). The humans in these tournaments need not be grandmasters at chess but need to be adept at using strategy using computers to beat other centaurs. In fact, several amateur chess players have exceled in these tournaments. The same is true today in the world of data science. Barbarian scientists often outperform Roman scientists in a corporate setting. The elegant romans of course thrive in academia and research.  

With the exception of purely scientific applications (like vaccination or pharmaceutical trials), several applications focused on human behavior (like measuring marketing effectiveness or renewal impact of buying specific SKUs) need not have the same level of rigor (at least initially). In the absence of establishing true causality through randomized control trials or causal models, synthetic controls and observational studies can be amazing alternatives which can be much quicker to deploy and answer the same questions in most cases. 

Early on in my career, I was a cultured roman myself. I studied the gospel of randomized control trials and looked down upon my barbaric friends who talked about using priors to fix coefficients in their observational studies. As I laboriously tried to maintain this fanatic lifestyle, a mentor of mine cheerfully commented “We’re not in the business of saving lives. We’re trying to sell more perfume and lotion”. Over time, I converted to being a barbarian, for in the end, it proved to be more effective. Though I wield a battle-axe today and teach the barbarian arts, I still play the fiddle occasionally when the situation demands and entice my cultured fans with a nicely designed experiment and well documented DAG. 

PS: All opinions expressed in the article are personal. There is no one master key that unlocks all doors and most problems need tailored solutions.