The intersection (or lack theoreof) of Software Design and Applied Mathematics

This is a long-overdue rant. With the recent boom in Data Science and Artificial Intelligence, the broad worlds of Computer Science and Mathematics have come closer than ever. And yet, I find that the specific worlds of Software Design and Applied Mathematics are not mixing properly. I find this to be more of a cultural issue in the sense that the respective communities don’t properly understand each other, which is resulting in poor software design of models and algorithms within the space of Applied Math (which includes the modern world of Data Science and AI). I care about this because my technical interests lie in this intersection and I believe this issue is relevant to AI delivering on its promise.

Please bear with me as I express my frustrations at how these two beautiful worlds of Software Design and Applied Math should be mixing well by now, but strangely continue to stay apart. At the end of this article, I put forward some thoughts on creating a rich and vibrant intersection, and would love to hear your thoughts on this topic. For standardizing terminology, let us refer to this intersection of: A) Theory of Applied Math, and B) Implementations of Applied Math in code, as “Applied Math Implementations” (abbreviated as AMI for the rest of this article).

Clarifications on my affiliations (or lack thereof) with Programming

• My professional life has been more on the side of Applied Math than Software Design. In fact, I have never been employed as a "Software Developer/Engineer" but have had the good fortune to work with several amazing Software Developers who have influenced some of my thoughts here.
• Given my educational background in Theoretical Computer Science, I’ve always had a strong interest in and bias towards functional programming (FP), especially Haskell, Scala, OCaml. I'm absolutely fascinated with the fact that the thinking that goes into writing FP code is the same type of thinking that goes into designing mathematical models and algorithms. However, I am also aware of the performance concerns and risks of over-complications that FP languages bring.
• Although I have never been a professional developer, I have regularly coded for the past 35 years in more than a dozen languages but my coding is limited to developing prototypes or for purposes of learning/teaching (plus “just-for-fun” coding). These days, I mainly code in Python for developing math models and algorithms, and end up writing a few hundred lines of code every week.

History of Software Design and Current Languages in AMI

The world of “Software Design” was born in the 1930s when Alan Turing came up with the Turing Machine model of computation and simultaneously, Alonzo Church came up with the Lambda-Calculus model of computation. The Turing Machine corresponds to imperative programming and the Lambda-Calculus corresponds to functional programming. Although these are equivalent models of computation, they lead to vastly different mental models and vastly different programming cultures, which in turn have spawned not just very different programming languages but also very different software design paradigms. It turned out that Turing’s model was adopted (over Church’s) through the 50s, 60s, 70s (although Lisp has been around since the late 50s).

John Backus’ famous Turing Award lecture in 1977 on the FP approach being key to good software brought a lot of attention to Alonzo Church’s work and plenty of focus shifted to FP, but Fortran remained the main choice for AMI (coincidentally, Backus led the team that invented Fortran and here he was complaining about Fortran’s imperative style!). C++ was born a decade after Backus’ lecture, and yet AMI gravitated a lot more towards C++’s OOP paradigms than to the FP paradigms. Another decade later, Java showed up and strengthened the OOP paradigms. Unfortunately, these elaborate OOP paradigms made software design unnecessarily complicated/messy and Mathematicians began to look at software design as a “necessary evil” (even some Computer Scientists educated in previous decades got repelled by OOP).

The last 15 years have seen a boom in R and Python for AMI, but they both suffer from cultures of lack of emphasis on software design. Recently, these has been a great amount of appreciation for the value of FP and type specifications, which is resulting in significant enhancements in currently-popular languages (eg: functions are first-class now in Java, and Python has brought in type annotations). Today, we have 3 broad language categories in the world of AMI (with options of mixing them):

1. Performance-oriented languages like C or Rust
2. JVM-based languages like Scala or Kotlin
3. Languages with rich depth and range of libraries and convenient tools supporting Applied Math, like Python or R (Julia is an emerging language in this category)

All three of these categories offer nice features for AMI, but they also lack some importance features from the other categories. Sadly, no one language captures all of our needs in AMI. I have no interest in starting a language war here. I simply want to say that whichever language one picks, one needs to emphasize core software design principles like paying attention to data type specification, appreciating the elegance and utility of functional programming patterns, focusing on sound interface design, thinking hard about polymorphism, and recognizing that code readability and maintainability is as important as performance.

The split world of AMI

Now I get to the core of my rant. I see the following split in the current landscape:

• Most of those who are serious about Software Design view Applied Math as a strange country whose products one needs to re-package or simply use, but are not sufficiently curious about the art of that country underpinning the creation of these valuable products.
• Most of those who are serious about Applied Math view Software Design as a strange country that “over-complicates the architecture” or sometimes as “simply a means to an end”, and hence, have little inclination to learn about the art of that country underpinning the creation of elegant and robust finished products.

Thus, we end up with ML scientists and ML engineers who live in different countries, trade with each, but don’t quite work as one. Scientists are expected to code up models/algorithms in say R or Python that sort of work without an investment in software design, and then the engineers either wrap it with something “productionizable” (if that’s a word) or rewrite it in a “production-worthy” manner. Unfortunately, I am seeing these two "countries" drift apart, even though I had predicted 5 years ago that they will come together.

When I code in Python with popular packages like Pandas or TensorFlow or OpenCV or NLTK, I am appalled at how bad some of the interface design is, at the strange and unintuitive intertwining of imperative and functional patterns, and at the general apathy towards good old software design principles. What troubles me even more is that with stackoverflow as my friend, I can hack together (in Python) fairly elaborate models/algorithms very quickly, encouraging a culture of dev-speed at the cost of design. This doesn’t hurt the coding work I do to prototype or to teach Math concepts but I see this "speed trumps design" psychology having a detrimental effect on the general code quality in the space of AMI. I am often tempted to move to Scala (more recently, to Kotlin), but my dev speed here is much lower plus the general community-usage is much lower (within AMI), so I can’t quite jump ship from Python. I suspect many in AMI space feel the same. Sometimes I find myself telling my friends (in frustration) that “Python is becoming the new Excel and Scala is becoming the new Java”.

What do we need to do to reverse this drift-apart?

The quick answer is a focus on hybrid-education in Mathematics and Computer Science, which is actually being emphasized a lot in universities and in companies. But I’d like to provide a bit of a nuance here. This hybrid-education should be focused on understanding the essence of what makes the heart of Software Design beat and what makes the heart of Applied Math beat. In other words, it is important to develop not just good intuition and understanding of the topics but also a healthy sense of curiosity and taste for the other world. I think mathematicians need to appreciate why data typing is important, why modularity and interface design matter, how functional programming is not just beautiful but highly useful in practice. I think software developers need to appreciate the balance between statistical under-fitting versus over-fitting, about convergence challenges in optimization algorithms, about how linear algebra is not just beautiful but critical in practice in its numerical algorithms avatar. This means our leaders need to emphasize that their students/employees focus on foundational understanding and motivational aspects of the other world. I hope that sometime in the near future, we won't even have separate names like Mathematician and Software Developer (or Scientist and Engineer), and that technical people in AMI would seamlessly blend their Applied Math and Software Design skills to craft beautiful, robust and powerful AI products.