Wisdom over (Artificial) Intelligence

The fast evolution of Machine Learning techniques, computer hardware, and data availability has made it easier than ever to tackle engineering problems with a black-box approach. Through statistical methods of feature selection and dimensionality reduction, we can completely detach ourselves from the phenomenon we’re dealing with and focus instead on getting a “better score”. Although appealing in its apparent versatility and simplicity, this brute-force paradigm is incredibly wasteful in terms of computing resources and, more importantly, human understanding of reality.

Knowing how to approach ignorance has always been at the core of science and engineering; take for example the Buckingham π theorem. Formal definitions aside [1], this theorem professes that by cleverly multiplying the variables belonging to a physical system, it is possible to define a new set of a-dimensional parameters that determine its behaviour. This observation allowed for the study of large-scale and micro-scale phenomena through parametrically equivalent directly observable experiments, all of this before having a numerical model; and that is the main takeaway from this story: the complexity of a modelling problem can be heavily reduced by first questioning the reality of the data we are observing.

Going back to the subject of AI, this line of thought has already succeeded in the field of Machine Learning with the introduction of Physics-informed neural networks (PINNs). Here, we find ourselves one step ahead of the above theorem in the sense that we may already have numerical models, but with these being either analytically unsolvable or numerically too complex to solve within a reasonable time window. Such is the case of the dynamic Navier-Stokes flow equation [2], where instead of relying on data points generated by an exact numerical simulation (which would be inefficient for the number of samples required to train the network), the loss function is instead defined by combining the PINN’s estimation error at the boundary and initial state of the flow (as would be the case with a traditional numerical method), and its coherence with the Navier-Stokes differential equations, which allows for virtually infinite singular training samples.

The benefits of approaching AI design with a heuristics-first mentality aren’t limited to modelling applications. In the field of computer vision, before Convolutional Neural Network (CNN) frameworks like YOLO basically took over the field of object detection, extensive heuristic feature engineering was fundamental before even attempting to train a Machine Learning model. In fields so tightly connected to human experience such as Written Character Recognition [3], where common shapes we as humans are used to drawing can be targeted for agile discrimination, or even in more technical fields such as image-based fault detection [4], where certain textures are expected to develop in damaged components, deterministic feature extraction can be easily exploited. While it is true that, behind the scenes, CNNs are probably able to extract very similar features after several filtering layers, it is also true that they require far more computational resources than deterministic feature extractors, while also hiding their relevance inside of the network, thus inhibiting any kind of ulterior analysis.

In conclusion, following the excursus of applied cases described, we can make a strong case that it is extremely important for the future to confidently wield the power of human imagination, in addition to the aid that can be provided by AI and machine learning processes. At a point in human history where technology no longer bottlenecks creativity, it shall be the latter what ultimately separates an AI user from an Engineer.

[1]  E. Buckingham (1921) LXXIX. Notes on the method of dimensions, The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 42:251, 696-719, DOI: 10.1080/14786442108633812

[2]  X. Jin, S. Cai, H. Li, and G. E. Karniadakis, ‘NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations’, Journal of Computational Physics, vol. 426, p. 109951, 2021.

[3]  G. Kumar and P. K. Bhatia, “A detailed review of feature extraction in Image Processing Systems,” 2014 Fourth International Conference on Advanced Computing & Communication Technologies, 2014. doi:10.1109/acct.2014.74

[4]  K. Kaplan, Y. Kaya, M. Kuncan, M. R. Mi̇naz, and H. M. Ertunç, ‘An improved feature extraction method using texture analysis with LBP for bearing fault diagnosis’, Applied Soft Computing, vol. 87, p. 106019, 2020.