The philosophy of operating Microsoft workloads in AWS Cloud

Abstract

Today, Cloud is increasingly embraced as a new normal. As organisations tend to move their IT operations into the Cloud, IT practitioners such as developers and sysops administrators are seeking to keep up with the bullet-train of technology. However, in today’s world it is easy to get lost amid all the technical details. This article invites readers to take a step back, and fathom about the big picture of recent trends in information technology. It is shown that, despite all its sophistication, Cloud and IT follow abstract and simple laws in their evolution. These insights throw light on the reasons behind rapid growth of the Cloud, and why falling behind might no longer be a plausible option for businesses.

Introduction

As IT practitioners, we often forget to see technical details in the context of a bigger picture. We tend to think IT has nothing to do with, say, art or philosophy. We focus all of our attention to details, hoping to maximise our intake of directly practical knowledge. It works most of the time and we get rewarded for this trait. However, sometimes we are so much drowned in technical details, that certain extremely important trends are beyond the reach of our eyes. Consequently, we might make decisions that at a later stage prove to have been regrettable.

Here, I’m inviting you to take a different approach. For a few minutes, as you go through this article, you can contemplate about the most abstract laws of nature and how they rule the most sophisticated aspects of your professional work as an IT practitioner. We will start from observations of most prominent philosophers, and try to apply those to illuminate technological trends that those philosophers themselves could never imagine!

Our starting point is, none other than, George Hegel.

Leaping from quantity to quality

“Quantitative changes, beyond a certain point, leap to qualitative changes.” (George W.F. Hegel)

For those who might not know him, Hegel is recognised as the greatest philosopher of the idealist school of thought. He is mostly known for two things:

-         First, his theory of dialectics, about which he elaborated in his famous work entitled Science of Logic.

-         Second, his philosophy of history. In summary, Hegel believed history has a purpose and moves toward rationality.

One of the most important tenets of Hegel’s dialectics is the leap from quantity to quality, as quoted above. It might initially sound very abstract and hard to understand, but a few simple examples make it quite easy.

Water and ice

A classic example is water and ice. If you pour water in a container and measure its temperature, you might get a value such as 50 degrees Celsius. Cooling down this water to 49 degrees does not affect physical properties of water. You still have liquid water with all the same physical properties as was observable when it was at 50 degrees. In other words, the change in temperature is a quantitative change, and has not resulted into any qualitative changes. As you keep cooling it down, at temperature 0, suddenly the quantitative change leaps into a qualitative change. Liquid water turns into solid ice.

Chemistry

We can also observe Hegel’s law in chemistry. Consider TNT, an explosive substance with chemical formula C6H2(NO2)3CH3. At room temperature, TNT is solid. It is insensitive to roughness and normal temperature. Heating it to 91 degrees results into melted TNT. If you pummel it with a stick or something hard, you might deform its physical shape, but its chemical composition does not change. However, if you keep hitting it harder every time, at some point a hard-enough shock will trigger a chemical reaction (in practice, this shock is often induced using a lighter starter explosive, often known as explosive booster). At this point, TNT molecules rapidly decompose in an exothermic process, quickly turning into gas forms with a thousand-fold increase in volume. This process happens to be lethal!

Once again, you can see a sudden leap from quantity to quality, manifested in the tumultuous explosion of TNT.

However, Hegel’s law asserts itself, not just in some quantities, but in all quantities. Let’s take a look at physics.

Physics

In everyday life, we understand and directly observe the laws of physics in the form of Newtonian laws that we all studied at Highschool. However, if we change our scale to very small quantities, an entirely new set of laws, such as wave-particle properties of matter, reveal themselves. These are laws of Quantum Mechanics. Likewise, if we go to extremely large scales (i.e. planets, starts and galaxies), yet another set of laws emerge, explained by Einstein’s General Relativity (e.g. the space-time curvature).

As you see, here too, change in quantity leaps to change in quality.

Evolution of science

Hegel’s law also firmly asserts itself as we move between different branches of science. In an interesting article, entitled “More Is Different”, Physics Nobel laureate, Philip Anderson, observed the following:

“The elementary entities of science X obey the laws of science Y, … this hierarchy does not imply that ‘science X is just applied Y’… At each stage entirely new laws, concepts, and generalisations are necessary, requiring inspiration and creativity to just as great a degree as in the previous one. Psychology is not applied biology, nor is biology applied chemistry.” (Anderson, 1972)

In other words, if we take magnitude of sophistication in a given scientific discipline, such as physics, as our quantitative measure, as per Hegel’s law, beyond a certain point, incremental change in this quantity leaps into a qualitative change. This means a new scientific discipline, such as chemistry, springs forth and an entirely new set of laws begin to emerge. These new laws are no longer reducible to the laws of the more fundamental scientific discipline.

Evolution of IT

Not surprisingly, Hegel’s law also applies to IT evolution! Those who have studied one of IT engineering courses might already know how it works. Everything started from very simple electrical circuits. These circuits are composed of basic elements such as resistances, inductances or capacitors. Putting these elements together, we could create electrical circuits capable of performing very basic functions.

The discovery of semi-conductors was a turning point and enabled a leap forward by adding new elements such as diodes and transistors. At that point, electronic circuits were developed. These electronic circuits enabled design of digital elements, such as AND, OR, and NOT gates.

Putting these digital elements together, digital circuits were developed. This was another revolution in IT industry. Digital design eliminated noise and the uncertainty of analogue designs. Therefore, it enabled high-precision calculations and operations.

Ultimately, by putting various digital and analogue components together, computers were invented. This was a supernova event in IT industry. Up to this point, we had to develop a new piece of hardware for every new function. However, computers were pieces of hardware that could be reprogrammed to perform entirely new functions. The birth of software that runs on top of hardware marked the beginning of the information age.

Primitive forms of software were tightly dependent upon hardware. Each computer platform understood its own language. If you needed to run the same program on two platforms, you had to redevelop the entire code in corresponding machine language of each platform. This would not be a problem if there was only one or few platforms around. But new computer platforms quickly superseded older ones.

The next leap forward was introduction of compilers and high-level languages. This meant you could write a program once, and recompile it for various computer platforms. An even more popular technology was Java Virtual Machine (and later others such as .Net Core). They allowed programs to be compiled once and run on various platforms without having to recompile them.

After that, virtualization technology emerged and completely decoupled applications and operating systems from underlying physical hardware. Businesses could run their applications on virtual machines that could seamlessly move, or span across underlying physical servers in their data-centres.

Finally, the Cloud technology enabled the final leap forward, completely eliminating reliance on hardware, and replacing physical hardware with software APIs and Infrastructure as Code.

Therefore, Cloud is the last qualitative change in the evolution of IT industry. It’s the final stage in the journey that started from pure hardware, turned into mix of hardware and software and ended up with pure software.

Ending the tyranny of hardware!

We could say, before the Cloud, hardware was always dominant and software submissive. The main implication of Cloud technology is emancipation of software from the yoke of hardware. Instead if focusing on servers and infrastructure, now IT practitioners can focus on applications that solve their business needs.

This new freedom means now we have to face problems that were not considered as problems before. E.g. as long as a server was a physical piece of expensive hardware, we had to take good care of it. Therefore, application or OS problems had to be fixed in-place. But, now that servers are nothing but pieces of software running in the cloud, we could simply throw them away and provision a new one at any time! Why would we bother remotely logging into a server, incurring hours of application outage to investigate what is wrong, hoping to fix it in-place?

Likewise, with this new power, we can deploy hundreds, thousands or even hundreds of thousands of applications in a short time. Manual deployment and configuration are no longer as plausible as they used to be.

Qualitative leaps in IT operations

We can see Hegel’s law in IT operations too. Consider following cases:

-         It’s much easier to manage a single server using GUI rather than using shell commands, but situation inverts when we move to managing dozens or hundreds of servers. Beyond a certain point, managing large numbers of servers necessitates new tools, procedures and best practices.

-         You could host an application for just a few users on a single server. As the number of users grows, you would start thinking about better ways to scale your application, such as moving to three-tier architecture, microservices, using cache layers, CDN, etc.

-         Deploying few app versions every year could be performed by manually testing the application and moving it to production. As frequency of deployments increases, new tools and best practices are needed (e.g. CI/CD).

-         If there are only few applications around, each application could handle its own authentication/authorization. As the number of applications grows beyond a certain point, SSO technologies become an obvious exigency.

-         A few MB of data can be stored in a spreadsheet format. Moving to GB scale makes spreadsheets obsolete and a relational DB could be a better option. Moving to TB and PB scale we begin to see the need for more sophisticated tools and procedures.

Thus, change in quantity beyond certain points, leads to qualitative changes in IT operations. Cloud is both a response, as well as an initiator of this change. The age of the Cloud ushers entirely new rules into the IT industry and businesses. In the same way that it’s not possible to reduce chemistry to physics, it is also not possible to reduce the cloud to just another data-centre vendor. This is the core reason that the IT industry is increasingly enthralled by digital transformations embracing the Cloud.

The legacy of the past

“Freedom is what you do with what's been done to you.” (Jean-Paul Sartre)

About 70% of applications that run on-premises today, are running on Windows platform. Many of these are legacy applications that run on old versions of Microsoft Windows, such as Windows 2008 which is approaching its end of life. Without modernisation, these applications will become vulnerable, uncompliant and potential threats to security of organisations.

Businesses are increasingly looking at the Cloud as a way to modernise their applications. However, many of these old applications don’t lend themselves to modern cloud deployment patterns. Therefore, moving these applications to the Cloud will bring many of old challenges into the new world. E.g. consider following problems:

Track assets and applications

Businesses need to understand what applications are deployed and in which environments. Solving various IT problems might need an understanding of what resources are used by each application. Businesses need to have fine-grain cost reports. Furthermore, multiple versions of each application add another layer of complexity to IT operations. Another problem is how to distribute application media, especially for Commercially Off the Shelf (COTS) applications.

Patching, security and compliance

Large organisations often have to manage hundreds or thousands of servers. Applying patch baselines to such large numbers of servers is always a challenge. Since outdated software is a vulnerability, special solutions need to be deployed to keep everything up to date. This is not just to eliminate vulnerabilities, but also necessary to remain compliant. Visibility into compliance status is another major challenge. Once a vulnerability is detected, the time to take action should be minimised as well.

Obsolescence

Today businesses and many national security authorities agree that obsolescence is one of the biggest reasons behind vulnerability. It also implies higher operation costs, as it necessitates extended support contracts and sometimes procurement of resources that have become scarce due to the obsolete status.

Obsolescence often leads to protracted and expensive infrastructure and application refresh projects. This includes refreshing in-house line-on-business applications, Open Source Software (OSS) and COTS applications.

Complex and expensive licensing

Commercial software often comes with complex and cumbersome licensing terms. These terms restrict agility of businesses and sometimes lead to unforeseen expenses. In fear of these expenses, business devote their resources to track owned vs. used licenses. Nevertheless, in many cases tracking licenses continues to be a challenge.

If there is a problem, there must be solution!

“Mankind always sets itself only such problems as it can solve.” (Karl Marx)

Before the discovery of semiconductors, no one considered miniaturisation of Integrated Circuits (IC) in the form of LSI, VLSI and ULSI as a problem. It was the semiconductor technology that enabled development and evolution of ICs. Likewise, before invention of computer software, no one thought about designing compilers and multi-platform applications.

Marx’s proposition in the above quote is a generalisation of the same logic. If something strikes us as a problem, it means we already have the means to solve it. Therefore, the IT problems mentioned in previous sections are signs of another qualitative change.

AWS Systems Manager

Whilst many applications are undergoing transformation to harness the full power of the Cloud, many legacy applications remain submissive to their run-time environment and refuse to undergo transformation. AWS Systems Manager makes management of these applications easy at large scale.

AWS Systems Manager is:

-         Hybrid: It can manage resource in AWS Cloud, on-premises or in other clouds.

-         Cross-platform: It runs across both Windows and Linux operating systems.

-         Scalable: It can seamlessly scale to manage small or massive environments.

-         Secure: It is designed to be secure.

-         Easy automation: It comes with pre-published and customisable automation documents, and allows new automation development using an easy to understand scripting language.

-         Reduced TCO: AWS Systems Manager is a free service. Furthermore, it is a managed service and does not have complexity of other commercial alternatives on-premises. Therefore, total cost of operating AWS Systems Manager is far less than other alternatives.

Let’s take one example of how AWS Systems Manager helps simplify IT operations. PowerShell Desired State Configuration (PS DSC) is a popular solution to manage Microsoft Windows workloads, and it can manage Linux workloads as well. However, those who have used it know there are challenges for deploying PS DSC Pull servers and running DSC configurations at scale.

AWS Systems Manager enables to directly run PS DSC with following benefits:

•      Cloud scale management. Apply MOFs to any number of instances by simply specifying instance tags.

•      Dynamic Token Substitution allows you to insert “tokens” into your configurations that are resolved at runtime, from a variety of sources including environment variables, managed instance tags, Parameter Store entries, or combinations (e.g. check Tags first, then check Parameter Store).

•      Integration with Systems Manager Compliance – a queryable “compliant/notcompliant” answer, as well as “what resource failed/is not compliant”.

•      PowerShell DSC integration doesn’t rely on a DSC compliance server. MOF files can be stored in S3, an HTTP/HTTPS endpoint, an SMB share, or a local file system.

•      Ability to run a “Prereboot script”. This is useful for server farms where you might want to remove an instance from a load balancer, or you might want to evict cluster resources before rebooting.

For simple but very frequent tasks, such as OS or application updates, you don’t even need to use PS DSC. AWS Systems Manager has pre-built automation documents that can perform these tasks for you seamlessly at very large scale. These operations can be triggered on-demand with a single click, or you can schedule them to automatically run on regular intervals.

Sometimes you might need to remotely connect to a server in an interactive shell to do something. Traditionally, enabling this required managing local user permissions on target servers, network infrastructure planning and opening firewall ports. However, AWS Systems Manager Session Manager allows you to have an interactive PowerShell for Windows, and Bash for Linux instances, inside your browser, without creating any local (or Domain) users on target server, and without opening any ports or having any network connectivity to that instance.

There are many other capabilities in AWS Systems Manager that solve previously complex IT problems.

Furthermore, AWS License Manager enables businesses to automatically enforce commercial license terms on their entire fleets and rest assured that they are always compliant.

Conclusion

We have seen there is a dialectical relationship between technology and innovation. New technologies solve old problems, but they also open new and unexplored horizons. This leads to a new set of problems, demanding innovative ways to overcome them. This cycle leads to incessant revolutions in technology, which changes operating environment of businesses and, by setting in the forces of competition, pushes them toward embracing new technologies.

Cloud is the new qualitative change in this ongoing trend, not as a static and ephemeral moment, but a dynamic and continuous process. The power of the Cloud, therefore, lies in its abstract form. Various moments in evolution of Cloud services and features are particular manifestations of that abstract form in their own slices of time.

From another point of view, Cloud is a response to the general need of businesses to eliminate uncertainty and risk, as it enables a shift from long-term and unpredictable capital investment model to on-demand and predictive consumption model. Superficially, it might appear that digital transformation introduces a new set of risks to replace old ones, as it involves exploration of an unchartered territory (i.e. fear of innovation). However, empirical evidence proves otherwise and astounding growth of Cloud technology, as well as success stories of those who have already embraced it, is a strong attestation to this.

DISCLAIMER: All of the statements in this article are personal views of the author and do not necessarily represent views of his employer.