Building Software Applications with Gen AI

Previous technology revolutions disrupted business users more than the IT teams that supported them. In fact, the prominence of technology as a key component of corporate strategy has increased along with increased IT budgets for every technological revolution over the last 100 years as organisations race to take advantage of the benefits these new technologies provide. IT budgets have been increasing over time as a result.

Contrary to previous trends, the advent of GenAI promises to be as disruptive to the IT teams that deploy these new technology solutions as to the business users they support. As a result of GenAI, you may actually see a decrease in IT spending over time, no more so than in application development.

In this article, I'll discuss how GenAI can be used to build software applications more cost-effectively and how It can be incorporated within those same business applications to provide more business value and reduce ongoing IT costs. By some estimates, GenAI is expected to reduce software development costs by upwards of 40 to 50%. The hardest hit will be commoditised offshore development organisations that traditionally do the lower-value development work.

This article is targeted at business executives and will give you the right questions to ask your IT Leadership teams about how they should be using Generative AI to reduce IT costs while still meeting the business objectives you've set for them. I will also discuss the risks and issues associated with using GenAI to build and support your software applications, which are manageable as long as you understand what they are.

Introduction and Background

Traditionally, Artificial Intelligence (AI) has required organisations to build a specific AI model for each individual use case, whether it was machine vision, natural language processing or other. These models, trained on specific use case data, needed to be tuned and constantly monitored for performance, updating the models regularly where performance degrades. Organisations needed to pay for the people and tools necessary to build and manage these individual AI models. As a result, AI was only really economical for higher volume use cases where the costs of AI model development and management were offset by the benefits of using AI. As a result, the broad-based adoption of AI was limited.

In 2017, the University of Toronto, in partnership with Google, wrote a research paper called "Attention is All You Need". See here if you'd like to read this paper. Without going into detail, this research paper fundamentally changed AI by giving organisations a computationally efficient way of building extremely large AI models. These extremely large models, trained on a variety of data, could be used for a variety of use cases, not just one. As a result, the concept of the "Foundation Model" was born. The term "Foundation Model" was first coined by the Stanford AI team (see here) to refer to AI models that could be applied across a variety of use cases. These Foundation Models allow organisations to adopt a build once-use many times approach. This radically changes the economics of AI by making even lower volume use cases economical. It also allows organisations to use models built by other organisations, reducing even further the investments necessary and improving the economics of AI even further. Generative AI (GenAI) is any time you use AI to generate content, whether text, images or voice. Large Language Models (LLMs) are a form of Foundational, Generative AI that is used specifically for text generation. The popular ChatGPT is based on large language models that are foundational and generative in nature. I recently published a video on what Generative AI is, which you can watch if you'd like further information and explanation by clicking on the box below.:

Foundational Generative AI models have multiple potential applications in the software development life cycle (SDLC), which makes them very disruptive for business and IT teams alike. In the subsequent sections, I will walk through each of those SDLC use cases.

Using GenAI to Build Your Software Application from the Ground Up

Those who have used ChatGPT know that the basic way a Foundational GenAI Large Language Model works is the user asks the model a question in text format (also called a prompt), and the model provides a response usually in text format but can also be an image or a video. In very basic terms, LLMs are very good at predicting the next logical set of words based on the prompt and/or previous conversation. This basic prompt and response process makes LLMs very good for a variety of use cases, including language translation. If you think of code generation as a process of translating a natural language into another language, in this case, programming language, then you can see how GenAI can be quite powerful.

Using GenAI to build the Infrastructure your Software Application sits on

So, the first step in building a software application is to set up the infrastructure on which the software application will operate. These are the so-called Landing Zones, with the compute, storage, firewalls etc, that they contain. All the major cloud providers, including Microsoft Azure, Amazon WebServices and Google Cloud, now support something that is called Infrastructure as Code or IAC. Rather than manually configuring your infrastructure each time you need a new cloud server or database, you can run an IAC script. IAC is by far the most cost-effective way to set up cloud infrastructure. By far, the most common IAC language is Terraform by Hashicorp. See here. Terraform works across all the major cloud platforms. GenAI makes the use of IAC languages like Terraform even more cost-effective by allowing you to use natural language statements to generate the Terraform code (or other IAC languages) that are then executed by your cloud provider (Azure, AWS or Google) to provision the necessary infrastructure. Therefore, using GenAI in combination with IAC can significantly reduce your infrastructure management costs!

Using GenAI for Software Code Creation Itself

Just like you can use GenAI and IAC to reduce your infrastructure costs, GenAI can also reduce your actual software development costs by directly writing software code in the language of your choice. Each of the major vendors has developed its own purpose-built software code development environments, which use purpose-built LLMs to help write code. For example Microsoft has Github Copilot (see here), AWS has Code Whisperer (see here) and IBM has Code Assistant (see here). All these GenAI code development tools have a number of things in common. They all are based on LLMs that are specifically training for code development, they all are able to code in a number of languages and they all use context (i.e. the previously written code) to help more accurately predict what the next code segment should look like based on the comments within the code. If you want a good video that talks about how GenAI generates code, see:

Using GenAI to test your developed Software

Once you've built the infrastructure using GenAI to provision your application and then have written the software code with assistance from GenAI, you will then need to test your application to make sure it is working and meeting user expectations. You can also use GenAI to write the Unit Test scripts for the code that you've written. See here for an example.

Risks and Security Considerations of Using GenAI for Application Development

Whilst there is research that shows using GenAI to write software code, IAC and test scripts can make a developer up to 40% to 50% more efficient (see here and here), organisations need to be careful on their selection of GenAI code development offerings. Some GenAI code development tools have been demonstrated to have introduced security flaws in the code they have helped to write (see here). So, I would suggest you go through a proper evaluation and selection process before settling on your tool of choice.

Overall, GenAI can be deployed across the SDLC to significantly reduce your infrastructure, software development, and software testing costs. Having used GenAI to build your application, you can now use it to reduce support costs and provide a richer experience to end users. I will provide some examples of this in the next section.

Building GenAI into Your Applications

In my first section, I discussed how you could build your software applications with GenAI. In this section, I will discuss how organisations have been building GenAI into their software applications to provide greater feature richness and reduce ongoing IT Support Costs.

GenAI for Reporting, Process Analysis and Process Automation

Once an application has been built and deployed and end users are putting information into that application to manage their business processes, one of the first requests from users to IT is to support the writing of reports to get that same information back out in a usable format to be able to operate the business with. Gen AI helps here in two ways. Firstly, you can use GenAI to allow your users to ask natural language questions to the application about the processes it is managing, and it can give you natural language responses. The GenAI bot can also take action for you as instructed. Salesforce Einstein is an excellent example of this type of functionality:

You can also use GenAI to write complex reports and dashboards for you. An excellent example of this is Microsoft PowerBI Copilot:

As you can see, using GenAI can reduce the requests to IT for ongoing support by providing your end users with a self-service channel where they can use natural language to ask questions about their business processes, take actions, and write reports—all without IT support needing to be involved.

Gen AI for End User Support

The use of conversational agents to provide a self-service support channel for your end users is nothing new. But using GenAI with your self-service support channel can dramatically improve the customer experience in various ways, including, but not limited to...

1. Providing End Users with the ability to source better-quality answers themselves
2. By Automated the allocation of support tickets to the correct support analysis for faster resolution times
3. By using GenAI to recommend to your IT support analysts the next best-recommended action to take to make triage faster
4. By summarising a support call, taking notes and assigning actions to allow the call centre analyst to move on quickly

According to Gartner, by 2026, 50% of customer service and support organisations will have implemented Gen-AI-driven Virtual Agents, driving a 20 to 30% reduction in labour costs. See here. So, it is clear that using GenAI to reduce the ongoing support costs of business applications is a heavy area of investment.

Gen AI with Retrieval Augmented Generation to Provide a Better User Experience

Retrieval Augmented Generation (RAG) is the idea that the responses of an LLM can be improved by providing the LLM with a large database of information and asking the LLM to source answers from that database when making it's responses. This reduces the impact of hallucination and provides end users with easy access to high quality information. If you want a better understanding of the concept of RAG see a previous blog article I've written see here.

As GenAI technology improves, the organisations that have GenAI in their business applications will win the GenAI race. Because it is not GenAI that is powerful, but GenAI + High Quality Data is powerful. Examples of this business strategy are Microsoft 365 Co-Pilot and Salesforces Einstein. Both are GenAI solutions that are tightly bound to the business application. GenAI will become a source of competitive advantage for these organisations as companies pick application vendors that have GenAI built-in over those that don't and question the cost of building their own GenAI solutions when the application vendors have already connected GenAI to the data for them.

Risks and Considerations of Building GenAI into Your Applications

As with any application functionality, security and the use of role-based access control are important. While GenAI provides you greater access to the information within your application and significantly reduces the number of ongoing requests to IT for application support, it does create the potential for the wrong information to get into the wrong hands and provides a greater attack service for malicious actors to hack into your application. Therefore, security and tightly designed role-based access control will be critical to avoid issues with this greater access to information.

Conclusions

In conclusion, as a business user, you should ask your IT Leadership the following question: Are you using GenAI to...

1. Reduce infrastructure provisioning costs
2. Reduce software development by 40 to 50% costs
3. Reduce software testing costs
4. Reduce the costs of custom report development and the costs of process analysis and management by 40 to 50%
5. Reduce the cost of supporting that software with end users through self-service challenges by 20 to 30%
6. Provide a richer and more cost-effective end-user experience

... because if you are not, you are missing the full benefits of GenAI and probably paying more for your IT services than you should be! There are risks and considerations with using GenAI in Application Development and within your applications, but they are manageable as long as you understand those issues and risks.

Dr David Goad is the CTO and Head of Advisory for IBM Consulting Australia and New Zealand. He is also Microsoft Regional Director. David is frequently asked to speak at conferences on the topics of Generative AI, AI, IoT, Cloud and Robotic Process Automation. He teaches courses in Digital Strategy and Digital Transformation at a number of universities. David can be reached at david.goad@ibm.com if you have questions about this article.