How do you update your Machine Learning solution as requirements change?

When assessing the impact of the new solution part of a machine learning problem, it's important to collaborate and communicate with stakeholders. From the beginning, there needs to be as much inclusion as possible along with considerations on accessibility and ethics. There are often key performance indicators to assess the impact of the changes to the data, features, or models. It's also important to document everything on version control and this could include tools like git or Confluence. Adding emails, publications, Jupyter notebooks, scripts, or thoughts and ideas is quick and increases transparency and clarity. Ideally, we can optimize for things like the least amount of change in the user experience, latency, and ethics.

Various leadership theories can give a framework for planning the changes with stakeholders. These include Agile and tools like JIRA help assist with realizing the changes. For example, we can utilize sprint planning to identify the tasks, resources, timelines, and dependencies involved in making the changes. We can identify five stages of team development. Firstly, during "Forming," the team comes together, navigating roles and expectations. In the "Storming" phase, members voice opinions, often leading to conflicts. By "Norming," the team starts to gel, recognizing strengths and setting shared values. In "Performing," they work efficiently towards shared goals. Finally, "Adjourning" are retrospectives to understand the conclusion.

In the software development of the changes, best practices include code review by another peer and evaluation of test cases. Tools including GitLab, GitHub, and BitBucket provide a secure and collaborative platform to update the solution or app. These are often included in continuous integration of changes which allow automated updating of the changes. An important requirement in every change is to consider integration or unit tests that warn us if things will go wrong before pushing to production. When committing changes, we can utilize tags in our messages that align with Issues, JIRA, or other workflow system.

A standard way to evaluate the changes between the proposed changes and the original is to start a pull request. This gives developers and scientists a detailed understanding of the changes including text differences, unit test coverage, development deployment reports, among others. This is then used to gather feedback from the team on what can improve which can include efficiency, scalability, and robustness.

Once we have final approval, we can deploy to production. There are numerous different production cycles which can range from immediately to weekly and so on. Ideally, the systems in place allow our continuous integration to securely and quickly update to production. After deploying the solution, one of the final considerations are user acceptance. This includes communicating with customers, users, stakeholders, or colleagues about the changes.