Your colleague is struggling with code optimization. How can you guide them towards better performance?

"Premature optimisation is the root of all evil" - is a phrase attributed to Don Knuth and if you have worked with software this probably speaks to you already. First and foremost it is important to establish a set of tests to verify the correctness of the code. Once that is established the developer (or your colleague in this case) has the freedom to mess with the code (there is a chance that optimisation hurts readability) in different ways and use the tests to assert correctness and even benchmark the results. It is also important to consider what gains one is to expect here. Small gains might not be worth the effort in the first place - therefore it is good to discuss gains/different approaches with colleagues for clarity.

Ask Questions: Identify which parts of the application are performing poorly. > Is it a particular function? > Is it a database query? > Is it a page load? > Is it an external API call? Once you've identified a general area with performance issues: > Attempt to reproduce the issue to narrow it further, from a module to a specific query or function. > If reproducing the problem directly is difficult, perform a high-level review of the logic within the identified module. > Determine if the issue could be due to an inefficient algorithm or a problematic database query.

Understanding the root of the problem not the leafs. As most of QA, PM, clients when run an app and has a slow performance they blame the system at whole without understanding that there are multiple pieces that create the puzzle. I'm starting from my experience. We were about to upgrade the front end of an "ancient" (AngularJS ) web app. The new framework we picked was Angular, one of the best ( for me the best 😄) Since the beginning we had performance issues. How? 1. The backend APIs was slow. 2. The server didn't had the right parameters to handle it. (Several pages were going blank time by time) 3. The desires of the company were so strange to implement external packages(3 part, jQuery ect). Several of them were not optimized.

I have been doing all of this for 30 years and have developed tools that analyze and identify where the problem is and provides the solution.

Before optimizing code, identify performance issues by asking your colleague which parts are slow. Use profiling tools to find bottlenecks and understand the root causes, like inefficient algorithms or excessive queries. This targeted approach ensures your optimizations effectively address the real problems, improving overall performance.

Optimizing logic can often lead to complex, hard-to-maintain systems. It’s essential to ensure that any changes in logic offer clear advantages in both maintainability and performance. This principle is particularly relevant for legacy systems, where the existing architecture might not easily accommodate changes. Legacy systems are often burdened by outdated code that’s been patched over time, making them fragile. Therefore, any optimization should be approached cautiously, ensuring that it simplifies the codebase and enhances performance without introducing new issues. The goal is to make the system more efficient while preserving or improving its maintainability.

While optimising logic first you should try create brute force logic. When your brute force logic successfully passes all test case then try to reduce the time complexity of your solution and then try to reach optimal solution.

Actually, before trying other techniques like refactoring or parallel processing, it's important to ensure the main logic of the code is efficient. The biggest improvements often come from using better algorithms or data structures. Once the logic is optimized, other methods will be more effective.

To guide a colleague in code optimization, and identify performance bottlenecks using profiling tools. Review their code for common inefficiencies and suggest best practices, such as using efficient algorithms and appropriate data structures. Encourage the use of tools like profilers and static analyzers to detect issues. Emphasize the importance of benchmarking to measure the impact of optimizations and recommend making incremental changes. Sharing knowledge through resources and team discussions can help improve understanding and skills. Lastly, continuous monitoring in production should be encouraged, and supportive feedback should be provided, fostering a collaborative environment for ongoing improvements.

Efficiency is the art of achieving more with less. Start by examining the algorithms used; often, a more efficient algorithm can drastically improve performance. Consider using memoization or caching to avoid recalculating results that are already known. Optimize critical paths by ensuring that functions and methods perform only necessary computations.

Refactoring is the art of cleaning the canvas to reveal a masterpiece. Encourage your colleague to start by removing redundant or unused code, which not only streamlines the codebase but also reduces potential sources of bugs. Next, suggest breaking down large, complex functions into smaller, more manageable pieces. Smaller functions can be more easily tested and debugged, leading to more reliable performance improvements. Remind them to follow consistent naming conventions and modular design principles to keep the codebase organised and efficient.

Refactor to make things more readable. Sometimes it's better to have one easy to read function than a highly abstracted set of sub functions. Draw on the knowledge of more senior members of the team. Refactoring is an art and there's more than one way to go about it. It's best to be in alignment with those working on the same codebase

Refactoring is essential for improving code readability and addressing technical debt. By cleaning up redundant code and simplifying complex functions, you can reduce debt and improve overall performance, making future development and optimization efforts more straightforward and effective.

Understanding the root cause of performance issues is crucial. My experience upgrading a Node.js monolithic app to a web service API revealed unoptimized external packages in the old code and mixed functions that made comprehension difficult. Identifying specific performance bottlenecks and addressing them methodically leads to better overall performance.

Some time modularisation of the code is required to gain the effective code writing practice and making it more understandable. The refactor code must be tested with all scenarios with proper exception handling.

These are some important pointers to know about caching: 1. Distributed Caching: Use distributed caching systems like Redis or Memcached to share cache across multiple servers, improving scalability and reliability. 2. Cache Hierarchy: Implement multi-level caching, where data is cached at different levels (e.g., in-memory, local disk, and distributed cache) for faster retrieval based on access patterns. 3. Cache Warming: Pre-load cache with commonly accessed data during off-peak hours or application startup to reduce cache misses and improve initial performance. 4. Adaptive Caching: Implement algorithms that dynamically adjust cache policies based on real-time usage patterns, ensuring that the most relevant data stays in the cache.

Sometimes the best way to improve performance is to remember what you’ve already done. By storing the results of expensive or frequently performed operations, caching allows your application to retrieve these results quickly without recalculating or reprocessing them each time. This technique is particularly effective for operations involving data retrieval or complex computations. Encourage your colleague to implement caching strategies such as in-memory caching for frequently accessed data or results, and consider using distributed caching solutions for applications that scale across multiple servers

- 🚀 Accelerates Data Access - 💾 Reduces Server Load - ⚡ Enhances Application Speed - 🧠 Optimizes Resource Usage - 📈 Improves User Experience - 🔄 Minimizes Data Retrieval Time - 🛠️ Supports Scalability - 🌐 Reduces Network Latency - 🛡️ Increases System Reliability - 🔍 Enhances Performance Monitoring

Caching is important when there is latency in data retrieval but at the same time we should consider if the caching is required for real time data. Data should not be cached which is frequently getting updated. In memory technique is most crucial in this step.

When using well-tested products like Redis, caching can be a powerful optimization tool. However, it's crucial to ensure that optimization doesn't introduce additional complexity to your codebase. The goal of caching should be to enhance performance without complicating the system or making it harder to maintain. Properly implemented, caching should simplify access to frequently used data, reducing load times and improving efficiency without adding unnecessary layers of difficulty. The focus should always remain on making the system more efficient while keeping the code clean and manageable.

Parallel processing enhances performance by dividing tasks across multiple processors or threads. In Python, the multiprocessing module allows for concurrent execution of tasks, but for more advanced parallelism, consider the Message Passing Interface (MPI) with libraries like mpi4py. MPI is a standard for parallel computing that enables efficient communication between processes on distributed systems, making it useful for high-performance tasks. Utilizing MPI allows tasks to be distributed across multiple nodes in a cluster for scalable and efficient computation. MPI provides robust tools for synchronizing processes, managing data distribution, and handling inter-process communication, ensuring efficient large-scale computations.

Parallel processing isn't always a magic solution, especially for legacy systems. While modern systems are designed with multi-core processors in mind, making parallelism a natural part of their architecture, legacy systems often lack this foundation. Implementing parallel processing in such older systems can be challenging and costly, as they were typically designed for single-threaded execution. The effort to retrofit parallelism into a legacy system can introduce significant complexity, potentially outweighing the benefits. In contrast, modern systems can leverage parallel processing with minimal effort, seamlessly improving performance.

Efficiency often lies in the ability to work in parallel rather than sequentially. Parallel processing involves dividing a task into smaller sub-tasks that can be executed concurrently across multiple processors or threads. This approach can significantly reduce the time required to complete complex or resource-intensive operations by leveraging the full potential of multi-core processors.

If you want to speed up processes that could be made massively parallel consider using a GPU. If you are on Windows this is as easy as calling DirectX 12 math functions on a computer that has a GPU. If your computer doesn't have a GPU it falls back to using the CPU but still in parallel mode. If you are on Linux or Mac you can still do it but it takes more learning on your part to setup and use CUDA.

Si vous n'êtes pas familier avec le multiprocessing, si votre langage ne le permet pas, si la toolbox est trop chère ... Voici une solution possible que j'ai exploité pour réaliser le post-processing d'une très grande masse de données répartie dans une constellation de fichiers. Créer une petit fichier exécutable capable d'ouvrir un fichier, de faire du postprocessing, et d'enregistrer le résultat dans un autre fichier. Ce programme communique avec le programme qui coordonne la lecture via deux fichiers : Un pour recevoir des commandes, l'autre pour renvoyer des réponses. La suite consiste à lancer l’exécution d'autant de ces programmes qu'il est possible et de leur donner des ordres. Old school, basique, mais vraiment efficace !

1) Monitoring the application and understanding the statistics while high peak and when there is no load or traffic is crucial to understand the performance. 2) identification of important KRA could result to efficient analysis

Amélioration continue. Faire un code qui marche c'est déjà une très bonne chose, "regarder" son code tourner en s'interrogeant sur comment l'améliorer, se questionner sur son interaction avec le matériel, fera de vous un codeur encore meilleur.