Como você escolhe entre coreografia e orquestração para implementação de padrões de saga?

A "Saga" is like a story that keeps things in order when you have to do several important tasks one after the other. Imagine you're booking a flight online. First, you choose your flight, then you enter your personal info, and finally, you pay. A Saga helps to ensure these steps happen correctly. Sometimes, things can go wrong and if a problem occurs, the Saga helps to fix it. For example, if your payment fails, it can automatically cancel your flight selection. So, a Saga is a way to manage a series of important actions and handle any problems along the way to keep everything in order and avoid chaos when things don't go as planned. It's used in computer systems to make sure complex tasks are completed correctly, step by step.

I simple terms Saga is long-running transactions across multiple microservices in a distributed system. It's a way of managing long-running operations where each microservice plays a part in a sequence of steps, ensuring that the entire process can either succeed as a whole or gracefully handle failures by undoing or compensating for completed steps.

The Saga design pattern is a method used in distributed systems to maintain data consistency across multiple transactions or microservices. It's particularly useful in scenarios where traditional ACID transactions are not feasible due to the distributed nature of the system. In a Saga pattern, a long-lived transaction is broken down into multiple smaller transactions, each of which is executed independently by the involved services. These smaller transactions are often referred to as "saga steps." If one of the steps fails, compensating transactions are executed to undo the changes made by the previously successful steps, ensuring that the system reaches a consistent state.

Choosing between choreography and orchestration for saga pattern implementation hinges on multiple factors. Choreography emphasizes decentralized communication among services, suited for scenarios with autonomous and loosely coupled services. It's beneficial when each service manages its transactions and compensating actions independently, enhancing scalability and resilience. Conversely, orchestration centralizes the coordination logic in a dedicated service, beneficial for complex workflows with dependencies or when a higher level of visibility and control is necessary. #SagaPattern #Choreography #Orchestration #Microservices #TransactionManagement #DecentralizedCommunication #CentralizedControl

In software architecture, a SAGA is a design pattern used for handling distributed transactions in a microservices environment. This involves breaking into smaller and more manageable steps or sub transactions called "saga activities". Let us consider ordering some product through an e-commerce system, starting from searching the product until the order confirmation steps, SAGA pattern enables the successful completion of end to end transaction even in the presence of failures or errors during the process. If one of the process fails, then entire process can also be rolled back and compensated without affecting the other services. There are two main models for implementing SAGA - Orchestration and Choreography.

Choreography is a decentralized communication pattern in software systems. Components or services exchange events, each deciding when to perform their tasks. It's event-driven, loosely coupled, and allows for complex, asynchronous interactions. Ideal for dynamic processes and scalability, it offers service autonomy but can be challenging for error handling and monitoring. For example, in a ride-sharing app, when a driver accepts a ride request, it emits an event that informs nearby drivers to stop searching for rides, and the user app to display the driver's location, all without a central controller.

Choreography is a decentralized and collaborative approach with no central orchestrator to coordinating interactions between independent services and here the interactions between microservices are loosely coupled. Example : Consider choreography as a team of dancers on a stage. Each participants responds to the music independently without a conductor. Here each dancer refers to individual microservice and they know when to perform the moves by listening to the rhythm of the events. This enables different microservices to collaborate by responding to events by providing an agile and decentralized approach to managing the overall process. This also supports flexibility and scalability in managing the activities.

Choreography-Based Saga: In this approach, each service involved in the saga is responsible for coordinating its own actions and compensations. Services communicate with each other using events to determine the sequence of actions and compensations to be executed.

Orchestration is like a conductor in an orchestra. It coordinates different parts (services) to create a harmonious performance (complete a task). It's needed when you have complex processes with many steps that need careful control and coordination. You decide the order and timing of each task, making sure they all work together smoothly, like a symphony. Orchestration helps control, monitor, and automate processes, making them more efficient and reliable in computer systems.

I'd like to make things more clear. Orchestration is a coordination pattern or approach that can be used to coordinate different systems to achieve a common or big goal. In other words, it's a kind of integration of several services to expose them as a single service. Which means it's not bound to the Saga pattern. When implementing the orchestration pattern, we need to ensure atomicity in some cases across multiple nodes if the function performed has side effects, and we can ensure that by using some algorithms (depending on the given use case) that fit the orchestration pattern, such as the two-phase commit and Saga pattern.

Orchestration is a coordination pattern in which it manages the flow of work and hiccups along the way. It breaks down the big task into more detailed step-by-step tasks to achieve the overall success of the process.

In Orchestration model, there is a central component known as "Orchestrator" who is responsible for the execution of the individual steps which are known as saga activities in a predefined order. The orchestrator directly communicates with each microservices to initiate, monitor, handling the entire flow and compensate transactions based on the outcome each steps. Example : Think of orchestration like a movie director guiding actors and actress through scenes. The director(orchestrator) has a script which is nothing but workflow that outlines how each actor/actress(microservice) should perform their respective tasks. The director coordinates when every scene start, how actor/actress interacts and when the movie reaches its conclusion.

Orchestration-Based Saga: Here, a central orchestrator service coordinates the saga's progress by directing each service when to execute its actions and compensations. The orchestrator maintains the state of the saga and ensures that all steps are executed in the correct order.

Choreography relies on eventual consistency as there is no guarantee on when the consuming service is going to consume the failure event and execute the compensation handler. For example, if the payment service produces a failure event and the inventory service consumes the event after a substantial delay then we might have rejected orders because of unavailable inventory. If we need immediate consistency then choreography might not be the best option.

From the high level, Choreography pattern seems very promising and it is. The implementation is also not complex because message broker like Kafka really helps. The important aspect to consider when implementing pattern is Observability. Since there is no conductor in this implementation, there is no control on how the events flow, when and which event is being triggered and when the rollback happened. So, observability is extremely crucial when choosing this pattern. - Metrics should be defined using tools like Grafana, Prometheus or AppDynamics - Traceability - If something goes wrong then it helps to find out where the problem is. Another point to consider is, testing is also not straight forward. Hope this helps.

Pros: 1. Loose coupling(e.g. a payment service doesn't need to know about the order service's internal working). 2. Flexibility(e.g. adding a new service can be integrated seamlessly without any adjustment to a central orchestrator). 3. Scalability with decentralized operations(i.e. handling more amount of orders is easier as each order service can independently scale horizontally). 4. Autonomy(e.g. an inventory microservice can independently without waiting for central instructions). 5. Easier maintenance(i.e. updating the logic in any of the service won't impact anywhere). Cons: 1. Complexity in understanding the flow. 2. Potential for inconsistency. 3. Managing long running transactions. 4. Dependency on events.

Choreography promotes decentralized decision-making and allows for more autonomy and flexibility in individual services. However, it can lead to increased complexity as the interaction logic is distributed across services. Orchestration provides centralized control and visibility over the transaction flow, making it easier to manage and monitor complex business processes. However, it introduces a single point of failure and can create tight coupling between the orchestrator and services. It depends on factors such as the complexity of the business process, the level of coordination required, and the desired level of autonomy for individual services in a microservices environment

It's better for async flow but missing event need to handle properly for service continuity. It's not a good idea to start the flow from init event but it should have possibility to restart the flow from the failed state.

Pros: 1. Centralized control. 2. Easy to handle errors. 3. Workflow optimization. Cons: 1. Tight coupling. 2. Single point of failure. 3. Less flexibility and scalability. 4. Dependency on Orchestrator.

Orchestration-based sagas offer centralized coordination, simplifying consistency and error handling. They provide clear visibility into the saga's progress and simplify tracking and diagnosis of issues. However, they introduce a single point of failure, reducing resilience. Changes to the saga structure may require updates to the orchestrator, limiting flexibility. Synchronous communication between services can impact scalability and responsiveness. Despite these challenges, orchestration-based sagas excel in managing complex transactions and ensuring consistency across distributed systems.

Orchestration pattern is also plays a key role for those applications which are complex in nature and also follows the transaction. This seems simpler but if not properly implemented then soon the conductor application can become monolithic because it has the complete control on driving the workflow. This pattern is helpful in such application which are complex in nature.

With k8 and service registry/discovery, scalability is now not a problem in Orchestration approach as well. So only coupling is a major disadvantage in orchestration pattern.

Choreography is always better if there is no need of real time data consistency on the other hand orchestration is better if we need real time consistency of data.

if our priority is to gain control so the application will be more stable and more easily measurable and modifiable, we should consider the Orchestration .

It will depend on the complexity of your system and the level of control needed. Choreography offers a simple workflow where services can react to events without needing some central controller, hence increasing loose coupling and scalability. It becomes hard to manage when the system grows up with complex interservice communication. Orchestration, on the other hand, relies on a centralized coordinator to handle service interactions and actually provides control and visibility while making it best fit for complex workflows. For ease of implementation and scalability, choose choreography; if you care about control and visibility, orchestration is the best fit.

I understand that an orchestrated approach entirely based on event-driven architecture greatly facilitates the independence between services, enabling the addition of new services to a process without altering the existing ones. Another crucial aspect is its facilitation of visualizing distributed transactions and orchestrating processes to trigger retries and transaction compensation when needed.

There can be ripple effects of a transaction as well, which means they may not be transactional in nature and yet need to react to them. These microservices should be reactive in nature responding to events and they should not be part of the transaction. Defining a transaction boundary is key to success for the efficiency of saga pattern.

To remember the Microservice Saga pattern: Orchestration: Think of an orchestra where a conductor directs musicians to play in sync. Similarly, in microservices, orchestration involves a central orchestrator coordinating service calls for a specific outcome. Choreography: Picture a dance where each dancer knows their steps without a central leader. In microservices, choreography involves services communicating directly based on events, each knowing how to react. Orchestration is like a conductor guiding a symphony (centralized control), while choreography is like dancers following a routine independently (decentralized coordination) in the Microservice Saga pattern.