The first step in designing your solution is to understand the requirements of your problem domain, your target audience, and your business goals. You need to define the functional and non-functional requirements of your solution, such as the features, the user interface, the data, the security, the availability, the latency, the throughput, and the cost. You also need to identify the constraints and assumptions that affect your solution, such as the technical environment, the regulations, the dependencies, and the risks. By understanding the requirements, you can establish the scope, the quality, and the feasibility of your solution.
Step 1: Review the AWS Well-Architected Framework. It lays out all the best practices for security, cost optimization, operational excellence, reliability, performance efficiency, and sustainability.
Being able to speak the business language is as important as being able to speak tech. Architects often times play the translation role between business and tech in the design phase, which is crucial for expectations and goals to be aligned.
Cell-based architectures are commonly used in various domains, including distributed systems, microservices architectures, and event-driven systems. They provide a means to tackle complexity, enable scalability, and support the evolution of systems over time. By leveraging the principles of modularity, encapsulation, loose coupling, and autonomy, cell-based architectures offer a robust and flexible design approach for building complex software systems that are scalable, reliable and performant
When designing complex systems, consider scalability, performance, reliability, and impact. Understand the solution, organizational impact, customer experience, reputation, and business outcomes. Match system investment to business value. For example, quantify downtime or performance degradation to the business impact or revenue to get perspective on acceptable levels. Design thoughtfully by weighing technology and business factors like cost vs value.
The first step is often to gain an understanding of the language the business is speaking. Different business sectors, different businesses within a sector, different departments within a business, and so on can all speak a different language. Learning the language of the individuals you have access to is the fundamental first step in understanding their perspective on the requirements. Looking for overall requirements needs us to understand this from a variety of perspectives, to look for common understanding, to mediate on conflicting requirements (actual or perceived), and to drive towards concensus on solutions.
The next step in designing your solution is to choose the right architecture style for your solution. The architecture style is the high-level pattern that defines how your solution is organized, structured, and communicated. There are different architecture styles that suit different types of solutions, such as monolithic, microservices, serverless, event-driven, and hybrid. You need to choose the architecture style that best matches your requirements, your constraints, and your trade-offs. For example, if you need to scale your solution quickly and independently, you might choose a microservices architecture. If you need to minimize your infrastructure and operational costs, you might choose a serverless architecture.
A cruical part of choosing a design is also being able to efficiently operate it. While monoliths can be internally complex, switching to microservices does not necessarily fix the complexity, rather it sometimes simply moves it on to the network. Being able to operate a complex set of interconnected applications might be just as complicated as operating a monolith. A successful architecture needs to be able to be fully supported and operationalized.
When choosing an architecture style, there are several factors that companies should consider, like company size and resources, application complexity, and business goals. A monolithic architecture may be a good choice for small to medium-sized organizations with simpler applications. This approach is easy to manage and deploy, and it can be cost-effective. However, monolithic architectures can become difficult to scale and maintain as applications grow in size and complexity. A microservices architecture is usually a good option for larger organizations with complex applications. Of course, there are many factors to consider. The best architecture style for a particular organization will depend on its unique needs.
Architectural style is driven as much by the organisation of the engineering teams as it is from the requirements of the solution. Microservices mostly only make sense where there are teams requiring autonomous delivery. Monolith usually is the best option for very small engineering teams. We should be careful to factor the engineering organisation into choice of style rather than having sole focus on requirements.
The architectural style is the result of the weights and the trade-offs you as the solution architect need to decide for each of the six pillars of the well-architected framework I mentioned in my other post, i.e. operational efficiency, security, reliability, cost-effectiveness, performance and sustainability. You can’t have them all. It’s your job as a solution architect pick and choose which one is the most relevant and derive an architectural style that best suits your selections.
There are several key factors to consider when making this choice - System Type and Requirements: for Web applications consider microservices for modularity and scalability, or serverless for cost efficiency and scaling based on demand. For Data-intensive applications opt for distributed databases, data lakes, or stream processing frameworks. For Real-time systems go for event-driven architectures for responsiveness and scalability, or specialized platforms for message queuing. - Scalability: architectures that allow Horizontal and Vertical scaling - Performance: architectures with efficient resource utilization, parallelization and low communication overhead between components - Reliability: Implement redundancies in critical components
The third step in designing your solution is to apply design principles that will guide your design decisions and ensure scalability, performance, and reliability. Modularity, loose coupling, high cohesion, separation of concerns, abstraction, encapsulation, inversion of control, polymorphism, reusability, configurability, extensibility, and testability are all important design principles to consider. Modularity divides your solution into smaller components that are easily reused and tested. Loose coupling reduces dependencies between components to avoid complexity. High cohesion ensures components have a single responsibility and function. Separation of concerns separates your solution into different layers with different roles. Abstraction hides implementation details. Encapsulation protects internal state and behavior from external access. Inversion of control delegates control and dependency of components to external entities. Polymorphism allows for different implementations and behaviors based on type or context. Reusability makes components reusable across solutions and scenarios. Configurability lets you configure components through external parameters or settings. Extensibility allows for inheritance, composition, or plugins. And testability ensures components can be tested through unit tests, integration tests, or end-to-end tests.
No matter what the chosen strategy is, make sure to leave plenty of room for experimentation, as it is the key to innovation.
Choosing a platform that inherently provides these design principles is very important for your solution. These design principles make your solutions "Build For Change". An evolving solution Transforms the organization. Makes the organization stays competitive and allows Innovation.
When architecting complex systems, it's crucial to plan for the unexpected. The most robust systems anticipate failures and build in resiliency from the start. Rather than hoping for the best, engineers should assume components will fail at some point and prepare for uncertainties. Every dependency and integration point is an opportunity for problems. By modeling potential failure domains, we can add monitoring, circuit breakers, and redundancy where needed. Well-designed systems won't prevent all problems, but they mitigate downtime and data loss when the it happens. Building failure-aware systems takes forethought, but pays dividends in production.
The fourth step in designing your solution is to use the appropriate technologies that support your architecture style, your design principles, and your requirements. There are many technologies that you can use for your solution, such as programming languages, frameworks, libraries, tools, platforms, services, protocols, standards, and patterns. You need to choose the technologies that best fit your solution's needs, such as the functionality, the performance, the scalability, the reliability, the security, the compatibility, and the maintainability. You also need to consider the trade-offs and limitations of each technology, such as the learning curve, the cost, the support, the documentation, and the community.
A couple of additional considerations as they apply to technologies are vendor lock-in and technology lifecycle management —TLM. You as a solution architect need to think strategically rather than tactically and picture yourself (or in the role of another solution architect responsible for the same app) a couple of years from now. Will the technology your app was built on still be maintained by the vendor? Will there be limitations that impact the six pillars of the well-architected framework?
Be ready, willing, and able to do POCs, POVs at this stage to test and validate theories of how tech "A" performs -vs- tech "B". The cloud makes it insanely easy to spin up two competing stacks and run benchmarks against, say Kinesis -vs- Kafka on a small subset of data and extrapolate that out to your larger dataset, using the exact compression, encryption, and most importantly, representative data and network topology that you'll use in prod.
The fifth step in designing your solution is to implement best practices that optimize quality and performance. This could include following coding standards for readability and maintainability, conducting regular code reviews with peers and stakeholders, refactoring code to improve structure and design, testing code for various types of functionality, documenting code purpose with comments and annotations, using a version control system to manage changes, utilizing a deployment pipeline to automate build, test, and release processes, and employing a monitoring system to track performance, availability, and errors.
And security. Or better yet, security first and then all the other best practices. Oftentimes having too much focus on releases, performance tracking and availability leads to a laxed security mindset. "Let's wait to apply this patch because it is impacting performance" is the type of mindset that leads to massive data breaches.
As an industry we should move away from the concept of 'best practices'. We have known-good practices. They may be beter than other practices we know of, but that is likely highly context dependent. It is also likely to vary over time based on our discovering new practices (or new ways to apply old practices) that lead to better outcomes. We should also assess how we measure what is 'good'. Our measurements may provide bias as to what is good and what is less good. How good is that measurement? Should we question how we measure it?
The sixth step in designing your solution is to evaluate the results of your solution. You need to measure and analyze your solution's outcomes, such as the customer satisfaction, the user feedback, the business value, the technical quality, and the operational efficiency. You also need to compare your solution's results with your initial requirements, expectations, and goals. By evaluating the results, you can identify the strengths, weaknesses, opportunities, and threats of your solution. You can also use the results to improve your solution's design, implementation, and delivery.
Important that you get data *before* you begin your project, as well... especially on more "soft" questions like "satisfaction with turn around time re: bugs, new feature requests", and the like.
All of this is part of gathering requirements. If we don't know how to measure if a requirement is being met then we don't understand the requirement itself. We should build test cases first to determine if requirements are being met and then design/build systems to pass those tests. It's important to note that passing some tests may require information to be gathered prior to any change, otherwise the measurements can become meaningless. For example, hitting an 'all reports run in less than 10 seconds' does not need prior information, but 'all reports run at leat 40% faster' does.
Decouple, decouple, decouple where ever possible in order to reduce interlocking dependencies making security and ops segmentation and scale performance efforts easier.
I would include SECURITY as a foundation pillar to any solution. As a solutions Architect you also need to ensure security of application and infrastructure. Not to mention with data revolution data security and governance is a critical aspect for every data solution, product and service.
Whatever solution you end up choosing, always leave plenty of room for flexibility and innovation. By adopting a flexible approach, you can adapt to unforeseen circumstances and capitalize on emerging opportunities.