Load Balancer & Service RegistryAPI Gateway & BFF APIsSynchronous CommunicationAPI Style for microserviceResiliency PatternOpenAPI and API CatalogsAsynchronous CommunicationTransaction Manager for logical distributed transactionsEventual Consistency and Event-driven ArchitectureDeployment using virtual machinesDeployment using containersDeployment using cloudSecurityCentral LoggingCentral Monitoring
Load Balancer & Service Registry
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2F231bed55-0136-46c1-9f07-0ad7fde9822c%2FUntitled.png%3Fid%3Dbcfd41b2-ae06-4976-9ebd-d7d060bfb143%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3D_U3mHovpuQ_qJihMYJX5DA3YLRRHn2rYUJT7Jk48Qac?table=block&id=bcfd41b2-ae06-4976-9ebd-d7d060bfb143&cache=v2)
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2F2954f0ae-e400-4168-8cd2-9588aa3f69d3%2FUntitled.png%3Fid%3D5650f7ba-5cd8-4401-a6b3-9b1022b8a852%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3DtXAhAYXVW_RxQT-Ld2Ip3v5M8oiR8cWrYo17Az6WfWw?table=block&id=5650f7ba-5cd8-4401-a6b3-9b1022b8a852&cache=v2)
Â
Load balancers
- Support scalability by routing incoming traffic to a stateless service instance
- Upstream client is unaware of the load balancer
- Most promote availability and therefore resiliency using health probes and checks
- Health probes can work with custom health check endpoints
- Either provide or support service registry systems
- On-premise physical or software options, and managed options in the cloud
Â
Loading balancer routing method:
- Round Robin: Simplest type of routing, can result in uneven traffic
- Least Connections: Routes based on number of client connections to server, Useful for chat or streaming applications
- Least Response Time: Routes based on how quickly servers respond
- IP Hash: Routes client to server based on IP; Useful for stateful sessions
Â
Service registry
- Record and provide microservice instance locations
- Microservices register on startup and deregister on shutdown
- Used by traffic routing mechanisms like load balancers
- Develop your own or DNS on cloud PAAS solutions and container orchestration engines
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2Fa5158c19-fd47-47e8-a21e-8f3a4ea32e13%2FUntitled.png%3Fid%3Daa76c67a-cf00-4c7a-9cb9-075f852cd3a5%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3D29tZeChJHofB0J9iATmhJloywC-pLufDKV3FB1POly8?table=block&id=aa76c67a-cf00-4c7a-9cb9-075f852cd3a5&cache=v2)
k8s: know the service name you want to talk to, it takes care of the routing traffic to the container.
API Gateway & BFF APIs
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2F26a1a0e9-9e66-4b07-ac90-88d44615e093%2FUntitled.png%3Fid%3D1ec95896-faa5-47e8-ab92-eb689edb47a1%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3DRWhW3QSvkz5j2Dsm9mB43BEOWZ9MndoJ3qSqIOI4CDM?table=block&id=1ec95896-faa5-47e8-ab92-eb689edb47a1&cache=v2)
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2F78993149-e220-4e96-9fd6-19feace7a676%2FUntitled.png%3Fid%3D8d642d20-340c-4713-ac10-5d5b3ac6a454%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3DLlEw5LQMvJ1ZSdDZYsl8OqHHNDFX19uZ90q-kqJd9n0?table=block&id=8d642d20-340c-4713-ac10-5d5b3ac6a454&cache=v2)
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2Ff0c8647a-1ff4-4a53-9dad-1f8f336d41e2%2FUntitled.png%3Fid%3D0e99dd51-f520-4c0f-9b7b-20fc5b62e545%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3DZgL29UGcjr_iI45uO4qrxk0X4ResuP2ql3xDkrIqgcE?table=block&id=0e99dd51-f520-4c0f-9b7b-20fc5b62e545&cache=v2)
Â
API gateway
- Central entry point, exposing microservices using one interface
- Can handle security: Authentication and authorization
- Can also provide call rate limiting, monitoring, caching, and logging
- Can provide load balancing and service registry functionality
Â
Backend for frontend API (a type of API gateway!)
- Central entry point, exposing microservices using one interface for a specific client
- Can handle security: Authentication and authorization
- You would use in addition to load balancers and service registry
- Custom logic to consolidate and format data from downstream microservices
Â
Both components decouple frontend applications from microservices
Avoid a monolith API gateway or monolith BFF API
Synchronous Communication
Challenges: a microservice talks to another.
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2Fcef209c2-721f-4a58-9545-7448671c8184%2FUntitled.png%3Fid%3D93f7f451-5c76-4cab-bbd0-2e283334bfac%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3DmPcjytEckcJwYmB-kVHf84y_yBu7pP-4NTKB0tDU288?table=block&id=93f7f451-5c76-4cab-bbd0-2e283334bfac&cache=v2)
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2Fddb62d03-3662-41c9-af07-605f60bbaa00%2FUntitled.png%3Fid%3Dae29b137-e5b4-4b22-abdf-217805c477d8%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3DO7aw2LxDp48vakyPStLfAyjFfXFBP-h6dd2mAdkxGMQ?table=block&id=ae29b137-e5b4-4b22-abdf-217805c477d8&cache=v2)
Â
Request response communication
- Client to service, and service to service and service to external
- Your upstream software makes a request to a downstream software
- Upstream software waits for a response from the downstream software
Â
HTTP is the most used open communication protocol for this
- Huge compatibility across hardware and software thanks to its use on the Internet
- If you use the Internet, you are already using HTTP
Â
Microservice synchronous communication challenges
- Both parties must be available, and then you wait for a response
- Performance subject to network quality
- Client must know the location of the service (host/port)
- Reducing temporal coupling and avoiding distributed transactions
API Style for microservice
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2Ffe0872bd-bb1c-4910-a5d0-32cdc2eceaa3%2FUntitled.png%3Fid%3D48a878c7-953f-49dc-8c31-b6570ee8e216%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3DgjBt5pz0fsp6dZAV9V48yeTyeSX1V2OIr0zYNBPZRnA?table=block&id=48a878c7-953f-49dc-8c31-b6570ee8e216&cache=v2)
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2Fa1c68cfe-1e2c-4376-9feb-952306bb78cb%2FUntitled.png%3Fid%3Dae3f458b-ff36-4d94-ad3e-da5e3a721ea9%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3DUTRipE3Zb4W2PSvzn6j8xXeGYyi7SCy5WJvPoYMeuSk?table=block&id=ae3f458b-ff36-4d94-ad3e-da5e3a721ea9&cache=v2)
Â
REST-full microservices are popular
- Consumer and developer-friendly by providing a predictable style
- REST principles are in line with microservices
- Decouples client and service
- Functionality to honor contracts (versioned endpoints for backwards compatibility)
- Encourage stateless services
- Ideal contract and data format for caching strategies
Â
Other styles are available and should be considered
- RPC, GRPC, SOAP, GraphQL and many more
Resiliency Pattern
- Timeout
- Circuit breaker: after a number of trials, break the connection and return error
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2Fb8694097-62a0-403e-ba30-83e2873b2d8d%2FUntitled.png%3Fid%3Db63c8665-cb3e-42fa-a803-38e9d83a958e%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3DK12sZ12EON9guU65fKSv5LfqpxqGH8FACmAbK7Dzn48?table=block&id=b63c8665-cb3e-42fa-a803-38e9d83a958e&cache=v2)
- Retry
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2F81355793-1264-4d76-adc1-ba5cee795a61%2FUntitled.png%3Fid%3D6a526129-0d57-4fb6-addc-0dd80a316f3b%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3D1yv2TRDGLjZ09CFoP7ax2dAYaJzZ-ahCjDS7aFsTEaU?table=block&id=6a526129-0d57-4fb6-addc-0dd80a316f3b&cache=v2)
- Cache
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2Fb34efebd-1ee0-40cc-abcf-203456b4222f%2FUntitled.png%3Fid%3D1ef39332-8b90-40f1-ac43-3f9c8f296962%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3DpNRcHz4mnABmt3XcSivcqLRn6OJpV2ty5KTj_hdpvg8?table=block&id=1ef39332-8b90-40f1-ac43-3f9c8f296962&cache=v2)
Â
Timeout pattern
- Configure short timeouts on calling code to avoid waiting forever for a response
Â
Circuit Breaker pattern: Use to fail fast and return an error instantly
- Calling code stops calling downstream services if too many failures occur
- Calling of downstream service resumes after a specified threshold
Â
Retry pattern
- Ideal for transient faults: Momentary loss of connectivity or service
- Limit the number of retries and use with the circuit breaker pattern
Â
Caching Strategy
- Separate cache database serving microservice instances
- Serve matching data from a cache that hasn't expired instead of making a sync call
Â
Asynchronous communication using message brokers
OpenAPI and API Catalogs
Swagger
Â
OpenAPI
- Document your microservice contracts (endpoints)
- Document using machine readable format (JSON)
- Use document to generate interactive websites
- Create an API catalogue
- Use machine readable documentation with many other tools
Â
Useful for communicating contracts, data, and functionality
- Development teams working on microservices concurrently
- Business teams needing to see the data
- Scoping and designing new parts of your architecture
Asynchronous Communication
Message Broker
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2F5b9e92a1-4e4f-4fae-8214-8e60c942b1ee%2FUntitled.png%3Fid%3D2962c873-b7b2-4d4c-8f3c-9197f8fbde05%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3DRIchk6axKITlunw8_pWlHNXqn9OCQLS4Lx-92OKaBcY?table=block&id=2962c873-b7b2-4d4c-8f3c-9197f8fbde05&cache=v2)
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2Fc7c87af6-f3b6-4a03-a913-9567237404b1%2FUntitled.png%3Fid%3Dba04ff1e-6ae7-47bc-96ee-8afa582004cf%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3DhWxUbA32HkEyHtmSX7DZb0y6vo8kK0UBvRhmAIi1_sw?table=block&id=ba04ff1e-6ae7-47bc-96ee-8afa582004cf&cache=v2)
Asynchronous communication: Fire and forget and return to the caller
- Decouples client and service and reduces sync com and temporal coupling
Â
Messaging system / Message Broker / Service Bus / Event Bus
- Sender also can be called client and publisher
- Many queues with many listeners to send and receive data
- Data sent referred to has as a message (command) or event
- Many queueing patterns
- Message brokers provide resiliency in your architecture
Â
Asynchronous communication and messaging systems required for
- Event-driven architecture and eventual-consistency
- Both concepts are helpful for microservices architecture
Managing a logical distributed transaction (next section!)
Transaction Manager for logical distributed transactions
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2F1f345c99-3bc2-4862-9564-548593fc8d6c%2FUntitled.png%3Fid%3Dbf553e87-1afb-4bb9-ace4-fa95e9d5a53e%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3DZxFMOaEjOjEIOJHtVF44mJxjvq-WQQNnrEWiBLd7hkY?table=block&id=bf553e87-1afb-4bb9-ace4-fa95e9d5a53e&cache=v2)
Avoiding physical distributed transactions
- We can avoid physical distributed transactions in terms of waiting
- Avoided using asynchronous communication and messaging systems
Â
Managing the logical distributed transactions
- We need to manage the state of a logical data transaction (saga pattern)
- Centralize the transaction state: Somewhere, we can see that state of each part
- Have failure compensation: Functionality to undo a partially complete transaction
- Transaction managers and the saga pattern also allow you to cancel a transaction
Eventual Consistency and Event-driven Architecture
Instead of calling 2 microservices, whenever a product updates, it will publish the product info into a message broker. Our inventory microservice then subscribes to that information and keeps a local copy of all the product names in its local database.
Advantage of this approach: performance, decoupled.
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2F4c496ef1-8447-43f0-8ead-8ad15c65fa6d%2FUntitled.png%3Fid%3D86c7e6bc-38ab-4a01-8ea4-f7c88e9f7d59%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3D2nzhLEJDUSEh2QZ9KAJW1itRniZdC4dH-klkSiLUGmw?table=block&id=86c7e6bc-38ab-4a01-8ea4-f7c88e9f7d59&cache=v2)
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2F1730209c-c6d8-443a-8e47-747b34a9ffeb%2FUntitled.png%3Fid%3D7f7cd9a0-b145-42a3-b21a-53b14a9b2c7a%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3Dy0I-AFOh9mzZBnL8VVq8a11BX6305vyVWtWsR8fMIGU?table=block&id=7f7cd9a0-b145-42a3-b21a-53b14a9b2c7a&cache=v2)
Duplicating data for performance
- A microservice has a local copy of data belonging to another service
- Local copy avoids having to make a sync call to retrieve the data from the source
Â
Eventual Consistency
- Uses publisher and subscriber model: Where client and service completely decoupled
- Microservice that owns the data publishes data changes as events (messages)
- Subscribers interested in the data subscribe to the events (optionally store them locally)
- This duplicated data across our microservices is eventual consistent
- Many eventual consistency patterns that balance availability and consistency
Â
Event-Driven Architecture
- Complete decoupling of publisher and subscriber: Publish data changes as events
- Create new subscribers, i.e., new use cases for your data without changing publishers
Deployment using virtual machines
Why not physical machines?
- Waste of hardware resources (CPU, Memory, and storage)
- Multiple services on one machine violates the autonomous principle
- Also violates the resiliency principle (scalability and availability)
Â
Virtual machines
- Better resource utilization (CPU, Memory, and storage)
- Templating virtual machine images and snapshot backups
- Each service instance on a small VM promotes design principles
- Support for AC (Infrastructure as code)
- Special VM operating systems to manage VM's
- Multiple services still venerable to host failures
- Each VM still has an unnecessarily large footprint and are slow to startup
Deployment using containers
Containers are a type of virtualization
- Like VMs, isolate services from each other
- Single service per container (single deployable unit containing all dependencies)
- Much more resource-efficient and startup faster than VMs
- Most popular way of deploying microservices
Simplify the running of complex microservices architecture and dependencies
- Multiple container images containing microservices architecture run using one file
- Recreate full microservices architecture environments (Spin up production on dev)
Â
Container orchestration engines
- Managing multiple containers on multiple servers: Load balancing and discovery
- Run containers: Ensure numbers and availability across these servers
- Containers without an orchestration engine are not ideal!
- Well supported on cloud platforms and on-premise
Deployment using cloud
Why not on-premise?
- Only if you already have the budget, teams, and location/equipment
- Cloud provides managed and serverless services
Â
IAAS: Deploy to virtual machines/containers within the cloud
- Infrastructure as a service: When you still need to manage the OS/Runtime
- You still must manage and update the OS and runtime
Â
PAAS: Deploy microservice to cloud-specific Blackbox
- Managing applications/services and data only
- No control over the runtime/OS running the application
Â
FAAS: Serverless functions: Deploy code, i.e., functions instead of applications
- Cloud platform handles the running, concurrency, and resources
- Cloud platform may limit concurrency, duration of running, and tech choices
Security
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2F41d76c4d-be24-434b-8061-c1702d0ff25c%2FUntitled.png%3Fid%3Dd924c4de-2d16-423b-9a0c-434d62a0163a%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3DHeFoPztgunxj5h1IfqpvDHrViqMoWNIjFZ9INF4xJ24?table=block&id=d924c4de-2d16-423b-9a0c-434d62a0163a&cache=v2)
![notion image](https://www.notion.so/image/https%3A%2F%2Ffile.notion.so%2Ff%2Ff%2F0923a366-bb69-4245-954f-30ad6cbb7483%2F85b98068-bc06-47ae-b567-220e8fa41f69%2FUntitled.png%3Fid%3D99d7294e-a31c-4251-99f0-3b164fd401c8%26table%3Dblock%26spaceId%3D0923a366-bb69-4245-954f-30ad6cbb7483%26expirationTimestamp%3D1712232000000%26signature%3D-zvz12g0sDcXLXbKfG916nkITC54Bf9J3zVL7FDkMyE?table=block&id=99d7294e-a31c-4251-99f0-3b164fd401c8&cache=v2)
- Use HTTPS everywhere!
- Network security: private and public network
- Call rate-limiting at API Gateway or BFF API level
- Use a reputable Identity provider/service
- OAuth2 and OpenID Connect standards
- Authentication: Identify the user (Identity token)
- Authorization: Can the client app access the microservice? (access token)
- Authorization: What can the user do? (access token)
- Pass these tokens between clients and services: all services validate tokens
- Standards: OAuth2, OpenID connect, and Token exchange
- Identity providers: Azure AD, Okta, Pingldentity, IdentityServer, and many more!
- Secure message broker using HTTPS, authentication, and message validation
Central Logging
Central logging Solutions
- Reputable product that supports scalability and availability
- Centralized portal for querying, filtering, and managing log data
- Managing data: Archiving and backing up log data
- Security: secure connections (HTTPS), authentication, and authorization
- Examples: Elastic/Kibana, Splunk, and Graphite
Â
Logging library
- Structured logging: Support for custom fields, logging levels, and correlation IDs
- Lightweight log data format like JSON
Â
Ensuring logging data and logging database are compliant
- Data compliance requlations
- Communicate data policy
Â
Cross-cutting concern for Architects to standardize
Central Monitoring
Centralized monitoring tool
- Real-time metrics across servers
- Visualize and aggregate data
- Vast alerting options: SMS, email, visual and custom calls!
- Examples: Nagios, PRTG, and New Relic
- Archiving and backup of data
Â
Monitoring library
- Custom or off-shelf library for consistent stats across architecture
- Preconfigured machine, VM, or container templates
Â
Cross-cutting concern for Architects to standardize
Â