Fine Granular Microservices Load Balancing with Azure Service Fabric and Application Gateway

In a Microservices world, we need to apply granular load balancing techniques based on services, not based on VMS (or bare metal servers).In a microservices based architecture, you will very often host multiple microservices on the same nodes, for instance, if I have two services (Svc1 and Svc2) hosted on the same VM, I have to apply a load balancing pattern that will manage traffic to Svc1 and to Svc2 separately, so whatever happens to a service, it doesn’t affect to the others.
We have to differentiate between internal load balancing (Service-to-Service) and External load balancing (Service to external consumers):
Internal Load Balancing: Service Fabric manages Service-to-Service communication (and service resolution, load balancing and many other greater things), using Service Fabric APIS we can very easily communication to any other service that resides in the Service Fabric cluster, so Service Fabric already provides fine granular load balancing for Service-to-Service Communication.
External Load Balancing: If you want to expose Services to external consumers, you can use an OWIN listener hosted on a Service Fabric Stateless, you would typically deploy the Stateless Service in all the Nodes (specifying -1 as Number of Instances); imagine now that you have to different APIs (or Stateless services) that you want to expose to external consumers, this is where you need to apply fine granular load balancing techniques, so you can completely isolate what happens across services.

This post will cover how you can apply fine granular load balancing patterns with Azure Application Gateway and expose services hosted on Azure Service Fabric Services.

The Scenario

The previous diagrams show a scenario in which:
• We have two Service Fabric applications; application 1 is called Hail and application 2 is called resources.
• Both applications expose an api (/resources and /hail) on port 80. That APIs are implanted as Service Fabric Stateless Services and using OWIN listeners.
• In addition to the APIS, both services expose a custom probe that the Azure Application Gateway will use to verify that services are healthy (/resources/healthcheck and /hail/healthcheck)
This fine granular load balancing logic can be achieved with any advanced reverse proxy like Nginx HAProxy , but Application Gateway provides a very efficient way to achieve this, as if we use it we won’t need to manage more VMs.

Fine granular load balancing with Application Gateway

Azure Application Gateway is a layer-7 load balancer. It provides failover, performance-routing HTTP requests between different servers, whether they are on the cloud or on-premises. Application Gateway has the following application delivery features: HTTP load balancing, cookie-based session affinity, and Secure Sockets Layer (SSL) offload. For more information , go to Application Gateway overview.
You can configure Application Gateway by using PowerShell or using ARM templates. This example will show you we can use custom probes and URL request routing rules to implement a fine granular load balancing approach.
Application Gateway also provides an additional secret weapon, SSL termination. We will also use Application Gateway to terminate SSL connections on the Application Gateway.

Application Gateway Configuration

You can configure Application Gateway front-services configuration (where we are listening calls) and back-end services configuration (how the Application Gateway is going to load balance)
In this case we will :
1. Expose a https listener so External consumers can call the API using SSL
2. Configure two back-end services (using urlPathMaps) : activitiesHealthCheckRule and resourceResolutionHealthCheckRule.
3. Each backend uses listens on a specific path and uses a different probe, which allows us to have implement granular load balancing approach.
The following diagram shows the Application Gateway configuration object model.

Get the Code !

Use this ARM template and PS Script to create a customized Azure Service Fabric in a fully automatic manner. This template will create an Application Gateway that will load balance two services using SSL Termination and a fine granular load balancing method.