Consul LB GCE

GCE load balancing based on Consul. It is a fork of @pires's project that works with network endpoint groups instead of instance groups.

Setup

Production

• Prepare VM instances to run Consul, your services and consul-lb-gce;
• Prepare URL map;
• Setup Consul;
• Run your services and register them in Consul via its API, Nomad or something else;
• Add firewall rule as described here to allow health checks requests;
• If you plan to use REUSE mode then create necessary network endpoint groups, health checks, backend services and configure URL map so that it uses created backend services;
• We use HTTP API to manage NEGs so no need for support of network endpoint groups by gcloud CLI;
• Build consul-lb-gce linux binary executable via make build-linux-amd64 (it also generates SHA256 checksum file);
• Create config.json file near the binary by copying sample.config.json and putting your values in;
• Run consul-lb-gce, if you want to have logging then use --stderrthreshold INFO option.

Development

make up runs application. Although it's difficult to test the app locally, some things can be tested: tracking Consul services, sending requests to GCE, GCE will not be able to work with your network endpoints coz it isn't aware of them.

make test runs tests.

Other commands are available via make help.

How it works

The application tracks services registered in Consul and manages corresponding network endpoint groups accordingly. Services registered under one tag in Consul have single network endpoint group. consul-lb-gce can work with tags in 3 modes: REUSE, TAG, DEFAULT. In REUSE mode, the application re-uses cloud resources managed externally (network endpoint groups, health checks, backend services are managed externally as well as URL map is configured externally). When services with tag are found for the first time, the application in modes TAG or DEFAULT tries to create and configure listed above resources itself if needed. The difference between TAG and DEFAULT modes is in a way resources properties are specified: 1) encoded into tag string; 2) listed as fields of JSON object. If list of registered with tag services is updated in Consul then consul-lb-gce updates list of endpoints in corresponding network endpoint group (attaches new ones and detaches old ones).

Example:

1. Watched tag is single, it is in TAG mode and it is consullbgce-cdn-ipaffinity-subdomain.domain.com/;
2. The app creates network endpoint group neg-consullbgce-cdn-ipaffinity-subdomain-domain-com if it doesn't exist;
3. The app creates health check hc-consullbgce-cdn-ipaffinity-subdomain-domain-com if it doesn't exist;
4. The app creates backend service bs-consullbgce-cdn-ipaffinity-subdomain-domain-com based on NEG and HC created above if doesn't exist with enabled CDN and IP affinity;
5. The app configures host rule for subdomain.domain.com with path matcher for /* which is targeted to created backend service. If path from tag has not only / (e.g.: /path) then it creates /path, /path/* path matcher and maps it to created backend service, but /* is targeted to default backend service of provided URL map.

Global/Zonal

• NEG - zonal;
• BS - global;
• HC - global.

In future we should support both global and zonal resources.

Configuration

You can see configuration example here. It has 3 main sections tags, consul, cloud.

Tags

It is a map where key is a tag, value is an object with tag related properties:

"api": {
  "mode": "REUSE",
  "networkEndpointGroup": {
    "name": "api-neg"
  }
},
"consullbgce-nocdn:noaffinity:subdomain.domain.com/": {
  "mode": "TAG",
  "healthCheck": {
    "type": "http",
    "path": "/health"
  }
},
"consullbgce-nocdn:noaffinity:test.domain.com/v1": {
  "mode": "DEFAULT",
  "networkEndpointGroup": {
    "name": "neg-name"
  },
  "backendService": {
    "name": "bs-name",
    "cdn": false,
    "affinity": "ipaffinity",
    "host": "test.domain.com",
    "path": "/v1"
  },
  "healthCheck": {
    "name": "hc-name",
    "type": "http",
    "path": "/healthcheck"
  }
}

In REUSE mode, we have to specify only name of network endpoint group.

In TAG and DEFAULT modes, health checks are specified explicitly since Consul doesn't provide information about them.

In DEFAULT mode, all properties are specified explicitly (tag will not be parsed to get them).

Tag format in TAG mode

consullbgce-<cdn|nocdn>:<ipaffinity|noaffinity>:<APPLICATION_ADDRESS>/<PATH>

Let's take the following initial values:

• We don't need CDN;
• We want to use IP affinity;
• APPLICATION_ADDRESS is api.application.com;
• Path is empty.

So we have the following tag: consullbgce-nocdn:ipaffinity:api.application.com/.

Consul

It is an object with URL to Consul agent or server:

{
  "url": "localhost:8500"
}

Cloud

It is an object with GCE properties:

{
  "project": "project-id",
  "network": "network-name", // e.g. "default"
  "zone": "region-and-zone", // e.g. "europe-west4-b"
  "urlMap": "url-map-name"
}