⚠️ This repo contains the source for a component of the Brigade v1.x ecosystem. Brigade v1.x reached end-of-life on June 1, 2022 and as a result, this component is no longer maintained.Brigade v2 users can utilize the Brigade CloudEvents Gateway to support the use cases previously supported by this gateway.
Brigade gateway that responds to Azure EventGrid events.
First you need to clone this repo:
git clone https://github.com/radu-matei/brigade-eventgrid-gateway
and navigate to the root directory.
EventGrid needs an HTTPS endpoint to deliver the event, so you need to have TLS ingress for your cluster (use something like kube-lego or cert-manager), and this chart assumes you have an nginx
ingress controller deployed on your cluster. Once you have an HTTPS domain or subdomain, pass it to the ingress.host
property below.
Now install the Helm chart:
helm install -n brigade-eventgrid-gateway ./charts/brigade-eventgrid-gateway --set ingress.host=<your-HTTPS-endpoint>
You can also specify the host in
charts/brigade-eventgrid-gateway/values.yaml
By default, the chart assumes you have a cluster with RBAC enabled. If you don't, either modify the
rbac.enabled
value invalues.yaml
or pass--set rbac.enabled=false
to thehelm install
command.
If you don't have a domain or an ingress controller configured, you can change the ingress annotations in the ingress template file - but keep in mind that EventGrid will not pass events to a non-HTTPS endpoint.
At this point, you should be able to navigate to https://<your-endpoint>/healthz
and receive "message": "ok"
and you can start sending events to this gateway.
You can follow the instructions from the official Brigade documentation to create a new project - the gateway will use a token in order to make sure that if unauthorized people send events to your gateway, those will not become Brigade builds - the token is passed in the URL and it is checked whenever a new event is received, before creating a new Brigade build, and can be any string. In your project's values.yaml
file, add an eventGridToken
token:
project: "<your-project>"
repository: "<your-repo>"
cloneURL: "<your-clone-url>"
secrets:
eventGridToken: "<your-token>"
allowPrivilegedJobs: "false"
Then create the project:
helm install -n eventgrid-project brigade/brigade-project -f values.yaml
When creating an EventGrid subscription you will need both the project ID and the token, as they will be part of the event endpoint URL.
Azure EventGrid supports two JSON event schemas:
- the CloudEvents JSON schema, which is an open standard for describing event data in a consistent way (which is in preview at the moment of writing this document)
- the default Azure EventGrid schema
For the purpose of this tutorial we will use a storage account as the source of events, but the same concepts can be applied to any event source Azure EventGrid supports:
az storage account create \
--name <storage-account-name> \
--location northeurope \
--resource-group <resource-group-name> \
--sku Standard_LRS \
--kind BlobStorage \
--access-tier Hot
Please note that currently, Azure Event Grid has preview support for CloudEvents JSON format input and output in West Central US, Central US, and North Europe.
For more up-to-date information about regions and CloudEvents in Azure EventGrid, check out the documentation
In order to create an event subscription, we need to pass the id to the resource that generates the events - in this case, we need to pass the id to the storage account we just created:
storageid=$(az storage account show --name <storage-account-name> --resource-group <resource-group-name> --query id --output tsv)
Please that in this way you can get the ID of any Azure resource that supports generating events and sending them to EventGrid. For an up-to-date list of Azure resources that support EventGrid, check out the documentation
We want to generate events from Azure resources using the CloudEvents schema and handle them using the gateway we just deployed. Since the feature is currently in preview, we need to add an extension for the az
command line:
az extension add --name eventgrid
Then, we create the event subscription:
az eventgrid event-subscription create \
--source-resource-id $storageid \
--name brigade-cloudevents \
--endpoint https://<your-endpoint>/cloudevents/v0.1/<brigade-project-id>/<your-token> \
--event-delivery-schema cloudeventv01schema
You can get the Brigade Project ID using the
brig
tool:brig project list
Note that the path for CloudEvents is /cloudevents/v0.1/<brigade-project-id>/<your-token>
, and an example of the full URL would be:
https://$YOURDOMAIN/cloudevents/v0.1/brigade-9e0af40182d1ab201542ebfb7d795d189ad0ec0b73512190e93935/you-should-change-this-to-be-your-own
This is a sample event that follows the CloudEvents schema looks like:
{
"cloudEventsVersion" : "0.1",
"eventType" : "Microsoft.Storage.BlobCreated",
"eventTypeVersion" : "",
"source" : "/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.Storage/storageAccounts/{storage-account}#blobServices/default/containers/{storage-container}/blobs/{new-file}",
"eventID" : "173d9985-401e-0075-2497-de268c06ff25",
"eventTime" : "2018-04-28T02:18:47.1281675Z",
"data" : {
"api": "PutBlockList",
"clientRequestId": "6d79dbfb-0e37-4fc4-981f-442c9ca65760",
"requestId": "831e1650-001e-001b-66ab-eeb76e000000",
"eTag": "0x8D4BCC2E4835CD0",
"contentType": "application/octet-stream",
"contentLength": 524288,
"blobType": "BlockBlob",
"url": "https://oc2d2817345i60006.blob.core.windows.net/oc2d2817345i200097container/oc2d2817345i20002296blob",
"sequencer": "00000000000004420000000000028963",
"storageDiagnostics": {
"batchId": "b68529f3-68cd-4744-baa4-3c0498ec19f0"
}
}
}
az eventgrid event-subscription create \
--source-resource-id $storageid \
--name brigade-eventgrid \
--endpoint https://<your-endpoint>/eventgrid/<brigade-project-id>/<your-token>
Note that the path for the default EventGrid schema is /eventgrid/<brigade-project-id>/<your-token>
This is a sample event that follows the Azure EventGrid default schema:
[{
"topic": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/myrg/providers/Microsoft.Storage/storageAccounts/myblobstorageaccount",
"subject": "/blobServices/default/containers/testcontainer/blobs/testfile.txt",
"eventType": "Microsoft.Storage.BlobCreated",
"eventTime": "2017-08-16T20:33:51.0595757Z",
"id": "4d96b1d4-0001-00b3-58ce-16568c064fab",
"data": {
"api": "PutBlockList",
"clientRequestId": "d65ca2e2-a168-4155-b7a4-2c925c18902f",
"requestId": "4d96b1d4-0001-00b3-58ce-16568c000000",
"eTag": "0x8D4E4E61AE038AD",
"contentType": "text/plain",
"contentLength": 0,
"blobType": "BlockBlob",
"url": "https://myblobstorageaccount.blob.core.windows.net/testcontainer/testblob1.txt",
"sequencer": "00000000000000EB0000000000046199",
"storageDiagnostics": {
"batchId": "dffea416-b46e-4613-ac19-0371c0c5e352"
}
},
"dataVersion": "",
"metadataVersion": "1"
}]
You can get the Brigade Project ID using the
brig
tool:brig project list
An example of the full URL would be:
https://$YOURDOMAIN/eventgrid/brigade-9e0af40182d1ab201542ebfb7d795d189ad0ec0b73512190e93935/you-should-change-this-to-be-your-own
In both cases, a validation request will be sent to the endpoint, which this gateway handles - after this, the endpoint will receive events according to the subscription.
Following the example so far, blob storage generates two events: Microsoft.Storage.BlobCreated
and Microsoft.Storage.BlobDeleted
that we can handle in our brigade.js
file:
const { events } = require('brigadier')
events.on("Microsoft.Storage.BlobDeleted", (e, p) => {
console.log(e);
})
events.on("Microsoft.Storage.BlobCreated", (e, p) => {
console.log(e);
})
At this point, whenever events are fired, our simple handlers will log them to the console:
$ brig build logs 01cegwv9t48kva8wh093pw0hbn
==========[ brigade-worker-01cegwv9t48kva8wh093pw0hbn ]==========
prestart: empty script found. Falling back to VCS script
prestart: src/brigade.js written[brigade] brigade-worker version: 0.14.0
[brigade:k8s] Creating PVC named brigade-worker-01cegwv9t48kva8wh093pw0hbn
{ buildID: '01cegwv9t48kva8wh093pw0hbn',
workerID: 'brigade-worker-01cegwv9t48kva8wh093pw0hbn',
type: 'Microsoft.Storage.BlobDeleted',
provider: 'cloudevents',
revision: { commit: '', ref: 'master' },
logLevel: 1,
payload: '{"eventType":"Microsoft.Storage.BlobDeleted","eventTypeVersion":"","cloudEventsVersion":"0.1","source":"/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Storage/storageAccounts/<storage-account>#blobServices/default/containers/<path-to-file-in-blob>","eventID":"<event-id>","eventTime":"2018-05-27T13:33:18.1443969Z","contentType":"","extensions":null,"data":{"api":"DeleteBlob","blobType":"BlockBlob","contentLength":5698,"contentType":"application/octet-stream","eTag":"<e-tag>","requestId":"<request-id>","sequencer":"<sequencer>","storageDiagnostics":{"batchId":"<batch-id>"},"url":"https://<storage-account>.blob.core.windows.net/<path-to-file-in-blob>"}}' }
[brigade:app] after: default event handler fired
[brigade:app] beforeExit(2): destroying storage
[brigade:k8s] Destroying PVC named brigade-worker-01cegwv9t48kva8wh093pw0hbn
Prerequisites:
- the Go toolchain
dep
make
(optional)
To build from source:
-
dep ensure
-
make build
orgo build
to build the binary for your OS -
if running locally, you should provide an environment variable for the Kubernetes configuration file:
- on Linux (including Windows Subsystem for Linux) and macOS:
export KUBECONFIG=<path-to-config>
- on Windows:
$env:KUBECONFIG="<path-to-config>"
- on Linux (including Windows Subsystem for Linux) and macOS:
-
starting the binary you should see the initial Gin output:
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
- using env: export GIN_MODE=release
- using code: gin.SetMode(gin.ReleaseMode)
[GIN-debug] GET /healthz --> main.healthz (2 handlers)
[GIN-debug] POST /eventgrid/:project --> main.azFn (3 handlers)
[GIN-debug] POST /eventgrid/:project/:token --> main.azFn (3 handlers)
[GIN-debug] POST /cloudevents/v0.1/:project/:token --> main.ceFn (3 handlers)
[GIN-debug] Environment variable PORT is undefined. Using port :8080 by default
[GIN-debug] Listening and serving HTTP on :8080
- at this point, your server should be able to start accepting incoming requests to
localhost:8080
- you can test the server locally, using Postman (POST requests with your desired JSON payload - see the
testdata
folders used for testing) - please note that running locally with a Kubernetes config file set is equivalent to running privileged inside the cluster, and any Brigade builds created will get executed!
This Brigade project accepts contributions via GitHub pull requests. This document outlines the process to help get your contribution accepted.
A DCO sign-off is required for contributions to repos in the brigadecore org. See the documentation in Brigade's Contributing guide for how this is done.