Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Dynatrace output #575

Merged
merged 5 commits into from
Aug 2, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ It works as a single endpoint for as many as you want `Falco` instances :
- [**Wavefront**](https://www.wavefront.com)
- [**Spyderbat**](https://www.spyderbat.com)
- [**TimescaleDB**](https://www.timescale.com/)
- [**Dynatrace**](https://www.dynatrace.com/)

### Alerting

Expand Down Expand Up @@ -657,6 +658,11 @@ openobserve:
# password: "" # use this password to authenticate to OpenObserve if the password is not empty (default: "")
# customHeaders: # Custom headers to add in POST, useful for Authentication
# key: value

dynatrace:
apitoken: "" # Dynatrace API token with the "logs.ingest" scope, more info : https://dt-url.net/8543sda
blu3r4y marked this conversation as resolved.
Show resolved Hide resolved
apiurl: "" # Dynatrace API url, use https://ENVIRONMENTID.live.dynatrace.com/api for Dynatrace SaaS and https://YOURDOMAIN/e/ENVIRONMENTID/api for Dynatrace Managed, more info : https://dt-url.net/ej43qge
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
blu3r4y marked this conversation as resolved.
Show resolved Hide resolved
```

Usage :
Expand Down Expand Up @@ -1204,6 +1210,10 @@ order is
password is not empty (default: "")
- **OPENOBSERVE_CUSTOMHEADERS** : a list of comma separated custom headers to add,
syntax is "key:value,key:value"
- **DYNATRACE_APITOKEN** : Dynatrace API token with the "logs.ingest" scope, more info : https://dt-url.net/8543sda
blu3r4y marked this conversation as resolved.
Show resolved Hide resolved
- **DYNATRACE_APIURL** : Dynatrace API url, use https://ENVIRONMENTID.live.dynatrace.com/api for Dynatrace SaaS and https://YOURDOMAIN/e/ENVIRONMENTID/api for Dynatrace Managed, more info : https://www.dynatrace.com/support/help/get-started/monitoring-environment/environment-id
- **DYNATRACE_CHECKCERT** : check if ssl certificate of the output is valid (default: `true`)
- **DYNATRACE_MINIMUMPRIORITY** : minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)

#### Slack/Rocketchat/Mattermost/Googlechat Message Formatting

Expand Down Expand Up @@ -1516,6 +1526,10 @@ time akey bkey ckey priority rule value

![google chat text example](https://github.com/falcosecurity/falcosidekick/raw/master/imgs/google_chat_example.png)

### Dynatrace

![Dynatrace example](https://github.com/falcosecurity/falcosidekick/raw/master/imgs/dyntrace.png)

## Installing Policy Report Custom Resource Definition (CRD)

Information about how to find and install the CRD for the reports can be found [here](https://github.com/kubernetes-sigs/wg-policy-prototypes/tree/master/policy-report#installing). Installation of the Policy Report Custom Resource Definition (CRD) is a prerequisite for using Policy Report output.
Expand Down
6 changes: 6 additions & 0 deletions config.go
Original file line number Diff line number Diff line change
Expand Up @@ -470,6 +470,11 @@ func getConfig() *types.Configuration {
v.SetDefault("OpenObserve.Username", "")
v.SetDefault("OpenObserve.Password", "")

v.SetDefault("Dynatrace.APIToken", "")
v.SetDefault("Dynatrace.APIUrl", "")
v.SetDefault("Dynatrace.CheckCert", true)
v.SetDefault("Dynatrace.MinimumPriority", "")

v.SetEnvKeyReplacer(strings.NewReplacer(".", "_"))
v.AutomaticEnv()
if *configFile != "" {
Expand Down Expand Up @@ -727,6 +732,7 @@ func getConfig() *types.Configuration {
c.Telegram.MinimumPriority = checkPriority(c.Telegram.MinimumPriority)
c.N8N.MinimumPriority = checkPriority(c.N8N.MinimumPriority)
c.OpenObserve.MinimumPriority = checkPriority(c.OpenObserve.MinimumPriority)
c.Dynatrace.MinimumPriority = checkPriority(c.Dynatrace.MinimumPriority)

c.Slack.MessageFormatTemplate = getMessageFormatTemplate("Slack", c.Slack.MessageFormat)
c.Rocketchat.MessageFormatTemplate = getMessageFormatTemplate("Rocketchat", c.Rocketchat.MessageFormat)
Expand Down
7 changes: 6 additions & 1 deletion config_example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -475,4 +475,9 @@ openobserve:
# username: "a" # use this username to authenticate to OpenObserve if the username is not empty (default: "")
# password: "" # use this password to authenticate to OpenObserve if the password is not empty (default: "")
# customHeaders: # Custom headers to add in POST, useful for Authentication
# key: value
# key: value

dynatrace:
apitoken: "" # Dynatrace API token with the "logs.ingest" scope, more info : https://dt-url.net/8543sda
blu3r4y marked this conversation as resolved.
Show resolved Hide resolved
apiurl: "" # Dynatrace API url, use https://ENVIRONMENTID.live.dynatrace.com/api for Dynatrace SaaS and https://YOURDOMAIN/e/ENVIRONMENTID/api for Dynatrace Managed, more info : https://dt-url.net/ej43qge
# minimumpriority: "" # minimum priority of event for using this output, order is emergency|alert|critical|error|warning|notice|informational|debug or "" (default)
blu3r4y marked this conversation as resolved.
Show resolved Hide resolved
4 changes: 4 additions & 0 deletions handlers.go
Original file line number Diff line number Diff line change
Expand Up @@ -398,4 +398,8 @@ func forwardEvent(falcopayload types.FalcoPayload) {
if config.OpenObserve.HostPort != "" && (falcopayload.Priority >= types.Priority(config.OpenObserve.MinimumPriority) || falcopayload.Rule == testRule) {
go openObserveClient.OpenObservePost(falcopayload)
}

if config.Dynatrace.APIToken != "" && config.Dynatrace.APIUrl != "" && (falcopayload.Priority >= types.Priority(config.Dynatrace.MinimumPriority) || falcopayload.Rule == testRule) {
go dynatraceClient.DynatracePost(falcopayload)
}
}
Binary file added imgs/dynatrace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
13 changes: 13 additions & 0 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ var (
telegramClient *outputs.Client
n8nClient *outputs.Client
openObserveClient *outputs.Client
dynatraceClient *outputs.Client

statsdClient, dogstatsdClient *statsd.Client
config *types.Configuration
Expand Down Expand Up @@ -717,6 +718,18 @@ func init() {
}
}

if config.Dynatrace.APIToken != "" && config.Dynatrace.APIUrl != "" {
var err error
dynatraceApiUrl := strings.TrimRight(config.Dynatrace.APIUrl, "/") + "/v2/logs/ingest"
dynatraceClient, err = outputs.NewClient("Dynatrace", dynatraceApiUrl, false, config.Dynatrace.CheckCert, config, stats, promStats, statsdClient, dogstatsdClient)
if err != nil {
config.Dynatrace.APIToken = ""
config.Dynatrace.APIUrl = ""
} else {
outputs.EnabledOutputs = append(outputs.EnabledOutputs, "Dynatrace")
}
}

log.Printf("[INFO] : Falco Sidekick version: %s\n", GetVersionInfo().GitVersion)
log.Printf("[INFO] : Enabled Outputs : %s\n", outputs.EnabledOutputs)

Expand Down
126 changes: 126 additions & 0 deletions outputs/dynatrace.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
package outputs

import (
"log"
"regexp"
"time"

"github.com/falcosecurity/falcosidekick/types"
)

type dtPayload struct {
Payload []dtLogMessage `json:"payload"`
}

type dtLogMessage struct {
Timestamp string `json:"timestamp"`
EventId string `json:"event.id,omitempty"`
EventName string `json:"event.name,omitempty"`
EventProvider string `json:"event.provider,omitempty"`
Severity string `json:"severity,omitempty"`
HostName string `json:"host.name,omitempty"`
LogSource string `json:"log.source,omitempty"`
Content dtLogContent `json:"content"`
MitreTechnique string `json:"mitre.technique,omitempty"`
MitreTactic string `json:"mitre.tactic,omitempty"`
ContainerId string `json:"container.id,omitempty"`
ContainerName string `json:"container.name,omitempty"`
ContainerImageName string `json:"container.image.name,omitempty"`
K8sNamespaceName string `json:"k8s.namespace.name,omitempty"`
K8sPodName string `json:"k8s.pod.name,omitempty"`
K8sPodId string `json:"k8s.pod.id,omitempty"`
ProcessExecutableName string `json:"process.executable.name,omitempty"`
SpanId string `json:"span.id,omitempty"`
}

type dtLogContent struct {
Output string `json:"output"`
OutputFields map[string]interface{} `json:"output_fields"`
Tags []string `json:"tags,omitempty"`
}

const DynatraceContentType = "application/json; charset=utf-8"
const DynatraceEventProvider = "Falco"

// match MITRE techniques, e.g. "T1070", and sub-techniques, e.g. "T1055.008"
var MitreTechniqueRegEx = regexp.MustCompile(`T\d+\.?\d*`)

// match MITRE tactics, e.g. "mitre_execution"
var MitreTacticRegEx = regexp.MustCompile(`mitre_\w+`)

func newDynatracePayload(falcopayload types.FalcoPayload) dtPayload {
message := dtLogMessage{
Timestamp: falcopayload.Time.Format(time.RFC3339),
EventId: falcopayload.UUID,
EventName: falcopayload.Rule,
EventProvider: DynatraceEventProvider,
Severity: falcopayload.Priority.String(),
HostName: falcopayload.Hostname,
LogSource: falcopayload.Source,
Content: dtLogContent{
Output: falcopayload.Output,
OutputFields: falcopayload.OutputFields,
Tags: falcopayload.Tags,
},
}

// possibly map a few fields to semantic attributes
for fcKey, val := range falcopayload.OutputFields {
switch fcKey {
case "container.id":
message.ContainerId = val.(string)
case "container.name":
message.ContainerName = val.(string)
case "container.image.name":
message.ContainerImageName = val.(string)
case "k8s.namespace.name", "ka.target.namespace":
message.K8sNamespaceName = val.(string)
case "k8s.pod.name":
message.K8sPodName = val.(string)
case "k8s.pod.id":
message.K8sPodId = val.(string)
case "proc.name":
message.ProcessExecutableName = val.(string)
case "span.id":
message.SpanId = val.(string)
default:
continue
}
}

// map tags to MITRE technique and tactic
for _, fcTag := range falcopayload.Tags {
if MitreTechniqueRegEx.MatchString(fcTag) {
message.MitreTechnique = fcTag
} else if MitreTacticRegEx.MatchString(fcTag) {
message.MitreTactic = fcTag
}
}

return dtPayload{Payload: []dtLogMessage{message}}
}

func (c *Client) DynatracePost(falcopayload types.FalcoPayload) {
c.Stats.Dynatrace.Add(Total, 1)

c.ContentType = DynatraceContentType

if c.Config.Dynatrace.APIToken != "" {
blu3r4y marked this conversation as resolved.
Show resolved Hide resolved
c.httpClientLock.Lock()
defer c.httpClientLock.Unlock()
c.AddHeader("Authorization", "Api-Token "+c.Config.Dynatrace.APIToken)
}

err := c.Post(newDynatracePayload(falcopayload).Payload)
if err != nil {
go c.CountMetric(Outputs, 1, []string{"output:dynatrace", "status:error"})
c.Stats.Dynatrace.Add(Error, 1)
c.PromStats.Outputs.With(map[string]string{"destination": "dynatrace", "status": Error}).Inc()
log.Printf("[ERROR] : Dynatrace - %v\n", err)
return
}

go c.CountMetric(Outputs, 1, []string{"output:dynatrace", "status:ok"})
c.Stats.Dynatrace.Add(OK, 1)
c.PromStats.Outputs.With(map[string]string{"destination": "dynatrace", "status": OK}).Inc()
}
1 change: 1 addition & 0 deletions stats.go
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ func getInitStats() *types.Statistics {
Telegram: getOutputNewMap("telegram"),
N8N: getOutputNewMap("n8n"),
OpenObserve: getOutputNewMap("openobserve"),
Dynatrace: getOutputNewMap("dynatrace"),
}
stats.Falco.Add(outputs.Emergency, 0)
stats.Falco.Add(outputs.Alert, 0)
Expand Down
9 changes: 9 additions & 0 deletions types/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ type Configuration struct {
Telegram TelegramConfig
N8N N8NConfig
OpenObserve OpenObserveConfig
Dynatrace DynatraceOutputConfig
}

// MutualTLSClient represents parameters for mutual TLS as client
Expand Down Expand Up @@ -708,6 +709,13 @@ type N8NConfig struct {
CheckCert bool
}

type DynatraceOutputConfig struct {
APIToken string
APIUrl string
MinimumPriority string
CheckCert bool
}

// OpenObserveConfig represents config parameters for OpenObserve
type OpenObserveConfig struct {
HostPort string
Expand Down Expand Up @@ -785,6 +793,7 @@ type Statistics struct {
Telegram *expvar.Map
N8N *expvar.Map
OpenObserve *expvar.Map
Dynatrace *expvar.Map
}

// PromStatistics is a struct to store prometheus metrics
Expand Down