LogDeep is an open source deeplearning-based log analysis toolkit for automated anomaly detection.
Note: This repo does not include log parsing,if you need to use it, please check logparser
-
Modular Design
-
Support multi log event features out of box
-
State of the art(Including resluts from deeplog,loganomaly,robustlog...)
Model | Paper reference |
---|---|
DeepLog | [CCS'17] DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning |
LogAnomaly | [IJCAI'19] LogAnomaly: UnsupervisedDetectionof SequentialandQuantitativeAnomaliesinUnstructuredLogs |
RobustLog | [FSE'19] RobustLog-BasedAnomalyDetectiononUnstableLogData |
- python>=3.6
- pytorch >= 1.1.0
git clone https://github.com/donglee-afar/logdeep.git
cd logdeep
Example of building your own log dataset
SAMPLING_EXAMPLE.md
Train & Test DeepLog example
cd demo
# Train
python deeplog.py train
# Test
python deeplog.py test
The output results, key parameters and train logs will be saved under result/
path
Here is an example of the key parameters of the loganomaly model which in demo/loganomaly.py
Try to modify these parameters to build a new model!
# Smaple
options['sample'] = "sliding_window"
options['window_size'] = 10
# Features
options['sequentials'] = True
options['quantitatives'] = True
options['semantics'] = False
Model = loganomaly(input_size=options['input_size'],
hidden_size=options['hidden_size'],
num_layers=options['num_layers'],
num_keys=options['num_classes'])
HDFS | ||||
---|---|---|---|---|
Model | feature | Precision | Recall | F1 |
DeepLog(unsupervised) | seq | 0.9583 | 0.9330 | 0.9454 |
LogAnomaly(unsupervised) | seq+quan | 0.9690 | 0.9825 | 0.9757 |
RobustLog(supervised) | semantic | 0.9216 | 0.9586 | 0.9397 |