This repository is a fork from PekeDevil Gotham Testbed (X. Sáez-de-Cámara, J. L. Flores, C. Arellano, A. Urbieta and U. Zurutuza, "Gotham Testbed: A Reproducible IoT Testbed for Security Experiments and Dataset Generation," in IEEE Transactions on Dependable and Secure Computing, doi: 10.1109/TDSC.2023.3247166)
It contains improved and extended source code of the testbed called Gotham to generate customizable, legitimate and malicious IoT network traffic. Details about this traffic generator can be found in the paper GothX: a generator of customizable, legitimate and malicious IoT network traffic
If you use or build upon this testbed, please consider citing the article.
Manuel Poisson, Kensuke Fukuda, Rodrigo Carnier. GothX: a generator of customizable, legitimate and malicious IoT network traffic. CSET - 17th Cyber Security Experimentation and Test Workshop, Aug 2024, Philadelphia, United States. pp.1-9, ⟨10.1145/3675741.3675753⟩. ⟨hal-04629350v2⟩
- GothX: a generator of customizable, legitimate and malicious IoT network traffic
- Download the datasets generated with GothX
- Installation
- GothX usage
- Contact
You can download already generated datasets here. More details about labels, settings and command line used during the traffic generation are available in the dataset_details directory
Tested on Ubuntu 20.04.4 LTS, 22.04 LTS and fedora 35
Read the install.sh script to see commands for installation.
Some commands in install.sh only work with Linux distributions using apt-get.
Adapt accordingly if your distribution uses another package manager.
$ ./install.sh packages- Answer 'Yes' when the Wireshark installer asks 'Should non-superusers be able to capture packets?'
$ ./install.sh gns3$ ./install.sh docker$ ./install.sh groupsIf the user is not correctly added to the groups, restart the machine.
-
For large topologies with many nodes (containers, VMs) you might need to increase the maximum number of open file descriptors (in the machine running GNS3) to start all the nodes simultaneously.
- Check the current limit with
ulimit -n - Increase the current limit (to 2048 e.g.) with
ulimit -n 2048
- Check the current limit with
-
KVM virtualization support. When running GNS3 in a virtual machine, enable nested virtualization.
Create a Python virtual environment to install the Python dependencies of the project (requires the python3X-venv package). To interact with the project, activate the virtual environment.
Inside the project's repository directory, run:
$ python3 -m venv venv
$ source venv/bin/activate
(venv) $ pip install -r requirements.txt
- If you want to build and run the SINETStream topology, follow instructions about Alternative A
- If you want to build and run the Gotham topology, follow instructions about Alternative B
- Some instructions must be followed regardless of the chosen topology (marked with
Alternative A and B). - We do not guarantee good execution of Gotham related code (Alternative B)
All the Dockerfiles and the dependencies that describe the emulated nodes (IoT devices, servers, attackers) are inside the ./Dockerfiles directory.
The build process of some Docker images depend on other images; instead of building them manually, the project includes a Makefile to automate the process.
Build necessary docker images with
$ make sinetstreamRun make to automatically build all the Docker images in the correct order:
$ make
Currently make alone fails with
docker build --file Dockerfiles/malware/Mirai/Dockerfile.builder --tag iotsim/mirai-builder Dockerfiles/malware/Mirai
[...]
Step 5/25 : RUN git clone https://github.com/jgamblin/Mirai-Source-Code.git
---> Using cache
---> fb4b9797e4ee
Step 6/25 : RUN mkdir go && go get github.com/go-sql-driver/mysql && go get github.com/mattn/go-shellwords
---> Running in 0739859c03c6
# filippo.io/edwards25519
go/src/filippo.io/edwards25519/scalar.go:166:53: cannot convert x (type []byte) to type *[32]byte
The command '/bin/sh -c mkdir go && go get github.com/go-sql-driver/mysql && go get github.com/mattn/go-shellwords' returned a non-zero code: 2
make: *** [Makefile:81: buildstatus/Mirai_builder] Error 2
This make can take a long time (approx 30 min depending on the host and network speed); you can parallelize the build process running make with the -j flag.
If you modify any Dockerfile, configuration file, program or any other file inside the ./Dockerfiles directory, run make again. It will rebuild the updated images and other images that depend on them.
Open GNS3, the GNS3 server must be running.
- (Optional) If you want the graphical user interface (gui) run. This runs both gns3server and the gns3 client.
$ gns3- If you don't need the graphical user interface (gui) run
$ gns3serverInside the ./src directory run:
(venv) $ python3 create_templates.py
You can verify the created templates in GNS3 select: Edit > Preferences > Docker containers
Inside the project's repository directory (gotham-iot-testbed), run:
make vyosisoThe artifacts (a .iso file and a .qcow2 file) will be downloaded into the directory~/Downloads directory.
Follow the instructions to import appliances in GNS3 https://docs.gns3.com/docs/using-gns3/beginners/import-gns3-appliance/. The router appliance file is located at ./router/iotsim-vyos.gns3a.
Inside the project's repository directory (gotham-iot-testbed), run
(venv) $ ./install_import_appliance.shInside the src directory, run:
(venv) $ python3 create_templates.py vyos_templateGNS3 must be running.
- Optional: in config_sinetstream.py, choose a project name
You can choose among various pre-configured topologies of different sizes.
| type of topology | description | composition |
|---|---|---|
| sinetstream_small | recommended to get familiar with GothX | 14 sensors, 3 MQTT brokers, 1 Kafka broker |
| sinetstream_big | it is the topology represented on the image bellow | 115 sensors, 3 MQTT brokers, 1 Kafka broker |
| sinetstream_max | maximum GothX topology without code modification | 450 sensors, 3 MQTT brokers, 1 Kafka broker |
| mqttset | topology used to reproduce MQTTSet * |
10 sensors, 1 MQTT brokers |
* to reproduce MQTTSet see this other instructions file.
- run a preconfigured tologoly (here
sinetstream_big)
(venv) $ python3 create_topology_sinetstream.py sinetstream_bigImage shows topology sinetstream_big on GNS3 Graphical Interface
Inside the ./src directory run:
(venv) $ python3 create_topology_gotham.py
GNS3 must be running.
Inside the src/ directory :
- in config_sinetstream.py, choose settings about the legitimate and malicious traffic that will be generated
- run the scenario
(venv) $ python3 run_scenario_sinetstream.pyInside the src/ directory, run the scenario
(venv) $ python3 run_scenario_gotham.py
- from a pcap file, extract network flows with CICFlowMeter, you get a csv file with features
- Use the script labelling.py to add the labels of each attack step
You may need to restart your user session to refresh user permissions, in case you encounter error like one of the following
uBridge is not available, path doesn't exist, or you just installed GNS3 and need to restart your user session to refresh user permissions.
or
Traceback (most recent call last):
File "~/gotham-iot-testbed/venv_gotham/lib/python3.12/site-packages/gns3fy/gns3fy.py", line 141, in http_call
_response.raise_for_status()
File "~/gotham-iot-testbed/venv_gotham/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 409 Client Error: Conflict for url: http://localhost:3080/v2/projects/7666c9a4-ddc1-41e5-8c3d-5f235bd18073/nodes/faeece40-a02c-450c-8870-d7766b29b3bd/start
manuel[dot]poisson[at]irisa[dot]fr