Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add inventory monitor module #4680

Merged
merged 26 commits into from
Oct 23, 2020

Conversation

chimp1984
Copy link
Contributor

This adds a new module containing the new inventory monitor project.
It is running since a few weeks under http://46.101.179.224/ and some of the seeds have updated to that branch already.
It is the implementation of bisq-network/projects#45
@jmacxx is working on a webapp which should finally replace the simple http server used atm.
It will still need fine tuning of the deviation alert/warning thresholds but it is already pretty useful and helps to detect seed node issues.
Most of the code is only relevant to the monitor project and seed nodes, so it carries very low risk for normal users.

@chimp1984
Copy link
Contributor Author

@ripcurlx Please remove rule that empty catch with ignore as param name is causing warnings.

@chimp1984
Copy link
Contributor Author

@ripcurlx Please remove rule that empty catch when using ignore is causing issues

@ripcurlx
Copy link
Contributor

@ripcurlx Please remove rule that empty catch when using ignore is causing issues

Just removed the pattern.

Delay the boolean property setter as otherwise our listener might never
get triggered if property is set synchronously before listener registration.
Remove shutdown thread.
Cancel future in case tor is not created yet.
This has caused the slow tor startup of about 20-30 sec. Without that it is about 5 sec.
Simple monitor dumping json files with data result and request/response time to disk. Can be used by a simple web server to show state of seed nodes.
…a as well like DAO state

Add more data to inventory map and change type of value to String.
…es not make sense to limit requests).

Add html output and webserver
Add average and color codes to html
Add maxConnections
Add DeviationSeverity enum
Add custom seed node file
Use InventoryItem as key in inventory map instead of string
Various improvements...
Rearrange request info fields
Add all seeds
…ed shutDown

We also do not call listeners in that case as some listeners have some restart routines
on that event which is not what we want at shut down
Copy link
Member

@sqrrm sqrrm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Some comments that don't really have to be tended to.


@Override
public void onMessage(NetworkEnvelope networkEnvelope, Connection connection) {
if (networkEnvelope instanceof GetInventoryRequest) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As always, prefer to return early instead of huge blocks.

Comment on lines +98 to +121
p2PDataStorage.getMapForDataResponse(getInventoryRequest.getVersion()).values().stream()
.map(e -> e.getClass().getSimpleName())
.forEach(className -> {
Optional<InventoryItem> optionalEnum = Enums.getIfPresent(InventoryItem.class, className);
if (optionalEnum.isPresent()) {
InventoryItem key = optionalEnum.get();
dataObjects.putIfAbsent(key, 0);
int prev = dataObjects.get(key);
dataObjects.put(key, prev + 1);
}
});
p2PDataStorage.getMap().values().stream()
.map(ProtectedStorageEntry::getProtectedStoragePayload)
.filter(Objects::nonNull)
.map(e -> e.getClass().getSimpleName())
.forEach(className -> {
Optional<InventoryItem> optionalEnum = Enums.getIfPresent(InventoryItem.class, className);
if (optionalEnum.isPresent()) {
InventoryItem key = optionalEnum.get();
dataObjects.putIfAbsent(key, 0);
int prev = dataObjects.get(key);
dataObjects.put(key, prev + 1);
}
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
p2PDataStorage.getMapForDataResponse(getInventoryRequest.getVersion()).values().stream()
.map(e -> e.getClass().getSimpleName())
.forEach(className -> {
Optional<InventoryItem> optionalEnum = Enums.getIfPresent(InventoryItem.class, className);
if (optionalEnum.isPresent()) {
InventoryItem key = optionalEnum.get();
dataObjects.putIfAbsent(key, 0);
int prev = dataObjects.get(key);
dataObjects.put(key, prev + 1);
}
});
p2PDataStorage.getMap().values().stream()
.map(ProtectedStorageEntry::getProtectedStoragePayload)
.filter(Objects::nonNull)
.map(e -> e.getClass().getSimpleName())
.forEach(className -> {
Optional<InventoryItem> optionalEnum = Enums.getIfPresent(InventoryItem.class, className);
if (optionalEnum.isPresent()) {
InventoryItem key = optionalEnum.get();
dataObjects.putIfAbsent(key, 0);
int prev = dataObjects.get(key);
dataObjects.put(key, prev + 1);
}
});
p2PDataStorage.getMapForDataResponse(getInventoryRequest.getVersion()).values().stream()
.map(e -> e.getClass().getSimpleName())
.forEach(className -> addItem(dataObjects, className));
p2PDataStorage.getMap().values().stream()
.map(ProtectedStorageEntry::getProtectedStoragePayload)
.filter(Objects::nonNull)
.map(e -> e.getClass().getSimpleName())
.forEach(className -> addItem(dataObjects, className));

With addItem()

    private void addItem(Map<InventoryItem, Integer> dataObjects, String className) {
        Optional<InventoryItem> optionalEnum = Enums.getIfPresent(InventoryItem.class, className);
        if (optionalEnum.isPresent()) {
            InventoryItem key = optionalEnum.get();
            dataObjects.putIfAbsent(key, 0);
            int prev = dataObjects.get(key);
            dataObjects.put(key, prev + 1);
        }
    }

Res.setup(); // Used for some formatting in the webserver

// We do not set any capabilities as we don't want to receive any network data beside our response.
// We also do not use capabilities for the request/response messages as we only connect to seeds nodes and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is lacking a bit when it comes to the punch line.

Copy link
Member

@sqrrm sqrrm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK

@chimp1984 Comments can be fixed elsewhere if you feel like it, I want to merge this.

@sqrrm sqrrm merged commit 93a3188 into bisq-network:master Oct 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants