Implement Tls Tester

.eslintrc.js

@@ -18,10 +18,12 @@ module.exports = {
     'import/no-unresolved': ['error', { commonjs: true }],
     'no-unused-expressions': ['error', { allowShortCircuit: true, allowTernary: true }],
     'object-curly-newline': ['error', { multiline: true }],
-    'no-multiple-empty-lines': ['error', { max: 2, maxBOF: 0, maxEOF: 1 }]
+    'no-multiple-empty-lines': ['error', { max: 2, maxBOF: 0, maxEOF: 1 }],
+    'newline-per-chained-call': 'off'
   },
   env: { node: true },
   parserOptions: { ecmaVersion: 2021 },
+  parser: 'babel-eslint', // required for private class members as eslint only supports ECMA Stage 4, will be able to remove in the future
   settings: {
     'import/resolver': {
       node: {

README.md

@@ -6,7 +6,7 @@
   <br/>
   <br/>
   <h2>purpleteam orchestrator</h2><br/>
-    Orchestrator component of <a href="https://purpleteam-labs.com/" title="purpleteam">purpleteam</a> - Currently in alpha
+    Orchestrator component of <a href="https://purpleteam-labs.com/" title="purpleteam"><em>PurpleTeam</em></a> - Currently in alpha
   <br/><br/>
 
   <a href="https://www.gnu.org/licenses/agpl-3.0" title="license">
@@ -24,9 +24,11 @@
 </div>
 
 
-If you are setting up the orchestrator, you will be targeting the `local` environment.
+If you are setting up the _orchestrator_, you will be targeting the `local` environment.
 
-Clone this repository.
+Clone or fork this repository.
+
+If you are developing this project:
 
 `cd` to the repository root directory and run:  
 ```shell
@@ -38,11 +40,45 @@ npm install
 Copy the config/config.example.json to config/config.local.json.  
 Use the config/config.js for documentation and further examples.  
 
-**`testerFeedbackComms.medium`** Long Polling `lp` is supported in both `local` and `cloud` environments. Server Sent Events `sse` is only supported in the `local` environment due to AWS limitations. Both `lp` and `sse` are real-time. Both implementations have their pros and cons.
+### `testerFeedbackComms.medium`
+
+Long Polling (`lp`) is supported in both `local` and `cloud` environments. Server Sent Events (`sse`) is only supported in the `local` environment due to AWS limitations. Both `lp` and `sse` are real-time. Both implementations have their pros and cons.
+
+The client-side (_PurpleTeam_ CLI) will use the `testerFeedbackComms.medium` that the _orchestrator_ instructs it to use.
+
+#### `sse`
+
+Using `sse` is one way communications after the initial subscription from the CLI to the _orchestrator_. Redis pub/sub is used between the _Testers_ and the _orchestrator_ to publish _Tester_ feedback. If the CLI is stopped (not subscribed) at any point while the back-end is performing a _Test Run_, events will be lost. If the CLI runs `test` again before the _Test Run_ is complete, it will receive messages from then to _Test Run_ completion, and will also receive the _Outcomes_ archive on _Test Run_ completion. If the CLI runs `test` after a _Test Run_ is complete, no messages will be received from the previous _Test Run_, the previous _Test Runs_ _Outcomes_ archive is destroyed, and the CLI will start receiving messages as _Tester_ feedback is produced.
+
+#### `lp`
+
+Using `lp` is request-response communications. A request is made and only answered when there are [_Tester_](https://purpleteam-labs.com/doc/definitions/) feedback messages available, or the application specific (rather than AWS Api Gateway) time-out is exceeded.
+
+As soon as the CLI receives a set (one to many) of _Tester_ feedback messages, it makes another request to the _orchestrator_ (if running in `local` env), or API (if running in `cloud` env). Redis pub/sub is used between the _Testers_ and the _orchestrator_ to publish _Tester_ feedback.  
+
+A little more detail:
+
+If running in the `cloud` environment the first set of _Tester_ feedback messages are persisted in the _orchestrator_'s memory, and subsequent _Tester_ feedback messages are persisted to in-memory Redis lists. The reasons for this are at least two-fold. The _orchestrator_ sets up the subscriptions to the Redis channels on behalf of the CLI. The reasons for this is so that:
+
+* The _orchestrator_ knows when the _Testers_ are finished in order to clean-up before the next _Test Run_
+* and to make sure that no messages are missed due to the CLI either being off-line or late to subscribe
+
+This means that if the CLI is stopped momentarily during a _Test Run_ or if _Tester_ messages pile up before the CLI has subscribed (which normally occurs on receipt of the initial set of _Tester_ status messages), as long as the CLI subscribes before the _Test Run_ has completed, it will receive all stored _Tester_ feedback messages and the _Outcomes_ archive when the _Test Run_ is complete.
 
-See the [purpleteam CLI documentation](https://github.com/purpleteam-labs/purpleteam#configure) for further details around this value.
 
-<br>
 
-Once you have cloned, installed and configured the orchestrator, head back to the [local setup](https://doc.purpleteam-labs.com/local/local-setup.html) documentation to continue setting up the other purpleteam components.
 
+
+
+
+
+
+
+
+
+> Additional background: This may change in the future, WebSockets is also an option we may implement in the future, but implementing WebSockets would mean we would have to change our entire authn approach. Our chosen cloud infrastructure AWS Api Gateway does not support streaming and it does not support the OAuth Client Credentials Flow with Cognito User Pools.
+
+
+<br>
+
+Once you have worked through the above steps, head back to the [local setup](https://purpleteam-labs.com/doc/local/set-up/) documentation to continue setting up the other _PurpleTeam_ components.

compose/orchestrator-testers-compose.debug_tls-scanner.yml

@@ -0,0 +1,14 @@
+# https://docs.docker.com/compose/extends/
+
+version: "3.7"
+
+services:
+
+  tls-scanner:
+    ports:
+     - "9229:9229"
+    command:
+     - node
+     - "--inspect-brk=0.0.0.0"
+     - "./index.js"
+

compose/orchestrator-testers-compose.yml

@@ -27,7 +27,7 @@ services:
       # Build-time args
       args:
         LOCAL_USER_ID: ${APP_SCANNER_USER_ID}
-        LOCAL_GROUP_ID: ${APP_SCANNER_GROUP_ID}
+        LOCAL_GROUP_ID: ${PT_GROUP_ID}
     environment:
       - NODE_ENV=local
       - AWS_ACCESS_KEY_ID=arbitrary_aws_access_key_id
@@ -43,6 +43,33 @@ services:
        source: ${HOST_DIR}
        target: /var/log/purpleteam/outcomes
 
+  tls-scanner:
+    depends_on:
+     - redis
+    networks:
+      pt-net:
+        ipv4_address: 172.25.0.140
+    ports:
+     - "3020:3020"
+    build:
+      context: ../../purpleteam-tls-scanner/
+      # Build-time args
+      args:
+        LOCAL_USER_ID: ${TLS_SCANNER_USER_ID}
+        LOCAL_GROUP_ID: ${PT_GROUP_ID}
+    environment:
+      - NODE_ENV=local
+    # Following values are analogous to their Dockerfile counterparts:
+    # -i : Keep STDIN open even if not attached
+    # -t : Allocate a pseudo-tty
+    stdin_open: true
+    tty: true
+    container_name: pt-tls-scanner-cont
+    volumes:
+     - type: bind
+       source: ${HOST_DIR}
+       target: /var/log/purpleteam/outcomes
+
   orchestrator:
     depends_on:
      - redis
@@ -57,7 +84,7 @@ services:
       # Build-time args
       args:
         LOCAL_USER_ID: ${ORCHESTRATOR_USER_ID}
-        LOCAL_GROUP_ID: ${ORCHESTRATOR_GROUP_ID}
+        LOCAL_GROUP_ID: ${PT_GROUP_ID}
     environment:
      - NODE_ENV=local
     # Following values are analogous to their Dockerfile counterparts:

config/config.example.json

@@ -11,12 +11,12 @@
       "active": true
     },
     "server": {
-      "url": "http://172.17.0.4:3010",
+      "url": "http://172.25.0.130:3010",
       "active": false
     },
     "tls": {
-      "url": "http://172.17.0.5:3020",
-      "active": false
+      "url": "http://172.25.0.140:3020",
+      "active": true
     }
   },
   "outcomes": {

config/config.js

@@ -15,10 +15,12 @@
 // along with purpleteam. If not, see <https://www.gnu.org/licenses/>.
 
 const convict = require('convict');
-const convictFormatWithValidator = require('convict-format-with-validator');
+const { duration } = require('convict-format-with-moment');
+const { url } = require('convict-format-with-validator');
 const path = require('path');
 
-convict.addFormat(convictFormatWithValidator.url);
+convict.addFormat(duration);
+convict.addFormat(url);
 
 const schema = {
   env: {
@@ -34,6 +36,18 @@ const schema = {
       default: 'info'
     }
   },
+  processMonitoring: {
+    on: {
+      doc: 'Whether or not to capture and log process events.',
+      format: 'Boolean',
+      default: false
+    },
+    interval: {
+      doc: 'The interval in milliseconds to capture and log the process events.',
+      format: 'duration',
+      default: 10000
+    }
+  },
   host: {
     port: {
       doc: 'The port of this host.',
@@ -63,11 +77,6 @@ const schema = {
       doc: 'The type of browser to run tests through.',
       format: ['chrome', 'firefox'],
       default: 'chrome'
-    },
-    reportFormat: {
-      doc: 'The supported formats that reports may be written in.',
-      format: ['html', 'json', 'md'],
-      default: 'html'
     }
   },
   testers: {
@@ -84,7 +93,8 @@ const schema = {
         default: true
       },
       testPlanRoute: '/test-plan',
-      runJobRoute: '/run-job'
+      initTesterRoute: '/init-tester',
+      startTesterRoute: '/start-tester'
     },
     server: {
       name: 'server',
@@ -99,7 +109,8 @@ const schema = {
         default: true
       },
       testPlanRoute: '/test-plan',
-      runJobRoute: '/run-job'
+      initTesterRoute: '/init-tester',
+      startTesterRoute: '/start-tester'
     },
     tls: {
       name: 'tls',
@@ -114,14 +125,15 @@ const schema = {
         default: true
       },
       testPlanRoute: '/test-plan',
-      runJobRoute: '/run-job'
+      initTesterRoute: '/init-tester',
+      startTesterRoute: '/start-tester'
     }
   },
-  buildUserConfig: {
+  job: {
     version: {
-      doc: 'The version of the build user config accepted by this API.',
-      format: ['0.1.0-alpha.1'],
-      default: '0.1.0-alpha.1'
+      doc: 'The version of the Job accepted by this API.',
+      format: ['0.1.0-alpha.1', '1.0.0-alpha.3'],
+      default: '1.0.0-alpha.3'
     }
   },
   outcomes: {

healthcheck.js

@@ -1,18 +1,18 @@
 // Copyright (C) 2017-2021 BinaryMist Limited. All rights reserved.
 
-// This file is part of purpleteam.
+// This file is part of PurpleTeam.
 
-// purpleteam is free software: you can redistribute it and/or modify
+// PurpleTeam is free software: you can redistribute it and/or modify
 // it under the terms of the GNU Affero General Public License as published by
 // the Free Software Foundation version 3.
 
-// purpleteam is distributed in the hope that it will be useful,
+// PurpleTeam is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
 // GNU Affero General Public License for more details.
 
 // You should have received a copy of the GNU Affero General Public License
-// along with purpleteam. If not, see <https://www.gnu.org/licenses/>.
+// along with PurpleTeam. If not, see <https://www.gnu.org/licenses/>.
 
 const http = require('http');
 require('convict');

index.js

@@ -1,18 +1,18 @@
 // Copyright (C) 2017-2021 BinaryMist Limited. All rights reserved.
 
-// This file is part of purpleteam.
+// This file is part of PurpleTeam.
 
-// purpleteam is free software: you can redistribute it and/or modify
+// PurpleTeam is free software: you can redistribute it and/or modify
 // it under the terms of the GNU Affero General Public License as published by
 // the Free Software Foundation version 3.
 
-// purpleteam is distributed in the hope that it will be useful,
+// PurpleTeam is distributed in the hope that it will be useful,
 // but WITHOUT ANY WARRANTY; without even the implied warranty of
 // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
 // GNU Affero General Public License for more details.
 
 // You should have received a copy of the GNU Affero General Public License
-// along with purpleteam. If not, see <https://www.gnu.org/licenses/>.
+// along with PurpleTeam. If not, see <https://www.gnu.org/licenses/>.
 
 require('app-module-path/register');
 const server = require('src/server');