v1.5.0-beta - testing invited

This beta release of cordial v1.5.0 is quite extensive and brings a number of new features and breaking changes. In addition lots of code internals, especially in the geneos tool introduce further flexibility for the future and increased security of memory and credentials.

One new highlight is the dv2html program, perhaps probably better calleddv2email and it may well get a name change before release, which lets you snapshot and email Dataviews from a Geneos Action or Alert. The README file is very much out-of-date (see below) and instead please see the [dv2html.yaml](tools/geneos/dv2html.yaml] configuration file for more details during this test period.

Documentation has not had much of an update and this is where work will be focused before final release. Any bugs found will also be addressed where possible. The aim it to have a release within a week or so.

One other change worth noting is that the geneos and dv2html binaries will look for configuration files that match their base-names, so if you download the attached geneos-v1.5.0-beta and do not rename it then it will look for configuration files in ${HOME}/.config/geneos-v1.5.0-beta and so on. So, if you download either directly then rename them before use.

The (incomplete) changelog is below:

Version v1.5.0-beta - 2023/05/16

• BREAKING CHANGES

  • pkg/logger:

    • REMOVED. This simplistic internal logging package has been removed
      in favour of more mature solutions, such as zerolog.

  • tools/geneos:

    • The way SAN instances handle gateway connections has been fixed to
      allow gateway represented as FDQNs or IP addresses. The old way
      resulted in a mess - viper defaults to dots ('.') as configuration
      item hierarchy delimiters and this caused issues. Most users with
      SANs should not notice any change, but if you see problems please
      check the san XML file and correct the gateways section as
      necessary. The easiest way is probably to remove and re-set them
      using geneos set san XXX -g gateway.example.com ...

    • Similarly to the above any variables defined in either SAN or
      Gateway configurations for use in the XML templates will have not
      worked in a case sensitive manner to align with how Geneos does
      it. To fix this the format of the variables section has been
      changed to move the variable name from the configuration key to a
      separate structure as a value. Code has been added to
      automatically convert from the old format to the new when the
      configuration file is updated however there is no fix for the
      correction of variable name case being wrong. Please review and
      adjust as necessary.

    • Support for running under sudo or for an instances where the
      user is different to the user running the command has been
      deprecated. Security is hard, and the support for these was poorly
      implemented. A better way will be coming in a later release.

      This may mean that where users has configured netprobes to run as
      different users and have run sudo geneos start to let the tool
      do the right thing will sun into issues. Please be careful if any
      of your instances run as other users and so not run the geneos
      tool with sudo. There is no additional checking/rejection of
      running under sudo or any other privilege escalation system so
      this is important!

• Changes

  • There has been a large amount of refactoring and rebalancing of the
    code-base. Most of this should not be user visible, but some
    previous public APIs have changed. As with all major changes there
    may be problems that have no been caught in testing. Please report
    anything you see as either a github issue or via the ITRS Community
    Forum.

    Specific changes worth mentioning include:

    • memguard
      support for protected memory. Most credentials should now be
      handled as Enclaves (for plaintext or private keys) or as
      LockedBuffers (for ciphertexts of sensitive data).

      The changes are ongoing and in addition to adding a layer of data
      security to cordial an added benefit is the catching of memory
      misuse etc. If you see errors, panic etc. please report them!

    • A number of the previous package APIs have undergone review and
      refactoring. In particular the pkg/config API has been through
      the wringer and if you have any code that relies on it from v1.4
      and before then it will need updating. There are many new
      functions, which is normal, but also some older entry points have
      been renamed or had the argument signatures changed. Please review
      the documentation to see what the methods and functions have
      become.

    • Credentials support. There is both general purpose and geneos
      specific support for the local storage of credentials. At rest the
      passwords are stored in Geneos AES256 format with a key that is
      auto-generated if not found. To decode these passwords you must
      have both the key file (which is by default only user readable)
      and the credentials file. Future support for other credentials,
      such as OAuth style client secrets and tokens, will be
      forthcoming. The username and the domain the the credentials apply
      to are not encrypted, by design. This is however subject to
      change.

      The credentials support currently works with a plain test domain
      that is used to match the destination "longest match wins", e.g.
      for a URL this may be a full or partial domain name, and for
      Geneos component authentication, e.g. the REST command API, the
      domain is in the form gateway:NAME. More will be added later,
      including SSH password and private keys.

  • tools/geneos:

    • Move aes and tls to their own directories as new subsystems.
    • Add host and package subsystems and create aliases for
      original commands, e.g.
      • add host becomes host add
      • install becomes package install
      • etc.
    • The set user, show user etc. commands are now under single
      config sub-command, e.g. geneos config set mykey=value
    • The set global and related commands have been deprecated.
    • The new package subsystem command pulls all Geneos release
      management into one place
    • New login and logout commands to manage credentials.

  • tools/dv2html:

    • This new program can be run as an Action or Effect to capture a
      complete Dataview and send it as email. The configuration is
      extensive and the layout and contents are completely configurable
      through the use of Go templates.

• Fixes

  • tools/geneos:

    • Version checking of local release archives was broken because of
      overloading of a common function. This is now split and checking
      should work once again.

    • Many reported issues on github have been fixed.

• To Do:

  • tools/geneos:

    • Local storage of encrypted passwords for remote SSH access needs documenting

Version v1.4.4 - 2023/04/12

• Fixes

  • New Default expand option should NOT itself default to nil

Version v1.4.3 - 2023/04/12

NOTE: Later error found in the geneos tool, so please do not use this release.

• Fixes

  • tools/geneos: fix ps not showing open ports on systems with IPv6 enabled
  • tools/geneos: make ls and ps command flags more consistent
  • tools/geneos: add an -el8 runtime to docker images when built
  • tools/geneos: fix RHEL8/Centos8 download support for localhost using new SetStringMapString() method
  • pkg/config: add SetStringMapString() methods to support settings maps (which viper doesn't support until you write a file out and read it back)
  • tools/geneos: adjust the way we choose package version, convert "-el8" to "+el8" to satisfy semver ordering
  • tools/geneos: package version number are now prefixes only

• Changes

  • tools/geneos: add -f flag to ps command to show open files. formatting subject to change for now.
  • tools/geneos: add a update ls command to show available package versions
  • pkg/config: added more ExpandOptions and support more Get* functions such as GetInt
  • pkg/geneos: added more Geneos XML config support, specifically Sampler Schemas and Standardised Formatting
  • libraries/libemail: added initial msTeams notification function

Bug fix release

Change Log

Version v1.4.2 - 2022/12/21

• Fixes

  • tools/geneos: fix update to only optional restart (-R) the component type given
  • tools/geneos: check RHEL8 download in a case independent way - fixes remotes
  • tools/geneos: create user config directory for remote hosts in case of old location for main config
  • tools/geneos: install should error out is passed @host instead of -H host
  • tools/geneos: ssh known hosts handling improved (for mixed IP / hostnames)
  • tools/geneos: remote hosts with IP names are now renamed A-B-C-D to avoid issues with viper names

Release v1.4.1

Change Log

Version v1.4.1 - 2022/12/19

• Fixes

  • tools/geneos: check return from user.Current() as it can fail (but shouldn't)
  • tools/geneos: numerous fixes for logic around handling of remote hosts
  • tools/geneos: fix remote host naming to be work with capitalisations
  • tools/geneos: actually load SSH private key files, if available
  • tools/geneos: re-order SSH HostKeyAlgorithms so that, bizarrely, IP based remotes work
  • tools/geneos: better handling of instance config aliases when writing config files
  • tools/geneos: fixes to unset to ignore values that may be passed in with keys to unset
  • tools/geneos: refactor CopyInstance() to preserve ports, other details
  • build: create static executables, using alpine, and a centos 7 compatible libemail.so
  • tools/geneos: add the beginnings of support for YAML instance config files. not enabled yet.
  • tools/geneos: fix crash when importing to common directories of components without the component name
  • tools/geneos: fix fileagent support by adding implicit imports with side-effects for all component packages
  • tools/geneos: skip failed permissions on /proc/*/fd - let 'ps' work for restricted processes
  • tools/geneos: fix update-during-install support, add --force flag for this too
  • tools/geneos: fix logic to match latest packages when major number changes

• Changes

  • tools/geneos: clean-up various comments, refactor methods, add license/copyright notices to many files
  • pkg/config: Add an options expr prefix to expansion items which supports [github.com/maja42/goval] syntax
  • pkg/config: API change: Add options to the config expansion functions rather than just lookup maps
  • tools/geneos: add SSH password support for remote hosts
  • tools/geneos: support embedded SSH passwords in hosts config, using new 'set host' sub-command
  • tools/geneos: support additional SSH private key files per host via 'set host sshkeys=X,Y' sub-command
  • tools/geneos: begin implementation of support for YAML config files via 'configtype' user setting
  • pkg/geneos: add EnvironmentRef and fix periodStartTime attribute

• Other

  • tools/geneos: ongoing documentation and command help usage updates
  • tools/geneos: update README.md with more information about instance configuration files and their values (@gvastel)

Release Candidate 1

Change Log

Version v1.4.1-rc1 - 2022/12/12

• Fixes

  • tools/geneos: numerous fixes for logic around handling of remote hosts
  • tools/geneos: fix remote host naming to be work with capitalisations
  • tools/geneos: actually load SSH private key files, if available
  • tools/geneos: re-order SSH HostKeyAlgorithms so that, bizarrely, IP based remotes work
  • tools/geneos: better handling of instance config aliases when writing config files
  • tools/geneos: fixes to unset to ignore values that may be passed in with keys to unset
  • tools/geneos: refactor CopyInstance() to preserve ports, other details
  • build: create static executables, using alpine, and a centos 7 compatible libemail.so
  • tools/geneos: add the beginnings of support for YAML instance config files. not enabled yet.
  • tools/geneos: fix crash when importing to common directories of components without the component name
  • tools/geneos: fix fileagent support by adding implicit imports with side-effects for all component packages
  • tools/geneos: skip failed permissions on /proc/*/fd - let 'ps' work for restricted processes
  • tools/geneos: fix update-during-install support, add --force flag for this too
  • tools/geneos: fix logic to match latest packages when major number changes

• Changes

  • tools/geneos: clean-up various comments, refactor methods, add license/copyright notices to many files
  • pkg/config: Add an options expr prefix to expansion items which supports [github.com/maja42/goval] syntax
  • pkg/config: API change: Add options to the config expansion functions rather than just lookup maps
  • tools/geneos: add SSH password support for remote hosts
  • tools/geneos: support embedded SSH passwords in hosts config, using new 'set host' sub-command
  • tools/geneos: support additional SSH private key files per host via 'set host sshkeys=X,Y' sub-command
  • tools/geneos: begin implementation of support for YAML config files via 'configtype' user setting
  • pkg/geneos: add EnvironmentRef and fix periodStartTime attribute

• Other

  • tools/geneos: ongoing documentation and command help usage updates
  • tools/geneos: update README.md with more information about instance configuration files and their values (@gvastel)

v1.4.1-beta - Lots of fixes and updates

Change Log

Version v1.4.1-beta - 2022/11/25

• Fixes

  • tools/geneos: many fixes for logic around handling of remote hosts
  • tools/geneos: fix remote host naming to be work with capitalisations
  • tools/geneos: actually load SSH private key files, if available
  • tools/geneos: re-order SSH HostKeyAlgorithms so that, bizarrely, IP based remotes work
  • tools/geneos: better handling of instance config aliases when writing config files
  • tools/geneos: fixes to unset to ignore values that may be passed in with keys to unset
  • tools/geneos: refactor CopyInstance() to preserve ports, other details
  • build: create static executables, using alpine, and a centos 7 compatible libemail.so
  • tools/geneos: add the beginnings of support for YAML instance config files. not enabled yet.
  • tools/geneos: fix crash when importing to common directories of components without the component name
  • tools/geneos: fix fileagent support by adding implicit imports with side-effects for all component packages
  • tools/geneos: skip failed permissions on /prox/*/fd - let 'ps' work for restricted processes
  • tools/geneos: fix update-during-install support, add --force flag for this too
  • tools/geneos: fix logic to match latest packages when major number changes

• Changes

  • pkg/config: API change: Add options to the config expansion functions rather than just lookup maps
  • tools/geneos: add SSH password support for remote hosts
  • tools/geneos: support embedded SSH passwords in hosts config, using new 'set host' sub-command
  • tools/geneos: support additional SSH private key files per host via 'set host sshkeys=X,Y' sub-command
  • tools/geneos: begin implementation of support for YAML config files via 'configtype' user setting
  • pkg/geneos: add EnvironmentRef and fix periodStartTime attribute

• Other

  • tools/geneos: update README.md with more information about instance configuration files and their values (@gvastel)

Fix issues reported in #8

Fix issues creating user configuration directories and file permissioning when running 'init' commands as root but directed at a specific user.

CHANGELOG will be updated later

Patch release: fix file ownerships for 'init' command in tools/geneos

Change Log

Version v1.3.1 - 2022/11/01

• Fixes

  • tools/geneos: chown files and directories creates when run as root
  • tools/geneos: ensure plain 'init' creates all components dirs

Release v1.3.0

Change Log

Version v1.3.0 - 2022/10/25

Changes

• PagerDuty integration
• Merged ServiceNow integration, single binary build
• tools/geneos: add instance protection against stop (and related) or delete commands
• tools/geneos: support legacy command through emulating *ctl named commands
• tools/geneos: allow remote operations without local directories

Fixes

• tools/geneos: fix logic around creating user default AES keyfiles and directory permissions
• tools/geneos: round certificate expiry to midnight
• tools/geneos: round tls remaining column to seconds correctly
• tools/geneos: fix webserver command build typo. now webserver starts correctly
• libemail: fix default _SUBJECT handling
• tools/geneos: split over complex 'init' command into sub-commands
• updated command usage information and reordered various internal function calls
• tools/geneos: add password verify to aes encode and a --once flag to override
• tools/geneos: add local JoinSlash and Dir to use Linux paths on Windows builds
• tools/geneos: fix ssh-agent support on windows
• tools/geneos: build on windows
• integrations: Add PagerDuty integration
• Integrations: Merge ServiceNow binaries into one
• tools/geneos: change internal remote Stat() API