Release v1.14.0 - Fix Netprobe release downloads and more

Change Log

Version v1.14.0

Released 2024-06-05

Please report issues via github or the ITRS Community Forum.

⚠️ You will need to update to this version if you use geneos to install Netprobe releases direct from the ITRS website as the base name of the redirect URL has changed and the old version no longer works.

v1.14.0 Changes

• Update Go toolchain to use 1.22.4

• tools/geneos

  • Add options to tls create to specify destination directory, outputting certificate bundles and setting the expiry period in days. Also, do not initialise the TLS subsystem if it's not found unless the --force flag is used.

  • Hide --name option in deploy, repurpose short-form -n for new --nosave option

  • Rename logs option --nostandard to --no-stdout (with alias) for consistency

• cordial package

  • Only log file and function if logging at Debug level or greater

  • Split logging methods into their own file, add options to LogInit() to specify filename and/or lunberjack settings. Log filenames can also include ~/ prefix for home directory.

• tools/gateway-reporter

  • Add an optional XLSX password in output config section

v1.14.0 Fixes

• tools/geneos

  • Fix download base name for Netprobes to new value.

  • Fix deploy importing of keyfiles (only import if given on command line)

  • package install and package update - Fix the handling of package metadata (e.g. +el8) suffixes to that they are treated the same as a package without a metadata suffix - and the right one is selected for the system being installed

• pkg/config

  • Better recovery on failing to find the user's config directory, return an error instead of panic-ing

Release v1.13.2 - Breaking Changes - Beware! Minor Patch #2

Change Log

Version v1.13.2

Released 2024-05-23

Please report issues via github or the ITRS Community Forum.

⚠️ This release replaces v1.13.0 and v1.13.1 and addresses two issues found after release, one for dv2email not authenticating with all SMTP servers correctly and changes to geneos package uninstall to protect against unintended updates

⚠️ In addition to the breaking changes below please note that some documentation is still out-of-date, and the details in this changelog are more likely to be up-to-date.

v1.13.2 BREAKING CHANGES

⚠️ Note there are a significant number of BREAKING CHANGES in this release both for the geneos program, specifically around TLS and AES command line options and behaviour, but also for some of the pkg APIs.

• tools/geneos

  • tls Subsystem

    After reviewing the usability of the TLS subsystem commands to manage secure connections with users, we've re-worked some of the functionality and added a new tls export command. The command line flags for the tls import, deploy and init command have been revisited and an attempt been made to align them more closely. Now, the following flags have common meanings:

    • --tls/-T - enable TLS secured connections. This applies to all the init commands and the the deploy command. Was previously --secure/-C in deploy and --makecerts/-C in init commands. Both long forms are aliased to the new flag.
    • --signing-bundle/-C - specifies the source of signing certificate and private key and optional verification chain. This is new to deploy and replaces some of the functionality of --importcert for init commands where the short form has changed case for consistency.
    • --instance-bundle/-c - specifies the source of an instance certificate and private key and optional verification chain. This is new to deploy and replaces the rest of the functionality of --importcert for init commands.

    Other flags have been deprecated, such as --importkey/-k, whose functionality has been merged with the above.

    The new tls export command has been added to help extract those parts of the local TLS configuration signing certificate and private key that you need to push to a remote server to be able to create and manage certificates with a common trust relationship. Use the output of tls export with the other commands above and their --signing-bundle flags to do this.

    What this means for you is that if you need to manage Geneos instances across multiple servers (remote probes, standby Gateways etc.) but cannot use the existing SSH host features, then you can more easily keep the installation in sync for TLS certificates. Typical usage may be like this:

    On your main server run this, and copy the output to your clipboard (or output to a file with the --output/-o option):

    $ geneos tls export 
    --- BEGIN CERTIFICATE ---
    ...

    Then, log in to the new remote system and

    $ geneos deploy san -u [email protected] -C -
    [follow-prompts]
    Paste PEM formatted certificate bundle:
    [CTRL-V ENTER CTRL-D]

    Now your new Self-Announcing Netprobe will have a certificate created signed by the signing certificate and key you have exported on the central server.

  • aes Subsystem

    The AES subsystem was inconsistent and the functionality incomplete and, in reality, a mess. In this release we have reviewed the functionality of all of the AES commands. The support for AES keyfile creation, importing and management on remote hosts has been improved, but some issues may remain.

    The aes ls, aes encode, aes decode and aes password commands are largely the same while their internal implementations have been cleand-up. The other commands - aes new, aes import and aes set have all been rethought and their functionality and command line flags have changed. Please review the help text or documentation for full details. They should be more consistent and useful now, but as always please let us know if anything doesn't work as you would expect.

  • deploy

    As mentioned in the tls subsystem changes above, the command lin flags for the deploy command have changed, especially the -C and -c have been swapped around for consistency. Please note this if you have previously used deploy to create TLS protected Geneos deployments. The short form for --override is now -O.

  • package and init Subsystems

    The commands that support an --override flag have had the short form changed to -O to free the -T in deploy to be hort form for --tls.

• pkg/config

  The config package

v1.13.2 Changes

• Go dependency updated to 1.22.3

• tools/geneos

  In addition to the breaking changes, there have been a number of added features and functional improvements:

  • package uninstall now has an --update/-U flag to be explcit that the user wants to update instances which use older releases. Without this flag no release will be removed if it is referenced by an instance.

  • webserver instances will now automatically manage TLS though a custom cacerts file and a local keystore and security.properties file changes. This allows for both client and server TLS with trust chains. The instance chain file is added to the Java cacerts to add trust of other Geneos components, such as Gateways and SSO Agent while instance certificate and private key can be "real" PKI ones to offer a trusted TLS web interface.

• cordial

  • The top-level cordial package provides convenience functions, like LogInit(). This has been updated to not log source file and function unless the global log level if debug or trace.

• Use a new SMTP package - <github.com/wneessen/go-mail> - Swap out the old SMTP email package used to a newer, actively maintained one. The changes should not be user visible. The affected components are pkg/email and tools/dv2mail. There have been no changes to libraries/libemail as the changes required have not yet been tested and will be completed in a later release.

v1.13.2 Fixes

Many of the changes above were prompted while tracking down and fixing issues around existing APIs and features in geneos.

• tools/dv2email

  • Fix the default HTML template to use the correct, new names for data identifying columns and rows.

  • Fix failure to authenticate to some SMTP servers.

• tools/geneos

  • Fix support for legacy .rc files, which was broken in a previous release.

  • Fix tls ls -a to show root and signing certificates even without instance certificates created.

  • Fix the handling of file paths on Windows with the geneos.exe binary.

• pkg/config

  • Fix the handling of file paths on Windows.

  • Use the newer AbbreviateHome / ExpandHome / ExpandHomeBytes functions to remove common code.

Patch Release v1.12.1 with minor feature updates too

Change Log

Version v1.12.1

Released 2024-04-18

Please report issues via github or the ITRS Community Forum.

⚠ Please note this release was built under a WSL2 (Windows Subsystem for Linux) environment, which should not matter as the releases are in turn built in containers to constraint the environment, but please report any potential issues anyway!

v1.12.1 Changes

• General

  • Update Go used to 1.22.2

• tools/geneos

  • Add support for Collection Agent logs to log command
  • #195 - Add append capability to set command

• pkg/rest

  • Add a SetupRequest function to allow for custom HTTP client settings

• pkg/config

  • Fix a deadlock in UnmarshalKey wrapper
  • Add a Bytes method for encryption
  • Add StopOnInternalDefaultsErrors() option to bail when loading bad internal defaults

• libraries/libemail

  • Add a (deliberately undocumented) _SMTP_TLS_INSECURE environment variable to allow for edge cases in TLS testing
  • Build using Go DNS resolver and user lookup packages

v1.12.1 Fixes

• tools/geneos

  • Fix tls import command and update docs
  • Deal with missing key files as errors in aes sub-system

• pkg/host

  • Allow no username for SSH config, which then defaults to the user running the command
  • Return an error from IsAvailable() on failure, and use this to send Dial() errors up the stack
  • Log a debug level message if Dial fails

• cordial (top level package)

  • Add support in LogInit for - and /dev/null/NUL as log destinations - STDOUT and discard, respectively

Release v1.12.0

⚠️ Note that the binaries for this release were built, for the first time, in containers running under WSL2. This should not cause issue, but please report anything unexpected.

⚠️Note 2: files2dv has not been included as a pre-built binary yet. This should be done for the next patch release after further testing.

Change Log

Version v1.12.0

Released 2024-01-15

Please report issues via github or the ITRS Community Forum.

v1.12.0 Changes

• tools/files2dv New

  A new program that can be run as a Toolkit (soon as an API sampler) to scan files and directories and create one row per file. The contents of files can be checked and extracted into columns - the difference between the standard Geneos FKM and Statetracker plugins and using files2dv is that the whole file is scanned until each column is either filled in or no matches found, in the latter case a failure value can also be substituted. This is not a line-by-line scanner, like FKM.

  If no contents are checked then just the matching file's metadata is used.

• tools/gateway-reporter

  gateway-reporter has had a number of issues fixed but also gets a new csvdir output format, which can be used to create a "live" directory of CSV files and an optional Gateway include file to read back reports into a Gateway-of-Gateways.

• pkg/geneos

  Added an ExpandFileDates() function to emulate Geneos expansion of file paths including <today ...> placeholders. Currently only today is supported.

• pkg/config

  Change LookupTable() options to take variadic args and deprecate LookupTables()

  Map dashes (-) to underscored (_) when using environment variable lookups, so configuration items like file-name can be references as FILE_NAME in an env var.

v1.12.0 Fixes

• pkg/geneos

  Fixes around the handling of merged output files (where the var- prefix is dropped by the Gateway on some XML) so thet gateway-reporter works again.

• tools/geneos

  Fix fa2 start-up to include -secure when in secure mode.

  Better handling of remote hosts and display of a Flags column to show if hosts have been "hidden" (disabled). While documentation needs to be updated, the difference between hiding a host and disabling it (which is not yet possible) is that a hidden host can still be referenced explicitly on the command line, but is not included in any "all" lookups, such as geneos start

Release v1.11.2 - fix dv2email output ordering and support secure fileagents

Change Log

Version v1.11.2

Released 2023-12-07

Please report issues via github or the ITRS Community Forum.

v1.11.2 Fixes

• pkg/commands

  • add custom UnmarshalJSON to Dataview type to record the received order of headlines, rows and columns

• tools/dv2email

  • use dataview ordering (above) to render dataviews as seen from the Gateway, matching the normal Active Console
  • add support to differentiate between unset and empty --type on command line

v1.11.2 Changes

• tools/geneos

  • add support for secure arguments for fileagent releases 6.6 and higher

Release v1.11.1 - Fix `ac2` start-up on Linux

Change Log

Version v1.11.1

Released 2023-12-05

Please report issues via github or the ITRS Community Forum.

v1.11.1 Fixes

• tools/geneos

  • Fix ac2 start-up on Linux by passing on DISPLAY and XAUTHORITY envs to process, when they are defined

  • Output ISO8601 date/times in geneos tls ls outputs, not Go default format

Release v1.11.0

Change Log

Version v1.11.0

Released 2023-11-23

Please report issues via github or the ITRS Community Forum.

v1.11.0 Changes

• Major updates to dv2email

  dv2email has has a major work-over. In addition to fixing numerous bugs we've also added the ability to send dataviews as XLSX or Text Table attachments, splitting multiple Dataviews into either individual attachments or per-Entity. The export sub-command allows to you save Dataviews as files in XLSX, HTML or Text Table formats. Take a look and let us know what you think.

• New gateway-reporter

  A new tool to build monitoring coverage reports from static Gateway configurations. This is a new tool and support is limited to a core set of plugins and the data reported for each. The tool does not interrogate running Gateways and so does not attempt to resolve dynamic parts of the configuration. Output is in XLSX, CSV and JSON formats.

  This is a new work in progress based on an internal project for a client and we welcome suggestions and contributions. Please use the links above to send us feedback!

• New example holidays

  Another new tool, but very early in development. This uses the Python holiday module (which seems the only reliable, free source of global holiday data) to generate a list of holidays and their names in Go. It has to run in a docker container as the cpy3 Go package only supports Python 3.7. It may prove simpler just to write the whole tool in Python and avoid Go, but the local skills are sadly lacking at the moment. Also, we have Go based tools in pkg/geneos to build Gateway configuration XML in a nice, clean way.

  The aim is to be able to create Gateway include files with predefined Active Times of holidays that can be incorporated into more general monitoring.

  Again, suggestions and, more importantly, contributions welcomed.

• geneos has had a number of enhancements (and bug fixes, of course).

  The geneos aes encode command can now also create "app key" files for use with Gateway Hub and Obcerv Centralised Configs. To help with that the logic around how the Gateway command line is constructed has also had a refresh, with new options for the Centralised Config options. See the documentation for encode and the gateway for more information.

v1.11.0 Fixes

• pkg/

  • Removal of all logging in packages. This was left over from early development and is considered bad practice.

• tools/geneos

  • Fix a mistake in handling remote hosts in geneos show -V for validation runs.

  • Fix geneos home matching of instances when the same name existed on local and remote hosts.

  • Fix issues in geneos package sub-system commands. They should now work more consistently and do what you expect.

Release v1.10.4 - Fix remote log tailing and aes commands

Change Log

Version v1.10.4

Released 2023-11-03

Please report issues via github or the ITRS Community Forum.

v1.10.4 Fixes

• pkg/hosts - disable sftp concurrent reads as this seems to mess with the file offset after io.Copy() and causes geneos logs to do strange things with remote log files

• tools/geneos

  • Fix a number of issues with geneos logs, including the results of the above but also how log file references are stored so that when you are tailing the same named instance on two hosts they get mixed up.

  • Update aes commands to be less error prone, change import to not automatically update key files in instances, but require a new --update flag. Update docs.

Release v1.10.3 - Fixes

Change Log

Version v1.10.3

Released 2023-11-02

Please report issues via github or the ITRS Community Forum.

v1.10.3 Fixes

• tools/geneos - Fix duplication of non-host qualified names introduced in last fixes - a string list was re-used instead of being locally allocated inside blocks.

• pkg/config - In Sub() copy the pointer to the mutex from the parent so that locks apply to the whole config object. Fixes concurrent access panic when methods are called on both the parent and the new sub-config. While here, also copy the other config fields, like delimiter.

Release v1.10.2 - Fix `geneos` wildcards and remote ssh sessions

Change Log

Version v1.10.2

Released 2023-10-31

Please report issues via github or the ITRS Community Forum.

⚠️ Note: The previous patch version was removed after finding the fix for wildcards was only partial. This second patch release addresses that issue.

v1.10.2 Fixes

• tools/geneos

  • The recently introduced wildcarding for instance names was causing non-matching instance names to be evaluated all "all", so geneos stop test would stop all instances. Fix this by checking the list of names returned from the pattern match and if empty assume the input is not a pattern and leave it alone.
  • The fix to pkg/host below tries to ensure that running multiple sessions to remote hosts works more reliably, especially those where commands take some time to return, like for start

• pkg/process - Fix Daemon() for Windows by adding a DETACHED_PROCESS flag to new proc attributes.

• pkg/config

  • Protect global map access with a mutex in expandoptions.go
  • Make the viper mutex a plain one, not RWLock. Reorder calls to Unlock to cover decode()

• pkh/host - Allow for remote SSH session limits, retry NewSession() up to 10 times with a 250ms delay. This limit is in the remote server, so cannot be overridden.