A tiny shell-script based TAP-compliant testing framework
______ ______ __ __ ______ /\__ _\ /\ ___\ /\ \_\ \ /\__ _\ \/_/\ \/ \ \___ \ \ \ __ \ \/_/\ \/ \ \_\ \/\_____\ \ \_\ \_\ \ \_\ \/_/ \/_____/ \/_/\/_/ \/_/
- Installation
- Usage
- API - core
- API - file
- API - string
- API - jq
- API - diff
- API - colordiff
- API - shxml
In general you don't have to install tsht
, simply add the wrapper script to your project.
- Create a test directory, e.g.
test
- Download the wrapper script
cd test && wget 'https://cdn.rawgit.com/kba/tsht/master/tsht'
- Create your unit tests
- Execute all tests using
./tsht
or specific tests using./tsht <path/to/unit-test.tsht>...
The first time you execute the wrapper script, it will clone this repository to
.tsht
and execute the runner. Whenever you want to update the tsht framework,
simply delete the .tsht
folder and an up-to-date version of the framework
will be cloned when you next run your tests.
To execute tsht scripts without the runner, you will need to have tsht
in
your $PATH
. You can either set $PATH
up manually to include the directory that contains the
tsht
wrapper or clone this repository and use the Makefile
.
To install system-wide:
sudo make install
To install to your home directory:
make PREFIX=$HOME/.local install
Tsht unit tests are written in a DSL superset of the Bash shell scripting language. This means that any bash code can be used in a tsht script.
All tsht scripts must end with .tsht
.
All tsht scripts should start with #!/usr/bin/env tsht
Tsht scripts are executed in alphabetic order, so prefix the scripts you want to run early with a low number.
Usage: tsht [options...] [<path/to/unit.tsht>...]
Options:
--help -h Show this help
--color Highlight passing/failing tests in green/red
--update Update the tsht framework from git
--version -V Show last revision of the runner
In addition to the core functionality, tsht can be extended with extensions. An extension is a subdirectory of tsht that has this structure:
/ext/<name>
ext/
└── <name>
├── Makefile
└── <name>.sh
The Makefile
must have an install
target that can install necessary binaries
into $(PREFIX)/bin
.
To use an extension, call the use
directive:
use 'jq'
This will call make install
in the extension directory and set the PATH
variable to let the extension use the locally installed software.
Currently, these extensions are available:
- jq (API, test): A wrapper around the jq CLI JSON query tool
- diff (API, test): Show differences with color-highlighted diff
- diff (API, test): Show differences with diff
- shxml (API, test) Use XML based tools in tests (XSD, XSD, XPATH…)
Some unit tests require setup work before they run and teardown work
after they run. To make this easier and reusable, these tasks can be
grouped in before
and after
hooks, which are shell scripts or
shell functions.
Hooks are always test-specific and are looked for in three places:
- Shell function
after
- Shell scripts named like the test with suffix
.before
/.after
- Shell scripts in the same directory as the test named
.before
/.after
Note: before
cannot be a shell function in the script because the script
must be sourced to declare it and once it has been sourced, it's too late to
call the before
hook.
#!/usr/bin/env tsht
plan 4
equals $(( 84 / 2 )) 42 "three score and six"
exec_ok "ls /"
match "oo" "foobar"
not_ok $(( 0 / 42 )) "Nothing divided is nothing"
If you are a vim user, try out the tsht.vim
plugin which will detect .tsht
files, highlight the builtin functions
and execute scripts with the closest wrapper.
There are various TAP consumers that can produce nice output, the tape NodeJS TAP-based framework lists a few.
For example, using the tap-spec TAP reporter can be installed using
npm install -g tap-spec
For the tests of tsht itself, it will produce output like this:
$ ./test/tsht | tap-spec
Testing ./runner/update/update.tsht ✔ Executed: git clone ../../../ /home/kb/build/github.com/kba/tsht/test/runner/update/test-project/.tsht ✔ Executed: git checkout master ✔ Executed: git reset --hard bd9fbafa643f10087cb24ff0f3b47a9d33a12a26 ✔ HEAD is bd9fbafa643f10087cb24ff0f3b47a9d33a12a26 ✔ Executed: ./tsht --update ✔ HEAD is 81ce4a381d416855eb37b99a087e8d01edae661c Testing ./runner/help/help.tsht ✔ Executed: /home/kb/build/github.com/kba/tsht/test/.tsht/tsht-runner.sh --help ✔ Executed: /home/kb/build/github.com/kba/tsht/test/.tsht/tsht-runner.sh -h ✔ -h == --help ✔ Matches '--color': 'Usage: tsht [options...] [...] Options: --help -h Show this help --color Highlight passing/failing tests in green/red --update ' ✔ Matches '--update': 'Usage: tsht [options...] [...] Options: --help -h Show this help --color Highlight passing/failing tests in green/red --update ' ✔ Matches '--version': 'Usage: tsht [options...] [...] Options: --help -h Show this help --color Highlight passing/failing tests in green/red --update ' ✔ Failed as expected (2) '/home/kb/build/github.com/kba/tsht/test/.tsht/tsht-runner.sh --foobar' Testing ./runner/color/color-test.tsht /home/kb/build/github.com/kba/tsht/test/.tsht/tsht-runner.sh --color thetest ✔ Color output as expected Testing ./runner/0tshtlib/tshtlib.tsht ✔ TSHTLIB is set ✔ TSHTLIB is relative to this dir ✔ /home/kb/build/github.com/kba/tsht/test/.tsht/tsht-runner.sh is the right tsht-runner.sh Testing ./file.tsht ✔ Failed as expected (2) 'ls does-not-exist' ✔ Executed: touch does-not-exist ✔ File exists: does-not-exist ✔ Not empty file: does-not-exist ✔ (unnamed equals assertion) Testing ./api/core/not_ok.tsht ✔ Empty string ✔ 0 ✔ "0" Testing ./api/core/exec_fail.tsht ✔ Failed as expected (2) 'ls does-not-exist' Testing ./api/core/ok.tsht ✔ Me testing my existence Testing ./api/core/exec_ok.tsht ✔ Executed: touch does-not-exist Testing ./api/file/file_exists.tsht ✔ File exists: does-not-exist Testing ./api/file/file_not_empty.tsht ✔ Not empty file: does-not-exist Testing ./api/string/match.tsht ✔ Matches '^\d+': '1234' ✔ Matches '^\d+$': '1234' ✔ Matches '^a\d+$': 'a1234' Testing ./api/string/equals.tsht ✔ (unnamed equals assertion) ✔ (unnamed equals assertion) ✔ (unnamed equals assertion) ✔ (unnamed equals assertion) Testing ./api/string/not_match.tsht ✔ Not like '^\d+$': 'string' Testing ./api/string/not_equals.tsht ✔ 1984 test Testing ./ext/shxml/shxml-basic.tsht ✔ Executed: shxml --help Testing ./ext/colordiff/diff.tsht ✔ A colordiff is a colordiff is a colordiff Testing ./ext/jq/jq.tsht ✔ Executed: jq --version ✔ From string ✔ From STDIN (1) ✔ From STDIN (2) ✔ JSON: .foo.bar[1] -> '42' ✔ Testing ./ext/diff/diff.tsht ✔ A diff is a diff is a diff Testing ./issues/issue_8.tsht ✔ Matches '--foo': '--foo' Testing ./before-after/ba2.tsht ✔ File does not exist: DELETEME Testing ./before-after/ba.tsht Sourcing suffix suffixscript ba.tsht.before for ba.tsht ✔ File exists: DELETEME Sourcing suffix suffixscript ba.tsht.after for ba.tsht total: 51 passing: 51 duration: 226ms
This library the core functions of tsht. It is always included and includes the most commonly used libraries:
Specify the number of planned assertions
plan <number-of-tests>
Fail unconditionally
fail <message> [<additional-output>]
The additional output will be prefixed with #
.
Succeed unconditionally.
See fail
Execute a command (or function) and succeed when its return code matches the parameter
exec_fail <expected-return> [<cmd-args>...]
Example
exec_fail 2 "ls" "-la" "DOES-NOT-EXIST"
Execute a command (or function) and succeed when it returns zero.
Example
exec_ok "ls" "-la"
Succeed if the first argument is a non-empty non-zero string
Succeed if the first argument is an empty string or zero.
Use an extension library
use 'jq'
Succeed if a file (or folder or symlink...) exists.
file_exists ".git"
Succeed if a file (or folder or symlink...) does not exist.
not_file_exists "temp"
ALIAS: file_not_empty
Succeed if a file exists and is a non-empty file.
Succeed if the first arguments match the contents of the file in the second argument.
Succeed if the contents of two files match, filenames passed as arguments.
This library contains functions testing strings and numbers
Test for equality of strings
equals <expected> <actual> [<message>]
Example:
equals "2" 2 "two equals two"
equals 2 "$(wc -l my-file)" "two lines in my-file"
Inverse of equals.
Succeed if a string matches a pattern
match "^\d+$" "1234" "Only numbers"
Succeed if a string does not match a pattern
not_match "^\d+$" "abcd" "Only numbers"
Extension that allows testing JSON strings.
Enable with
use jq
See jq
Github repo.
Test if jq
expression validates
Test if jq
expression is as exepected
Extension that replaces the builtin equals
with
a function that shows the difference as a unified diff
Enable with
use diff
Requires diff(1).
Test for equality of strings and output unified diff on fail.
equals <expected> <actual> [<message>]
Example:
equals "2" 2 "two equals two"
equals 2 "$(wc -l my-file)" "two lines in my-file"
Extension that replaces the builtin equals
with
a function that shows the difference in a colored diff output.
Enable with
use colordiff
Requires perl.
Test for equality of strings and output colored diff on fail.
equals <expected> <actual> [<message>]
Example:
equals "2" 2 "two equals two"
equals 2 "$(wc -l my-file)" "two lines in my-file"
Extension that makes shxml
available
Enable with
use shxml
See shxml
Github repo.