Skip to content

Latest commit

 

History

History
344 lines (309 loc) · 12.8 KB

todo.md

File metadata and controls

344 lines (309 loc) · 12.8 KB
Raw

Release roadmap

  • Broker verification - read and publish result
  • Initial docs
  • Query string, probably
  • Refactor to make Jest DSL extractable
  • Note about stability
  • Spike translation to C# etc
  • Determine how package structure will work
  • Extract Jest
  • Release Java
    • DSL call core
    • DSL start server
    • Example client define
    • Example client verify
    • Example server define
    • Example server verify
  • Broker docs
  • Broker configurability in CLI
    • Selectors on download
    • Publish selectors
  • Auto docs on Matchers
  • Auto docs on Mock types
  • Fix bug where it's not finding the CI token
  • Release python
  • Release C# ?

Todo list

Next:

  • Fix Broker download contracts

  • Do upload for verification results

  • Spike JSii - support list

    • Errors from JS to host
    • Errors from host's provided functions
    • Passing values between host's functions
    • Standard out
    • core calls a host callback that calls back to core
    • Arbitrary JSON
    • Arbitrary JSON with Case Matchers

  • Spike monorepo

  • Flatten types at the boundary

    • Classes for matchers
    • Classes for mocks
    • Types for contract definition
      • Types for Triggers
        • Implement triggerAndVerify
      • Types for State Handlers
      • Pass in core printer to Logger
      • Types for Logger
      • Types for Result Printer
    • Types for extensions
      • Split result printer out of exposed context
      • Refactor context to separate functions and data to make it easier to expose
    • Types for contract verification
    • Figure out types for the triggers
    • Figure out types for the check functions
    • Jest DSL
    • Error handling boundary
    • Crash messages
    • Case Matcher functions so that you don't have to import JSii
    • Java DSL
    • Set up maven publish
    • Error handling boundary
    • Crash messages
    • Case Matcher function so that you don't have to import JSii
    • Trigger functions should take config

  • Build mechanism to ensure that mocks have the right number of request / response present

Documentation todos

  • case-context ? Maybe move this out
  • case-contract format ? maybe move this out too
  • Extending case
  • Vs e2e
  • Vs schema
  • Vs pact
  • API coverage
  • Confirm understood client
  • Ways to write contracts easily
  • Where to mock
  • Write specific responses
  • Brokers
  • Example types
  • Matchers: Auto-generate matcher documentation
  • Verifying contracts
  • Deployment checks
  • Documentation for config object
  • Documentation for states
  • Documentation for verification
  • Remove "advanced topics" but keep content
  • Remove FAQ but keep content
  • Differences from Pact
  • -- Pass-through APIs
  • -- Mock model
  • -- How to do proxy testing
  • -- Extensibility
  • How to use
  • How states work
  • Why there's no optional
  • How log levels work
  • Maintainer documentation
  • Correct coverage information
  • Package versioning strategy

After grpc

  • Remove case-boundary types from java wrapper
  • Write up / modify implementation instructions
  • Remove case-boundary from case-connector (refactor so that ts dsl uses the same as the grpc?)
  • It's possible that mustResolveToString and mustResolveToNumber have been used in places where we meant shouldResolveToString - review uses, and consider introducing a version that doesn't throw CaseCoreError

Notes from refactor:

  • Add a way to configure individual mocks
  • Rename all the repeated types so they're not confusing
  • Rename case:matcher to _case:matcher
  • Rename all case:matcher:matcher to case:matcher:child
    • But not the ones where it's not a child
  • Make ending of matcher interfaces consistent
    • Some are:
      • Matches
      • Matcher
      • Match
      • Matchers / Children
  • Make list of all resolvesTo
    • string
    • number
    • null
    • boolean
    • should there be "array" ?
  • Properly use accepts or some other property to use resolvesTo
  • Validators need to move
    • Array length (including checking that it's sensible)
    • Array length sensible
    • Array contains with an empty array
    • Array starts with with an empty array
    • String prefix with an empty string?
  • Remove MatchContextByType from core
  • Make it an error to not have broker tokens or URLs when trying to publish

Connector

  • Make loggers for the boundaries & glue
  • Extract controller from grpc, maybe
  • Confirm if the state handlers should have config / params?
  • Move the links to the promise handler out of the mappers
  • Remove the unnecessary handles from the grpc
  • Do proper stack trace passing, as a first class object
  • Tie the proto in the java to the proto in the typescript
  • Start the server from java
  • Remove JSii from boundary
  • Redo boundary interface types now that we don't need JSii
  • Add back in validators for constants (probably contract tests)
  • Expose teardown correctly on boundary
  • Wrap connector errors so that the core can't explode the server

Java

  • Make junit wrapper

Pact Parity

  • JVM / Python / Go / C# bindings
  • Strip matchers
  • Logging
  • Random port startup
  • Convenience matchers
    • Primitive matchers
    • Cascading array and object matchers
    • shapedLike matchers
    • Array each like
    • Object keys
    • Object values only
    • String includes
    • Array contains
    • Integer
    • Timestamp with format (covers time, date and timestamp)
  • Matchers we don't need, but will need to build if we want to read pact files
    • "Treat this as a string"
    • Decimal "with decimal places" (what does this mean? Probably covered by 'number')
    • Semver - limited use case, but neat
    • NotEmpty - why would this be needed?
    • Regex - dangerous as it allows optionals. Maybe a subset?
    • Probably out of scope: Binary content type with magic file check
  • Provider state
    • Variables from provider state (Implementation detail: via resolved matchers)
  • Write contract file
    • Print errors nicely
    • Config for location
  • Broker compatibility
    • Publish contract
    • Get contract from broker
    • Publish verification result
    • Publish branch in verification (PUT /pacticipants//branches//versions/)
    • Choose version selectors for upload
    • Choose version selectors for download
    • Make download actually a nice CLI
  • Read contract file
    • Read all files in a location?
    • Verify contract file
    • Error handling when the files don't exist
  • Http Headers
  • Query strings
  • Crash messages
  • Helpers for setting up tests
  • Timeouts

Major features

  • Add context to start and end contract
  • Test harness integration
  • Provider defined examples
  • Authentication sources (authentication as a first class citizen)
  • SNS and SQS support
  • grpc support

Later features

  • Save a new matcher directly to contract, without a test (so that it can be used in tests)
  • Text bodies
  • XML Matchers
  • Http forms matchers
  • graphql support
  • Kafka support
  • URL matchers

Cool features

  • And matcher
  • Resolved values (eg state)
  • Automatic names for interactions
  • Better automatic names eg "a GET request to /health"
  • String matcher helpers
    • Prefix
    • Suffix
    • Pivot
    • Split
  • Matcher content verification
  • Matcher argument verification
  • Options + config
    • Options and config from environment variables
    • Options and config from .rc files
  • Logging matchers
  • JS Matchers - Loading plugins from a JS module
  • Matcher version compatibility
  • Pact compatibility (ability to read Pact files)
  • Call-out matchers that call out to another service
  • Ignore matcher ("pretend this isn't in the response for the following matcher")
  • Run as a server so that arbitrary frameworks can use it
  • "Is this interaction the same as that"?
  • Combine files for upload (does the broker support multiple contracts between the same two services?)
  • Parametrised matchers
  • Ability to change the default version strategy

Contract file access patterns

  • Enumerate interactions
  • List states ?
  • Join contract files ?
  • Union contract files
  • Intersect contract files

Internals

  • Http produce request body matcher
  • Http produce request path matcher
  • Http consume request verification matcher
  • Http response status matcher
  • Test context
  • Matcher acceptance
  • Location aware errors
  • Reference (named/lookup) interactions and matchers
  • Print objects properly
  • Errors that don't have associated matchers
  • Move interactions to matchers
  • Fail tests if there are configuration errors inside the handler
  • Tidy up strip matcher stuff
  • State setups should be available in the config
  • Make sure all errors include context
  • Do state handlers in order they're defined instead of all at once
  • Fix global import issue - probably post process the dist directory. Or rewrite all the imports :/
  • Indentation on test output
  • Release plan that includes version number - can we just import this like we did in pact? It would be nice if it was accurate.
  • Do I need runcontext:tree? Maybe I can remove it.
  • Review the invert contract logic, possibly this can be removed too - Update, it could
  • Make loggable context explicit instead of calculated - or, calculate it better.
  • Fix issue with multiple values for the same variable
  • Fix issue where a failed trigger results in open sockets
  • Sort out the LoggableContext / LogContext naming mess
  • Remove logLevel setting and do it with context instead, allowing localised logging
  • Do we need to give the and matcher an optional example?
  • Examples might have matchers in them, need to strip those too
  • Can / should we give every matcher an optional example?
  • Make dependency object instead of passing each dependency separately
  • Make it less likely that a key is missed during the location printing (maybe make the maintainer-only location strings an explicit type instead of implied by :)
  • Review test harness and separate into a dedicated module
  • Move the error reporting logic closer to the boundary so that it's not possible to miss an example
  • Ability to tell when the main run is taking place from context (maybe don't log during the sanity run?)
  • Make the triggerAndTests map have a more unique key (not just ::, and detect if it has been printed twice)

Configuration and ergonomics

  • Log levels
  • File locations
  • Silent mode (no logs or test output)
  • Write docs for initial setup
  • Check that contract is actually downloaded before trying to report results (etc)
  • Named shaped object types (so eg headers doesn't say 'an object shaped like')
  • Better formatting of locations (strip unnecessary stuff unless in maintainer debug mode)
  • Strip matchers should take states
  • Nice support for named examples in request / responses
  • Warn when a logLevel matcher is written to contract
  • Error types marshalled at boundary
  • Introduce error codes for different classes of error
    • Write documentation for every single CaseConfigurationError
  • Annotate code generally with maintainer debug info
  • Make raw errors on matcher save print a diff
  • Make it an error to specify a different consumerName in runExample and defineContract
  • Warn when an autogenerated name is used directly in a test
  • Debug logs should include state variables
  • Error collapsers, so if the same spot produces the same errors for different reasons it will only be reported once.
  • Better system for stripping matchers with and() - maybe { items, keys, structure }?
  • Accepts / resolvesTo sanity checks

Potential issues:

  • Double handling of lookup interactions if the same data is traversed (eg, by checkMatch and stripMatch)
  • Need to allow multiple values for variables
  • Dangling promises when 'No verifier or errorVerifier provided' CaseConfigurationError happens
  • Make teardowns run for successful state setup functions when one of the setups (or other teardowns) fails.
  • Worker process still running when the triggers aren't found
  • Server shutdown is slow when run via grpc - maybe we can assume shutdown is clean and not await it?

Implementation notebook

Here I write notes to myself to make sure I don't miss anything

  • Check error types that case can actually emit - do all of them make it to the boundary? Maybe we can remove some from ResultKindConstants