Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

beta release #282

Merged
merged 292 commits into from
Sep 15, 2023
Merged

beta release #282

merged 292 commits into from
Sep 15, 2023

Conversation

MatthewSteen
Copy link
Member

No description provided.

MatthewSteen and others added 29 commits March 24, 2023 15:09
* Add validatw API and UI

* Fix API

* Move to drawer

* Alter get shapes endpoint

* Update validate endpoint to work with new shape endpoint

* Get shapes async

* update docker compose

* fix library test

* default validate() call to use manifest

* implement validation endpoint

* running black to reformat

* use environment variables from .env consistently throughout docker-compose setup

* set -ex in Dockerfile for transparency

* buildingmotif apiserver prioritizes DB_URI environment variable

* switch to list of libraries

* add test for nonexistent library ids

* Fix form group console errors

* make sure formControlName is present

* add validationresponse object for structured output

* print list of reasons in validation output

* add validator

* move bmotif api dockerfile + extras to the buildingmotif/api directory

* remove bind to local directories because it can complicate distribution of the containers w/o the source directory

* use typing to remove need to explicitly call JSON.parse

---------

Co-authored-by: Hannah Eslinger <[email protected]>
* add basic ingress guide (more to do) and more docs on Record type

* working on ingress docs

* repo-visualizer [skip actions]

* work on docs

* better comments?

* more ingress handler

* repo-visualizer [skip actions]

* rearrange guides/explanations to better fit documentation structure

* fix typos

* Merge in develop; switch name to match other guides

---------

Co-authored-by: repo-visualizer <[email protected]>
* Remove template search resolver

* Put template evaulation in Model details

* Update evaulate template endpoint to take in model

WIP

* Add back in line lost in rebase

* Address comments

* Edit strip
* initial shape builder

* respond to comments

* change how I handle qualified and universal shapes

* add docstrings to shape methods

* add shape_builder tests

* change sqlalchemy logging level to warn

* Add additional test cases that account for full URIs

Sometimes pyshacl doesn't interpret the full URIs using the namespace
prefixes. This *should* make the test more robust to those kinds of
implementation details

* proof against namespace prefixes by expanding URIs

* remember to place the namespace/prefix changes in all test cases

* add shacl prefix declarations in Brick-full

* add prefixes to library query

---------

Co-authored-by: Gabe Fierro <[email protected]>
* talk about adding triples to Models directly

* fix import

* Update docs/tutorials/model_creation.md

Co-authored-by: Matt Steen <[email protected]>

* Apply suggestions from code review

Co-authored-by: Matt Steen <[email protected]>

---------

Co-authored-by: Matt Steen <[email protected]>
* add more detail to error message

* Raise warning on partial template eval

Adds a new flag, warn_unused, to Template.evaluate; defaults to True.
This raises a warning if not all of the mandatory parameters are
provided to the Template when it is evaluated. Will raise warning
if optional arguments are required (when using the corresponding flag on
Template.evaluate)

* Add tests for raising warnings when missing params on Template.evaluate

* add a pytest marker for integration tests

* fix behavior around inlining dependencies with optional args; add tests

* add fixtures for tests

* Update buildingmotif/dataclasses/template.py

Co-authored-by: Tobias Shapinsky <[email protected]>

* fix another edge case in inlining dependencies with optional args; add tests

* Fix 223P dependencies in the templates

Due to how we have the dependencies structured, the mapsTo relationship
needs to refer to a connection point that lies in a dependency's
dependency. This makes that relationship explicit

* Split template dep mgmt into two phases

New problem is when we are testing the libraries for validity,
sometimes we want to give an explicit argument to the 'mapsto'
optional arg w/n a dependency. However, we currently can't give
a binding to a nested dependency. The result is that we end up with
multiple mapsto (because the 'optional' mapsto argument gets a newly
minted URI from template.fill()), which fails validation.

This commit splits template dependency management into 2 phases: an
initial phase which adds the dependency links and args to the DB; a
second phase which checks that the args/params between the template and
its dependencies are all valid and work as expected.

* add flag to force adding optional_args when calling fill()

* don't treat SHACL warnings as violations

* adjust notebook to require optional args for 223P; add test to be sure

* Update buildingmotif/database/table_connection.py

Co-authored-by: Matt Steen <[email protected]>

* Update buildingmotif/database/table_connection.py

Co-authored-by: Matt Steen <[email protected]>

* Update buildingmotif/utils.py

Co-authored-by: Matt Steen <[email protected]>

* Update buildingmotif/database/table_connection.py

Co-authored-by: Matt Steen <[email protected]>

* Update buildingmotif/database/table_connection.py

Co-authored-by: Matt Steen <[email protected]>

* Update buildingmotif/utils.py

Co-authored-by: Matt Steen <[email protected]>

* Update buildingmotif/utils.py

Co-authored-by: Matt Steen <[email protected]>

* update method names + docstrings

* Update buildingmotif/dataclasses/template.py

* Update buildingmotif/utils.py

* add check-dependencies to tests

---------

Co-authored-by: Tobias Shapinsky <[email protected]>
Co-authored-by: Matt Steen <[email protected]>
* Add updaet model by file

* Append files

* Fix spelling
By including the shape and components that failed in the GraphDiff
objects, it becomes possible to generate more useful user messages. I
will also need this information to be present for better interpretation
of GraphDiff objects for automatically repairing a Model
* Fix nondeterministic test failure by correctly calculating the root node
of validation graphs

* assert -> exception
* add generate_spreadsheet method to templates;

* add test for spreadsheet generation from templates

* Add csv_generation; adjust csv/template generation to emit buffer or
write to file

* Parameterize tests, add testing of inmem + file based

Move template generation tests to their own file

* setup tables during template generation test

* use fixtures properly to avoid shared state

* remove unneeded inline arg from generate_ functions
* add dump functionality from records ingress handler

* add cli subcommand for scanning bacnet network

* add bacnet cli test

* add load methods and tests

* make bacnet cli test end to end

* point assert to correct path

* add doc strings

* BACnet ingress free socket after discover

* expose ports in docker compose

* run bacnet tests within a docker container on the same virtual network as the virtual bacnet device

* fix indentation

* cleanup tests

* replace assert with warning

* fix test issues
* remove --check flags from pre-commit config

* update developer documentation

* clarify pre-commit documentation
* Make dependencies easier to manage for development:

- the "extra" dependencies were not actually marked as optional, so they
  were all being installed anyway. This make sure they are not installed
  unless the developer wants them to
- "psycopg2" can be difficult to install on some systems like MacOS due
  to needing to tell the installation process about the location of the
  libpq library. Now, the much easier "psycopg2-binary" package will be
  installed as part of the development dependencies. To use postgres in
  production, one will need to install buildingMOTIF with the "postgres"
  option/flag which will install the "psycopg2" library which is
  recommended over "psycopg2-binary" for production.

* add documentation on using postgres in development vs production

* Make sure ingress deps are included in 'dev' group

* Update docs/reference/developer_documentation.md

Co-authored-by: Tobias Shapinsky <[email protected]>

* Update developer_documentation.md

---------

Co-authored-by: Tobias Shapinsky <[email protected]>
* Rename /template/<id>/evaluate to /template/<id>/evaluate

* [WIP] Add Ingress to evalate UI

* use Namespace objects correctly to create URIs and ensure that the Namespace is consistent between the input/output

* enforce Namespace type in Graph Ingresses

Not enforcing Namespace type on ingress.graph() would result in the
evaluation silently coercing URIs to strings; this is covered with
an assertion.

Also fixing up some type annotations and spellings too

* fix CSV file for ingress test; now uses the same identifiers as the unit test

* fix return type for bad model id

* fix expected length of graph for unit test

* correctly use API; model_id is a URL param not in the body

* correct test to use 404 response code; access the correct attribute on failed test too

* Clean up

---------

Co-authored-by: Gabe Fierro <[email protected]>
* Remove configs.py

* Update docs/reference/cli_tool.md

Co-authored-by: Matt Steen <[email protected]>

* Fix linter

* Raise error when DB_URI is not set

* add note for windows users to dev docs

---------

Co-authored-by: Matt Steen <[email protected]>
* Group diffset

* Try to fix test

* Add mode Dict
* fix developer docs for building docs

* fixing model correction doc page

* add comment for filling in params

* Update docs/tutorials/model_correction.md

Co-authored-by: Tobias Shapinsky <[email protected]>

* removing tutorial-specific comments

* clarify comment

* fixup usage of validation API

---------

Co-authored-by: Tobias Shapinsky <[email protected]>
* Add docker docs

* Fix

* Fix localhost
* updating docs; adding update_manifest function

* add some helpful flags for importing dependencies in shape collections

* update tests to cover new validation behavior

* update docs for validation with manifest

* add missing fixture

* fix test

* remove extraneous imports from 223 (exposed by bmotif tests)

* skip 223P tests until gabe can fix them

* update notebook

* adjust verbosity of output

* simplify manifest handling
* First pass at bacnet scan documentation for BuildingMOTIF CLI

* Update docs/reference/cli_tool.md

Co-authored-by: Matt Steen <[email protected]>

* Update docs/reference/cli_tool.md

Co-authored-by: Matt Steen <[email protected]>

---------

Co-authored-by: Matt Steen <[email protected]>
* update jupyter-book version

* license year

* docstring fixes for docs

* readme

* readme

* minor doc updates

* configs
@MatthewSteen MatthewSteen merged commit 87b3862 into main Sep 15, 2023
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants