Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Add support for numeric and precision with length and precision in Postgres Custom Query #723

Merged
merged 9 commits into from
Feb 17, 2023

Conversation

sharangagarwal
Copy link
Contributor

@sharangagarwal sharangagarwal commented Feb 15, 2023

Closes #681

@sharangagarwal
Copy link
Contributor Author

/gcbrun

@sharangagarwal
Copy link
Contributor Author

/gcbrun

@sharangagarwal sharangagarwal self-assigned this Feb 15, 2023
@sharangagarwal
Copy link
Contributor Author

/gcbrun

@sharangagarwal sharangagarwal marked this pull request as ready for review February 15, 2023 08:11
Copy link
Contributor

@nj1973 nj1973 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tested your code with non-numeric columns? I tried it with a date and got an error:

  File "/usr/local/google/home/neiljohnson/github/professional-services-data-validator2/third_party/ibis/ibis_postgres/client.py", line 107, in _parse_numeric
    return ty.parse(text)
  File "/usr/local/google/home/neiljohnson/github/professional-services-data-validator2/.venv/lib/python3.10/site-packages/parsy/__init__.py", line 87, in parse
    (result, _) = (self << eof).parse_partial(stream)
  File "/usr/local/google/home/neiljohnson/github/professional-services-data-validator2/.venv/lib/python3.10/site-packages/parsy/__init__.py", line 101, in parse_partial
    raise ParseError(result.expected, stream, result.furthest)
parsy.ParseError: expected one of 'decimal', 'numeric' at 0:0

I think you should add a unit test feeding each common data type string into the _get_type function to be sure we don't get any exceptions.

third_party/ibis/ibis_postgres/client.py Show resolved Hide resolved
third_party/ibis/ibis_postgres/client.py Outdated Show resolved Hide resolved
Copy link
Contributor

@nj1973 nj1973 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test column triggering the exception is actually timestamp rather than date.

col_ts          timestamp(6)

And looks like this from a describe:

    Column    |              Type              | Collation | Nullable | Default
--------------+--------------------------------+-----------+----------+---------
 id           | integer                        |           | not null |
...
 col_ts       | timestamp(6) without time zone |           |          |

@sharangagarwal
Copy link
Contributor Author

/gcbrun

Copy link
Contributor

@nj1973 nj1973 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

third_party/ibis/ibis_postgres/client.py Show resolved Hide resolved
@sharangagarwal sharangagarwal merged commit 742b77e into develop Feb 17, 2023
@sharangagarwal sharangagarwal deleted the issue681v2 branch February 17, 2023 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

validate custom-query: Exception when PostgreSQL column data type has length or precision
2 participants