Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

validate row: Oracle RAW Columns Cause Exception #773

Open
nj1973 opened this issue Mar 14, 2023 · 2 comments
Open

validate row: Oracle RAW Columns Cause Exception #773

nj1973 opened this issue Mar 14, 2023 · 2 comments
Labels
priority: p2 Medium priority. Fix may not be included in next release (e.g. minor documentation, cleanup) type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@nj1973
Copy link
Contributor

nj1973 commented Mar 14, 2023

Oracle RAW columns hold binary data and are sometimes used to store UUID/GUID data. This is not a common pattern but certainly something I've seen in the past.

Below are two test tables, one with a GUID column and another with a GUID column as a primary key.

create table dvt_test.tab_guid (
  id number(2), col_guid raw(16)
);
ALTER TABLE dvt_test.tab_guid ADD PRIMARY KEY (id);
insert into dvt_test.tab_guid values (1,sys_guid());
insert into dvt_test.tab_guid values (2,sys_guid());
commit;

create table dvt_test.tab_guid_pk (
  id raw(16), col_dec number(5)
);
ALTER TABLE dvt_test.tab_guid_pk ADD PRIMARY KEY (id);
insert into dvt_test.tab_guid_pk values (sys_guid(),1);
insert into dvt_test.tab_guid_pk values (sys_guid(),2);
commit;

tab_guid command:

data-validation validate row -sc ora_local -tc ora_local \
-tbls=dvt_test.tab_guid --primary-keys id --concat '*' \
--filter-status fail
...
sqlalchemy.exc.DatabaseError: (cx_Oracle.DatabaseError) ORA-00904: "ENCODE": invalid identifier

tab_guid_pk command:

data-validation validate row -sc ora_local -tc ora_local \
-tbls=dvt_test.tab_guid_pk --primary-keys id --concat '*' \
--filter-status fail
...
  File "pandas/_libs/lib.pyx", line 2859, in pandas._libs.lib.map_infer
  File "/usr/local/google/home/neiljohnson/github/professional-services-data-validator/.venv/lib/python3.10/site-packages/ibis/backends/pandas/execution/strings.py", line 286, in replacer
    return pattern.sub(replacement, x)
TypeError: cannot use a string pattern on a bytes-like object

I haven't tested PostgreSQL but that has a dedicated UUID data type. When working on this we should do the equivalent PostgreSQL tests as above.

@nj1973 nj1973 added type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. priority: p2 Medium priority. Fix may not be included in next release (e.g. minor documentation, cleanup) labels Mar 14, 2023
@nj1973
Copy link
Contributor Author

nj1973 commented Jul 11, 2023

Still fails after Ibis upgrade

@nj1973
Copy link
Contributor Author

nj1973 commented Sep 27, 2023

This is also true of Oracle BLOB columns.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: p2 Medium priority. Fix may not be included in next release (e.g. minor documentation, cleanup) type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

1 participant