DVT is failing with a "NotImplementedError: Oracle type <cx_Oracle.DbType DB_TYPE_LONG_RAW> is not supported" (Oracle to AlloyDB)

[violetautumn]

I'm running DVT with Oracle source system and AlloyDB (postgres) destination system. For a few of the tables, DVT is not running and giving the error NotImplementedError: Oracle type <cx_Oracle.DbType DB_TYPE_LONG_RAW> is not supported. I have confirmed that table exists in both the source and destination databases. The paths are also correct because DVT is running well for most of the other tables in the schema.

The table contains a string column and a column which contains Long Raw (oracle)/Binary data (alloydb). As per the the PostgreSQL documentation, DB_TYPE_LONG_RAW seems to be related to Binary data.

Can you please help with the issue and give a direction as to what DVT requires in order to handle raw/binary data type?

Table schema (oracle)

Table schema (alloydb)

Snapshot of the error log when running the DVT command

[helensilva14]

Hi @violetautumn, thank you for all the details you provided, it helps us a lot to address your problem!

@nj1973 could you please also take a look at this? I'll check this datatype support on Ibis versions.

[helensilva14]

@violetautumn Hi, we opened a PR to fix this issue, but could you please test this changes with your data to confirm our current approach works?

You'll need to clone our repo, checkout to this specific branch issue1904-oracle-dbtype-longraw-not-supported, create a virtual Python env if you don't have one already, activate it and then execute pip install .

Let us know your outcomes, thank you!

[violetautumn]

Hello @helensilva14, sure, I'll try it out and let you know, thanks for the fix!

[violetautumn]

Hello @helensilva14, the fix works 🙌 Just tested the tables with DB_TYPE_LONG_RAW for schema_validation and it works.

Thanks @helensilva14 and @nj1973 for the quick fix, this helps a lot!

[helensilva14]

That's great news, thank you for let us know! I'll merge the PR so you can use it on the develop branch as the feature branch is going to be deleted by the merge. Soon we'll be releasing a new DVT version on PyPi!