Teradata invalid strftime function - DATE colums failing

[nehanene15]

The Teradata implementation of strftime compiles FORMAT_DATE() which is an invalid function.

This needs to be changed to a CAST(date_column AS DATE FORMAT 'DD-MM-YYYY')

[nehanene15]

Error message when running a TD to BQ hash validation on a DATE column:

/usr/local/lib/python3.8/dist-packages/third_party/ibis/ibis_teradata/client.py:156: FutureWarning: Using short name for 'orient' is deprecated. Only the options: ('dict', list, 'series', 'split', 'records', 'index') will be used in a future version. Use one of the above to silence this warning.
  schema_list = schema_df.to_dict(orient="record")
Traceback (most recent call last):
  File "/usr/local/bin/data-validation", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/data_validation/__main__.py", line 600, in main
    validate(args)
  File "/usr/local/lib/python3.8/dist-packages/data_validation/__main__.py", line 576, in validate
    run(args)
  File "/usr/local/lib/python3.8/dist-packages/data_validation/__main__.py", line 532, in run
    config_managers = build_config_managers_from_args(args)
  File "/usr/local/lib/python3.8/dist-packages/data_validation/__main__.py", line 304, in build_config_managers_from_args
    config_manager = build_config_from_args(args, config_manager)
  File "/usr/local/lib/python3.8/dist-packages/data_validation/__main__.py", line 186, in build_config_from_args
    config_manager.append_calculated_fields(get_calculated_config(args, config_manager))
  File "/usr/local/lib/python3.8/dist-packages/data_validation/__main__.py", line 128, in get_calculated_config
    fields = config_manager.build_dependent_aliases("hash", col_list)
  File "/usr/local/lib/python3.8/dist-packages/data_validation/config_manager.py", line 822, in build_dependent_aliases
    col = self._apply_base_cast_overrides(column, col)
  File "/usr/local/lib/python3.8/dist-packages/data_validation/config_manager.py", line 741, in _apply_base_cast_overrides
    self._strftime_format(
  File "/usr/local/lib/python3.8/dist-packages/data_validation/config_manager.py", line 716, in _strftime_format
    if isinstance(client, clients.OracleClient):
TypeError: isinstance() arg 2 must be a type or tuple of types

[nj1973]

The above issue should be fixed by PR: #736

[nj1973]

But the problem in this issue still needs to be looked at separately.

[nj1973]

@nehanene15 I've made some changes the associated branch for this issue, including adding an integration test.
On the face of it, it appeared to be a simple task of duplicating the strftime code from either Oracle or PostgreSQL and fine tuning the format tokens. The previous code appeared to be a copy of the BigQuery code which is why it didn't work.
However I ran into trouble with the copied code, the column name was being turned into a string and I couldn't for the life of me figure out why.
So I've commited working code but it doesn't feel right so I wondered if you had someone else who could take a look? Perhaps they could backout my commit and correctly copy the strftime code?

[nehanene15]

@nj1973 Interesting... I can test this out now and try copying the code from scratch..

[nehanene15]

@nj1973 I had to make the same changes as you to get the code working.. I think its normal for it to look different compared to Oracle/Postgres since SQLAlchemy doesn't support Teradata so the functions/column name compilations look different