Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Teradata numeric overflow for count validation #207

Closed
oapa opened this issue Mar 9, 2021 · 2 comments
Closed

Teradata numeric overflow for count validation #207

oapa opened this issue Mar 9, 2021 · 2 comments
Assignees
Labels
priority: p1 High priority. Fix may be included in the next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@oapa
Copy link

oapa commented Mar 9, 2021

Encountered a Teradata numeric overflow error below when trying to run "count" validation on a table.  According to this post, it looks like count(*) can overflow on Teradata since it implicitly casts to INT.

pandas.io.sql.DatabaseError: Execution failed on sql 'SELECT count(*) AS "count", count("id") AS "count__id",
 ......
FROM {td_schema}.{td_table}
SAMPLE 10000': [Version 17.0.0.8] [Session 31075829] [Teradata Database] [Error 2616] Numeric overflow occurred during computation
@dhercher
Copy link
Collaborator

To summarize the issue which is occurring: Teradata implicitly casts any COUNT to INT. Running a normal count could be written more verbosely as
SELECT CAST(COUNT(*) AS INT) AS "count" ...

The overflow does not occur during the count, but rather during the cast which implicitly happens. There are Teradata configs which could override this behavior on a per installation basis (in TD15.10 there's a new dbscontrol flag to change the default to BIGINT or NUMBER).

We should likely automatically CAST all Teradata COUNT operations to BIGINT or NUMBER to avoid the error

@ryanmcdowell ryanmcdowell added type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. priority: p1 High priority. Fix may be included in the next release. labels Mar 22, 2021
@dhercher dhercher added the Beta label Mar 25, 2021
@freedomofnet freedomofnet added priority: p2 Medium priority. Fix may not be included in next release (e.g. minor documentation, cleanup) and removed Beta priority: p1 High priority. Fix may be included in the next release. labels Mar 25, 2021
@freedomofnet freedomofnet added Beta priority: p1 High priority. Fix may be included in the next release. and removed priority: p2 Medium priority. Fix may not be included in next release (e.g. minor documentation, cleanup) labels Jul 1, 2021
@nehanene15
Copy link
Collaborator

This is now possible with the --cast-to-bigint flag and via YAML. This will cast any int32 columns to int64 before running an aggregation. Closed with PR #461

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: p1 High priority. Fix may be included in the next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

5 participants