Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle TIMESTAMP data type in the backend #424

Closed
Tracked by #250
kgodey opened this issue Jul 18, 2021 · 6 comments · Fixed by #865
Closed
Tracked by #250

Handle TIMESTAMP data type in the backend #424

kgodey opened this issue Jul 18, 2021 · 6 comments · Fixed by #865
Assignees
Labels
type: enhancement New feature or request work: backend Related to Python, Django, and simple SQL

Comments

@kgodey
Copy link
Contributor

kgodey commented Jul 18, 2021

This issue is to ensure that Mathesar can handle the Postgres TIMESTAMP data type.

As part of this issue, we need to ensure that:

  • Users can use the API to change a column to TIMESTAMP(with time zone) if it's possible to do so.
  • Users can use the API to change a column to TIMESTAMP(without time zone) if it's possible to do so.
  • Users can set and change the precision of a given TIMESTAMP column via a type_options field in the API.
  • Automatic type inference during file import suggests TIMESTAMP when it makes sense to do so.

Additional Context

@kgodey kgodey added ready Ready for implementation type: enhancement New feature or request work: backend Related to Python, Django, and simple SQL work: database labels Jul 18, 2021
@kgodey kgodey added this to the 07. Initial Data Types milestone Jul 18, 2021
@kgodey kgodey assigned mathemancer and unassigned eito-fis Aug 31, 2021
@silentninja silentninja self-assigned this Nov 24, 2021
@silentninja
Copy link
Contributor

silentninja commented Dec 1, 2021

Timestamps data type cause a conflict with date type when inferring a date without time value as date allows lossy coercion, same with timestamp being able to accept a date without time. We can solve this problem by restricting only if the data contains exact details(date and time mandatory for timestamp).

There are two ways I can think of:

  1. Restrict inference to timestamps - When inferring, we don't consider date type, instead we always infer any date(with or without time) as a timestamp.
  2. Throw up an error when casting with inappropriate data - If data do not confer exactly to the data type(missing date or time for a timestamp or contains time for date type) we throw an error. We end up opinionated with this approach and restrict the user who would intentionally want to cast date to a timestamp.
  3. Add strict parameter to casting function - This extends upon the previous suggestion(Option 2) by adding a strict parameter. We enable strict casting only when inferring, but allow lossy cast when done manually.

@mathemancer @kgodey @dmos62 Would like your opinion on this issue.

@dmos62
Copy link
Contributor

dmos62 commented Dec 2, 2021

For context (probably just for me), Postgres date contains date (no time of day) information; Postgres time contains time of day (no date) information; Postgres timestamp contains date and time information. If we have date+time information, treating it as timestamp should be first priority. If we have only time or date information treating it as time and date respectively should be the priority. We can do inaccurate casting as well (date to timestamp), but not by default.

So this sounds like option 3.

A quick note on terminology, casting a timestamp from a date is not lossy, since information is not being lost. It's sort of the reverse where you're providing more information than contained in the date type (i.e. time of day information). Though, I guess if you're guessing the time of day, you could say that you're losing the knowledge that you didn't have time of day originally.

@silentninja
Copy link
Contributor

casting a timestamp from a date is not lossy,

By lossy I mean there is no way to get back the data once it has been coerced. For example

SELECT '1997-12-17 07:37:16-08'::DATE::TIMESTAMP;
1997-12-17 00:00:00.000000

The time data is lost upon downcasting to date and upcasting to a timestamp.

@mathemancer
Copy link
Contributor

mathemancer commented Dec 2, 2021

I've already been thinking along the lines of option (3) above. I think we're eventually going to need to have different casting functions based on whether we're trying to infer a type or not. For now, I suggest using a custom function for casting that disallows lossy conversion (see the one for casting to integer, for example). We already have an issue #402 involving setting up default casting behavior when appropriate. Perhaps that could be the first step to an "advanced mode" that would allow lossy casting.

@kgodey
Copy link
Contributor Author

kgodey commented Dec 3, 2021

Option 3 sounds fine to me. To confirm, this means that if we see both date and time information, we would infer it to be TIMESTAMP and only date information would be a DATE, correct?

@silentninja
Copy link
Contributor

@kgodey Yes, that's right.

@silentninja silentninja linked a pull request Dec 4, 2021 that will close this issue
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement New feature or request work: backend Related to Python, Django, and simple SQL
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

5 participants