JSON Schemas: support null or not?

[thekaveman]

Is your feature request related to a problem? Please describe.

This originally came up in the context of creating JSON Schemas for the Policy endpoint in #389 (comment).

The question: for optional fields across MDS, should the spec allow for null values? Or should the spec make it clear that optional fields should be omitted from payloads where values are not being provided for those optional fields.

Describe the solution you'd like

From a JSON Schema perspective, it is far easier to say null is not allowed on optional fields, and that optional fields should just be omitted from a payload if no value is being given.

Is this a breaking change

• Yes, breaking

In the real world, we have seen a mix of usages (at least on the Provider side): some Providers send optional fields as null, and some omit those fields from the payloads. So whatever we decide will likely be breaking for someone.

Impacted Spec

• agency
• policy
• provider
• others in the future

Describe alternatives you've considered

Do nothing, leaving somewhat of a mismatch between the spec language and schemas (and even between schemas of different APIs).

Additional context

Here's an example from the current (dev) schemas that shows how we allow null for properties of the top-level link property that gives pagination information.

Notice we need to add an additional entry to the type field for properties that allow nulls (the first property below):

"links": {
  "$id": "#/definitions/links",
  "type": "object",
  "required": [
    "next"
  ],
  "properties": {
    "first": {
      "$id": "#/definitions/links/first",
      "type": [
        "null", // <--- added to type array to support nulls
        "string"
      ],
      "title": "The URL to the first page of data",
      "examples": [
        "https://data.provider.co/trips/first"
      ],
      "format": "uri"
    },
// more ...

This can make it difficult to reuse a common definition across specs, where some specs/endpoints require fields of that definition, and others do not. An example is the timestamp definition:

"timestamp": {
  "$id": "#/definitions/timestamp",
  "type": "number",
  "description": "Integer milliseconds since Unix epoch",
  "multipleOf": 1.0,
  "minimum": 0
},

In Agency and Provider, uses of this type are always required and can use this common definition directly. In Policy we have optional timestamp fields. To handle both of these cases, a special Policy-only timestamp definition would have to be created with the additional allowance of null:

"optional_timestamp": {
  "$id": "#/definitions/optional_timestamp",
  "type": [
    "null", // <--- added to type array to support nulls
    "number"
  ],
  "description": "Integer milliseconds since Unix epoch",
  "multipleOf": 1.0,
  "minimum": 0
},

[schnuerle]

We will be talking about this on the Working Group call tomorrow: https://github.com/openmobilityfoundation/mobility-data-specification/wiki/Web-conference-notes,-2020.10.01-(Joint-Working-Group-City-Services)

[thekaveman]

We talked about this on the 10/1 Working Group call.

There weren't any strong feelings one way or another. Since the schema generator has already been updated to support ad-hoc nullable fields, and since making a choice one way or the other would probably break somebody - we decided to table this issue.

Future readers: feel free to leave comments if you have strong feelings one way or another.