Skip to content

Commit

Permalink
Add draft of immutable policy clarifications/explanations
Browse files Browse the repository at this point in the history
  • Loading branch information
marie-x committed Mar 4, 2022
1 parent 2670581 commit f99c06f
Showing 1 changed file with 40 additions and 0 deletions.
40 changes: 40 additions & 0 deletions policy/ImmutablePolicy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Mobility Data Specification: Policy and Immutible Data

The MDS Policy specification contains a note as to how a given Policy is updated:

"Published policies, like geographies, should be treated as immutable data. Obsoleting or otherwise changing a policy is accomplished by publishing a new policy with a field named `prev_policies`, a list of UUID references to the policy or policies superseded by the new policy."

This raises a number of questions about the practical implications of using this immutable data structure.

**Q:** Why are Policy objects immutable? <br/>

This comment has been minimized.

Copy link
@jiffyclub

jiffyclub Mar 7, 2022

Contributor

This is kinda outside the scope of this document, but it seems like we don't currently have any way of knowing when a policy (especially one that has been superseded) was current. For example, a policy with a start date of 2022-01-01 and an end date of 2022-12-31 could be replaced on 2022-05-26 with another policy with the same start/end dates and some adjusted fees. Someone looking at the record of policies in August 2022 wouldn't be able to tell that in February 2022 the policy of record was the first policy, not the one that replaced it in May. When I implement this internally I keep track of those other dates, when each policy was created and when a policy is superseded by another, but the policy spec doesn't have a way to communicate that.

This comment has been minimized.

Copy link
@marie-x

marie-x Mar 8, 2022

Author Collaborator

What you're proposing violates the rule that the effective_date must be greater than the published_date. Any schema validation will flag the 2022-05-26 rules as invalid. You cannot rewrite history, that's the whole point.

This comment has been minimized.

Copy link
@jiffyclub

jiffyclub Mar 8, 2022

Contributor

How would you correct a policy that was published with inaccurate information?

This comment has been minimized.

Copy link
@marie-x

marie-x Mar 8, 2022

Author Collaborator

You can only correct it from the moment that you correct it, not in the past. I suppose if you wanted to violate the schema and try to change history, you could go out-of-band and send an email or something.

This comment has been minimized.

Copy link
@jiffyclub

jiffyclub Mar 8, 2022

Contributor

I think this is a good candidate for a larger community discussion, I think it's important to be able to retroactively correct published policies. In the real world mistakes are going to be made in published policies.

This comment has been minimized.

Copy link
@jean-populus

jean-populus via email Mar 8, 2022

Collaborator

This comment has been minimized.

Copy link
@marie-x

marie-x Mar 8, 2022

Author Collaborator

Are you saying they retroactively change the fees?

This comment has been minimized.

Copy link
@jean-populus

jean-populus via email Mar 8, 2022

Collaborator
**A:** Policies may be used by cities for enforcement actions such as fees, fines, and permit suspensions. An unchangable history of all published policies makes this possible. Updating a Policy on the fly would potentially be re-writing history.

**Q:** But you left `end_date` as nullable. Wouldn't that make some Policy objects in effect forever? <br/>
**A:** No, a Policy object can be superceded by a subsequent Policy object.

This comment has been minimized.

Copy link
@jiffyclub

jiffyclub Mar 7, 2022

Contributor

How does one tell if a policy has been superseded?

This comment has been minimized.

Copy link
@marie-x

marie-x Mar 8, 2022

Author Collaborator

You scan all the policies. While the policy objects are immutable, you can store whatever computed metadata you want. It's simple to compute a "has this policy been superseded" flag in your local storage of the policies.

This comment has been minimized.

Copy link
@jiffyclub

jiffyclub Mar 8, 2022

Contributor

Yes I do that internally, I'm not talking about that. I'm asking about how to do this with the public info.

This comment has been minimized.

Copy link
@marie-x

marie-x Mar 8, 2022

Author Collaborator

I'd write a little script to add whatever derived data is needed locally. I've seen this done in javascript, python, and even SQL.


**Q:** How do I end a Policy with a null `end_date`, or end a Policy before its `end_date` if needed? <br/>
**A:** Publish a new Policy with a `prev_policies` list that includes the terminated Policy's `policy_id`.

**Q:** Does publishing a supercding policy immediately terminate the old policy? <br/>
**A:** Not necessarily; the new policy becomes effective as of its `start_date`, effectively overwritng the `end_date` of the old policy. That start date can be in the future, and in practice, *should* be in the future to give the consumers a chance to plan.

This comment has been minimized.

Copy link
@jiffyclub

jiffyclub Mar 7, 2022

Contributor

This runs counter to my understanding, I thought publishing a superseding policy always invalidated the old one. This is important to be clear on in the event the policy versions have overlapping start and end dates. I would prefer that any policy listed in the prev_policies list of another policy be considered invalid. Or we need to be very clear that policies cannot have overlapping dates of effect, or have some other metadata that indicates a policy is no longer in effect.

This comment has been minimized.

Copy link
@marie-x

marie-x Mar 8, 2022

Author Collaborator

This should have been spelled out in more detail in the spec. A policy is not invalidated until the superseding policy becomes effective (i.e. the moment of the start_date).

A policy and its superseding policy cannot overlap, but you can early-terminate policy A with policy B.

This comment has been minimized.

Copy link
@jiffyclub

jiffyclub Mar 8, 2022

Contributor

To make sure I understand this last part, it is possible for policies to have overlapping effective dates (e.g. if you're editing a policy with a null end date you have no other option), but when this happens the interpretation is that the superseded policy was in effect from its start date until the start date of the superseding policy.

This comment has been minimized.

Copy link
@marie-x

marie-x Mar 8, 2022

Author Collaborator

Yes.


**Q:** How do I terminate a policy without updaating it? <br/>
**A:** Publish a superceding policy (placing the prior policy's `policy_id` in the `prev_policies` field) with an empty list of Rules with an `end_date` of `null`.

This comment has been minimized.

Copy link
@jiffyclub

jiffyclub Mar 7, 2022

Contributor

Related to above, this makes it sound like listing a policy in the prev_policy field renders the policy terminated.


**Q:** If I publish a superceding empty policy with a non-null `end_date`, would that simply suppress the superceded policy until the new policy's `end_date`, and then the original policy would become effective again? <br/>
**A:** In the interest of simplicity, the official stance of the OMF at present is 'no.' A superceded Policy cannot be brought back to life.

**Q:** How much lead-time is required for obsoleting or updating a Policy? <br/>
**A:** This is outside of the spec and should be addressed in an SLA. Generally cities and other regulatory agencies should allow ample lead-time whenever possible, as fully-automated consumption of MDS Policy has yet to be implemented by any Provider to the best of our knowledge. OTOH one could imagine emergency situations such as a natural disaster where a Policy might need to be updated as soon as humanly possible.

**Q:** Why not make just `end_date` immutable? <br/>
**A**: Picking-and-choosing which fields are immutable seems perilous, and this document should make it easier to understand the way to reason about immutable policy data without resorting to short-cuts.

**Q:** Have you thought about digitially signing Policy objects to make sure that they haven't changed, and that the publisher is verifyable?
**A:** It has come up in discussions, but thus far no action has been taken. No standard for digitally signing JSON has been established.

**Q**: What if I accidentally terminate a Policy? How do I un-terminate it? <br/>
**A**: Re-publish the original Policy with a different `policy_id` and a `prev_policy` list containing the terminating (empty) Policy if you want to make sure the connection to the original is explicit, otherwise just publish a new policy and don't sweat the `prev_policy` field.

**Q**: What if all Policy objects were on a blockchain? <br/>
**A**: No.

6 comments on commit f99c06f

@jiffyclub
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this document together! It's definitely needed and similar would probably be helpful for other parts of MDS as well.

@marie-x
Copy link
Collaborator Author

@marie-x marie-x commented on f99c06f Mar 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your feedback @jiffyclub !

@sanjivnanda
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marie-x Is there a start_date and an effective_date? I see effective_date mentioned only once in the discussion above.

@marie-x
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, yes, there is only start_date. I'll fix that.

@schnuerle
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI I moved this over to the OMF Governance repo and made a pull request, so please leave future general and inline comments there:

openmobilityfoundation/governance#11

@marie-x
Copy link
Collaborator Author

@marie-x marie-x commented on f99c06f Mar 17, 2022 via email

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please sign in to comment.