GTFS-Fares v2 base implementation

[scmcca]

[Original message from 2021-10-14 below]

Update as of 2022-04-19:

As previously mentioned, MobilityData wants to drive this proposal and here is a summary of what has been done so far as well as the next steps.

• 2022-04-04 Vote did not pass (7 in favour, 2 against)
• 2022-04-14: MobilityData published a list of what has been identified as the in-scope open items and opened 2 issues to work on these items

These issues are listed below:

In scope items:

Below is the list of in-scope items that we will work on:

• Incorporate editorial changes (from @flocsy’s comment: points 7, 12, 26, 26 again, and 29)
• Clarify the wording in various files and fields (from @flocsy’s comment: points 8, 21 - 22, 25, and 27)
• Drive conversations on Primary Keys for fare_leg_rules.txt and fare_transfer_rules.txt (from @flocsy’s comment 10, 17)
• Discuss the uniqueness of fare_products.fare_product_id (from [@drewda](https://github.com/drewda)’s comment)

Timeline:

We are planning an accelerated timeline to address those items since we believe we are very close to consensus
Week of Apr 11:

• Open GitHub issues to discuss the uniqueness of fare_products.fare_product_id (here) and to discuss wording changes (here)

Week of Apr 18:

• Incorporate editorial changes
• Continue discussions on Primary Keys (here)

Week of Apr 25:

• Continue discussions on GitHub
• Host two meetings with interested stakeholders to resolve the open items (Apr 26 and Apr 27 at 11:00 AM EDT)

Week of May 2:

• Post a summary of the outcomes from the meetings
• Continue discussions on GitHub
• Finalize the files in the pull request
• If the community sees a need for more meetings, we are willing to host two calls for further discussions on this week

May 9:

• Open the vote

---

Update as of 2022-01-31:

Hello everyone,

As previously mentioned, MobilityData wants to drive this proposal and here is a summary of what has been done so far as well as the next steps.

• 2021-12-16: Vote did not pass (11 in favor, 5 against)
• 2021-12-22: MobilityData published a list of what has been identified as the in-scope open items.
• 2022-01-31: MobilityData opened 4 issues to work on the above-mentioned open items (to avoid overlap of conversation in the same thread).

These issues are listed below:

• Primary Keys for fare_leg_rules.txt and fare_transfer_rules.txt Primary Keys for fare_leg_rules.txt and fare_transfer_rules.txt #304: All fields in fare_leg_rules.txt and fare_transfer_rules.txt were initially marked as Primary Keys, this is not optimal as it results in high computational load for lookups. A group of stakeholders proposed a smaller subset of these fields to define Primary Keys.

• is_symmetrical in fare_leg_rules.txt and fare_transfer_rules.txt is_symmetrical in fare_leg_rules.txt and fare_transfer_rules.txt #305: Currently, two entries in fare_leg_rules.txt and in fare_transfer_rules.txt can be identical except for values in the is_symmetrical field. This leads to conflicting information on whether the fare is symmetrical or not.

• Improvements to fare_transfer_rules.txt Improvements to fare_transfer_rules.txt #306: The specification does not define how transfers from one fare_leg_group to another fare_leg_group if those leg groups are not listed as a pair in fare_transfer_rules.txt. Also, incorrect fare information might be displayed since fare_transfer_type is forbidden if the amount field is not defined.

• Changes to fare_transfer_rules.fare_transfer_type Changes to fare_transfer_rules.fare_transfer_type #307: Some stakeholders mentioned that the wording of the enumerations in fare_transfer_rules.fare_transfer_type is not very intuitive. Additionally, the current enumerations in fare_transfer_rules.fare_transfer_type may not cover all the industry use cases.

MobilityData is piloting a new process to drive this proposal and all your feedback is welcomed!

We plan to follow these steps:

• 2022-01-31: MobilityData will first open separate issues to focus the conversations on the outstanding in-scope items
• Week of 2022-01-31: Discussions with the community on GitHub
• Week of 2022-02-14: MobilityData will share a summary of these discussions
• Week of 2022-02-21: If the items are still unresolved, MobilityData will host up to 3 meetings with interested stakeholders to work those issues out and resolve them. Full details on the days and times of the meetings will be posted in each in-scope issue opened
• Week of 2022-02-28: Report back to the community on the outcome of the meetings
• Week of 2022-03-07: If resolved, open the vote
• Week of 2022-03-07: If still not resolved, MobilityData will facilitate another round of community discussions on the open items
• Week of 2022-03-14: Report back to the greater community on the outcome of the discussions
• Week of 2022-03-28: If resolved, open the vote
• Week of 2022-03-28: If still not resolved, MobilityData will shelve these items for the time being and inform the community. This leaves the door open for other stakeholders to pick up the work on adopting the base implementation of GTFS-Fares v2

---

### Original message posted by @scmcca on 2021-10-14:

Hi Everyone,

 we (MobilityData) wanted to draw some attention to the base implementation of GTFS-Fares v2.



As discussed in #252, GTFS-Fares v2 is a proposal that models simple and complex fare structures in GTFS where the current files fare_attributes.txt and fare_rules.txt lack. 







This PR presents the adoptable fields of GTFS-Fares v2, where at least 1 producer and 1 consumer have implementation. As documented here, Interline, Maryland DOT, and Cal-ITP (for 50+ agencies) have produced subsets of GTFS-Fares v2, with Transit app consuming.







Posted as separate commits to ease comprehension, the features in this PR include:

• Fare costs for individual legs of travel
• Fares for routes and networks of routes
• Fares for area location groups
• Detailed transfer rules between legs of travel

The complete suite of features in GTFS-Fares v2 include:

• Timeframes
• Distance-based fares
• Fare products (i.e., passes)
• Fare containers (i.e., magnetic cards)
• Rider categories
• Fare capping
• WIP: Fare point-of-sale and payment methods


The #gtfs-fares Slack channel has been a great resource and community for debugging, building documentation, and working through problems. I highly recommend anyone interested to join.

For other questions/concerns, don’t hesitate to reach out to [email protected].

Looking forward to feedback and contribution on this proposal. Thanks!

[flocsy]

I don't like that the amount + currency are moved to the rules. This makes no sense to me, IMHO it's a step back from fare_attributes. Until now basically 1 fare attribute represented (more or less) 1 ticket type. It also enabled us to add additional fields to the fare_attributes (i.e: ticket_image_url, so we can see how a bus ticket looks like as opposed to a tram ticket, or we could even have profiles, by adding :student_price, pensioner_price. Or in certain cases buses for tourists accept either local currency or USD or EUR but basically they have a price in these currencies (not moving with the currency rate)) All these features are gone if we want to move to the 2 new files (which either forbids the use of the old fare_attributes.txt or if we accept Paul's suggestion then it's not used any more if the consumer does read the new files)

What I suggest instead is to keep fare_attributes.txt (or if it's too complicated technically then add a new file) where we can "aggregate" these things, and instead of having (amount, currency) in each line of fare_leg_rules, fare_transfer_rules we could have fare_id.

[e-lo]

I just wanted to note that thanks to the great work by @mcplanner @drewda @jewel1965 and the enthusiasm to publish this data by many many transit providers in California....

Over 70 GTFS Datasets in CA now publish Fares v2 and we are on our way to having dozens more (maybe all?) by the end of the year.

We are looking forward to the full adoption of this PR so that we can move forward on firmed ground w.r.t. tooling and data publication.

[jewel1965]

Just wanted to give kudos to the Interline team for helping Metropolitan Transportation Commission of the San Francisco Bay Area to build the GTFS-Fares v2 as part of our Regional GTFS feed. The Regional GTFS feed has allowed us to create discounted fares for interagency transfers alongside Pathways data that facilitates navigation between services offered by multiple agencies at a transit station/hub. Thanks to Drew, Ian, and Ruth for their dedication to this work. The team will continue to deliver more exciting stuff as our work continues.

[scmcca]

@flocsy

I don't like that the amount + currency are moved to the rules. This makes no sense to me, IMHO it's a step back from fare_attributes. Until now basically 1 fare attribute represented (more or less) 1 ticket type.

The base of Fares v2 is built on defining cost for individual legs of travel (with exhaustive characteristics for zones, distance, networks, rider categories, etc) using fare_leg_rules.txt and processing detailed leg-to-leg transfer costs using fare_transfer_rules.txt.

This approach was taken to allow for granularity that was hard to model in the historical fare_attributes.txt and fare_rules.txt, which only serve relatively limited use cases indicated by the lack of detailed GTFS fare data in the world. As elaborated below, I'd love to see counter proposals to extend or rewrite the existing model (fare_attributes.txt, fare_rules.txt) with the granularity and use cases covered by Fares v2.

It also enabled us to add additional fields to the fare_attributes (i.e: ticket_image_url, so we can see how a bus ticket looks like as opposed to a tram ticket,

Something like this would be possible using fare_products.txt (not included in this PR). Fare products are essentially a way to define specific "passes" (i.e., monthly pass, 2-way pass) that can be used to travel legs in fare_leg_rules.txt. Meanwhile, fares defined in fare_leg_rules.txt with no .fare_product_id associated to them denote simple 1-way fare cost. In either case, you could certainly extend these files to show what the physical ticket looks like.

or we could even have profiles, by adding :student_price, pensioner_price.

Rider categories are possible in Fares v2 using rider_categories.txt to attribute "profiles" eligible for certain fares. This functionality was not included in this PR because of lack of adoption.

Or in certain cases buses for tourists accept either local currency or USD or EUR but basically they have a price in these currencies (not moving with the currency rate))

How do you handle multi-currency tickets? AFAIK this is not inherit in the official fare_attributes.txt file. A work-around in Fares v2 is to show the currency options by duplicating the fare in different currencies. If you have a more elegant way to represent multi-currency fares I'd be very interested to hear it!

All these features are gone if we want to move to the 2 new files (which either forbids the use of the old fare_attributes.txt or if we accept Paul's suggestion then it's not used any more if the consumer does read the new files)

I've changed all files to "optional" (details to be clarified). I think it would be ok to provide both "v1" and "v2", but the files do work independently of each other. It would be up to the consumer to choose which implementation they want to represent.

What I suggest instead is to keep fare_attributes.txt (or if it's too complicated technically then add a new file) where we can "aggregate" these things, and instead of having (amount, currency) in each line of fare_leg_rules, fare_transfer_rules we could have fare_id.

I'm interested in seeing a model built on fare_attributes.txt that handles the complexity proposed by the complete proposal of Fares v2 (legs of travel, transfer rules, and much more). If you have any specific ideas or counter-proposals of how this would look, I'm all for better models to describe fares. As you'll see from the document I linked here, fares can get pretty complex, and we're trying to cover a wide range on what agencies can model.

[e-lo]

Are there any other discussions or clarification questions? Or should this be staged to a vote?

[skinkie]

I still would like to know if the following separate cases are supported;

1. Entrance fare + distance fare, where a transfer (within N minutes) can skip the entrance fare.
2. Distance based reductions.

[scmcca]

@skinkie

1. Entrance fare + distance fare, where a transfer (within N minutes) can skip the entrance fare.

This scenario seems a bit nuanced. Can you please provide the link to the fare policy so I can do a thorough review? So far I understand the following:

1. The rider pays an entrance fare
2. The rider pays an additional X amount per Y distance interval
3. The rider transfers to another leg (within N minutes)
4. The entrance fare from step 1 is refunded
5. The rider finishes by paying the distance increments PLUS the transfer cost

Is this a correct interpretation?

2. Distance based reductions.

Distance based fares are done using fare_leg_rules.txt to assign prices to distance intervals. You can make it so that the farther the rider travels, the cheaper the cost is per distance:

min_distance	max_distance	distance_type	amount	currency
	3	0	1.00	USD
3	6	0	1.50	USD
6	9	0	1.75	USD

This is a very basic illustrative example. If you can provide it, I'd love to see the actual fare policy to work through the modelling scenarios in detail alongside all the features in the complete proposal to see how we can model it (http://bit.ly/gtfs-fares).

[flocsy]

@scmcca

rider categories in the proposal have 1 huge disadvantage: as opposed to any real world fare system I know of it is based on an assumption that different rider categories will have different rules. I don't know any place in the world like that. In other words: implementing the proposed rider categories would mean that if you have C rider categories and R rules, then you would need to copy each rule C times, and end up with C*R lines. This is unnecessary in my opinion as from what I have seen it's much easier: keep the R rules, and reference the A fare_attributes, and in each of the A fares you can have a different price for the C rider categories. Usually R >> A >> C and you end up with R+A lines (in the rules + attributes files) with an additional C columns in the attributes file.

The same is the problem with your workaround regarding the multiple currencies. Instead of having additional pairs of columns in fare_attributes (ie: amount1:3, currency1:EUR, amount2:4, currency2:USD) you would need to duplicate the rules.

Both of the above examples (by having the amount+currency in the rules) have 2 things I don't like: 1. duplicating the rules makes data unnecessarily bigger, 2. implementations become more complicated. How do you "compare the rules" to decide which one renders the cheapest price?

I'm interested in seeing a model built on fare_attributes.txt that handles the complexity proposed by the complete proposal of Fares v2 (legs of travel, transfer rules, and much more). If you have any specific ideas or counter-proposals of how this would look, I'm all for better models to describe fares. As you'll see from the document I linked here, fares can get pretty complex, and we're trying to cover a wide range on what agencies can model.

We do use a very complex (partially based on ideas from Fares v2 + additional fetures we needed) fare system. But we kept the "fare" as being an entity defined in 1 line of the fare_attributes.txt very close to the official definition. We "removed"* some fields that make no sense being there and moved them to our enhanced rules (payment_method, transfers, transfer_duration). On the other hand we added a new field: fare_type which is an enum: {Normal, TransferFee, AdditionalFee and we also have a special value: ContinuePreviousFare - not sure if it makes sense outside of our implementation though} And our fare calculator does it's thing, and filters out all the rules, and through them the fares as probably most of the other fare calculators do, but at the end we have the filtered list with multiple fares. So we don't just chose the cheapest fare, but we have 2 groups: {Normal, Continue} => chose the cheapest, and throw away the rest, {TransferFee, AdditionalFee} => add all these to the "wallet" (to the total price of the journey). Example of TransferFee: Sudney: you get a -$2 discount in certain cases of transfers, AdditionalFee: Sydney: if you board or alight at the airpor tyou pay an additional $14 fee. All our rules have a required fare_id that is a foreign key to fare_attributes.

The proposed changes in this PR make it very hard to import the new V2 files to our existing system because of this. The best I could do is to enumerate the different amounts in the 2 new rules files, and create a virtual fare_id for each. But then instead of fixing a not-too-good V1 fare system we created a V2 that might be a bit better here and there but lacks in other places.

*) removed means: that we still import these fields if there's an official GTFS for the fares, but when there isn't such then we build our own version that has the aforementioned fields not as a property of the fare but of the rule.

The problem with all the things we do in-house is that we're both the producer and the consumer so I'm not sure if everything will make sense to others (though I'm pretty sure that we do have some things that could easily benefit to many others and are easily implemented - at least in terms of adding new optional fields to existing files) or whether it constitutes the requirements that proposed changes should have at least 1 producer and 1 consumer :)

[flocsy]

@scmcca
I think a good example to what I think @skinkie referred to as "where a transfer (within N minutes) can skip the entrance fare" is in London, where there are certain "hubs" that are physically separated, so you have to exit the subway system to the street, maybe cross the street or walk a hundred meters and enter another subway entrance. If the time between you exited and then you entered is less than lets say 10 minutes then you're continuing your trip with no additional payment. If the time passed then you pay a new ticket. BTW for this we have ticket_boarding_validity_duration (kind of what fare_attributes.transfer_duration most probably was originally meant for - though it's not 100% clear) vs trip_gap_duration. We also differentiate between systems that check the tickets validity at the beginning of the leg or at the end (ie: usually it means that you start to count when you start the 1st leg where you use a specific ticket, and you "validate" it on each later leg when when you enter the vehicle. But in some cases your ticket should be valid until you alight. Or in other words if you have luck and finished the trip in less than X time you paid less than if you got stuck in the traffic and it took more than X time)

Your "interpretation" or example is probably correct for many agencies, but there are some more tricky ones that you couldn't do that way:
2b: rider pays per number of stops (as opposed to distance)
3b: see above
4: not sure what this entrance fare and refund means, but there are places in the USA where you can upgrade your ticket, is that what you meant?
5b: or maybe also some other extra/additional fee, like: airport fee

[omar-kabbani]

Hello everyone, we have addressed all the in-scope items from this message and it seems that we are set to reopen the vote. @scmcca as the advocate of this proposal can you please do that?

We are hoping to get this passed this time as fares is an important aspect of trip planning. Transit riders need accurate information on how much their trips will cost and what other pricing options are out there. This will enable them to make better decisions on how and when they travel to do what they need to do - whether it is work, school, shopping, healthcare, leisure, or anything else.

I also wanted to thank everyone who contributed to making the base implementation of GTFS-Fares v2 a robust extension. A special thanks to the data consumers and producers who have piloted the deployment of this extension, without that, we could not have been able to reach this point in development and open the vote.

On behalf of MobilityData, I am very excited to see how this will unfold!

[scmcca]

Hi everyone, with the proposal updated, and the conversations slowing down - we are opening the vote!

This vote is for the base implementation of GTFS-Fares v2 as outlined in this pull request.

There are at least three producers:

• Interline for SF Bay Area Regional (@drewda)
• Maryland DOT (@SteveGladstone)
• Cal-ITP for 70+ agencies (@e-lo)

And at least one consumer:

• Transit App (@gcamp)

Voting ends on 2022-05-16 at 23:59:59 UTC.

[jsteelz]

+1 Transit app.

Thanks again to everyone working to make sure this specification provides a stable base for displaying fare information. We look forward to seeing the benefits that this will bring to transit riders worldwide.

[drewda]

+1 from Interline

Thanks to the organizations and individuals involved in refining this PR. This will serve well as both a base core of functionality and as a foundation for additions to be formalized in future PRs.

[jewel1965]

+1 from MTC

Appreciate the engagement and hard work by everyone involved. I am looking forward to taking it beyond the base implementation.

[flocsy]

10, 17: Didn't we agree that fare_product_id is NOT part of the primary key (of fare_leg_rules.txt, fare_transfer_rules.txt) but we're allowing the same fare_product_id in multiple lines of fare_products.txt? Or did I misunderstand this?

26: transfer_count: "Non zero integer"? Like -3 ;) This is confusing IMHO. Maybe change it to "Positive integer or -1 or empty" (though it's ugly)...

[omar-kabbani]

@flocsy this is a bit too last minute since the files have been changed and the vote opened.

For points 10, 17 - this was the outcome of the discussion → Link. One of the reasons why fare_product_id is in the set of Primary Keys is that this is a more restricted condition - which can be relaxed in the future (the opposite is not true).

For point 26: The type is Non-zero integer, and the field definition outlines the Valid Options as:

Valid options are:
-1 - No limit.
1 or more - Defines how many transfers the transfer rule may span.

Hence a value of -3 is forbidden.

More discussions on fares are very important - and we encourage that at MobilityData - however, we are at a point where the industry needs standardized fares to enable transit riders to better plan their trips. With that in mind, I urge everyone to focus on what we have in the base implementation, and get it approved since it works. Afterwards, the community can continue conversations on how to improve and expand the specification.

[westontrillium]

+1 from Trillium

Thanks to everyone for your rigorous work in getting Fares v2 to this milestone.

We look forward to both the base implementation's continued refinement and the eventual integration of the extension's other components (we are particularly excited about the ability to apply fares to service_ids).

[davidlewis-ito]

+1 from Ito World

Like many here, we look forward to this forming a foundation for further enrichment of the capability of GTFS to represent real-world fares

[SteveGladstone]

+1 from Maryland Transit Administration. Still looking good and still excited at the prospect of riders finally having easy access to all this info!

[philip-cline]

+1 from IBI Group

Thanks for all the hard work and emphasis on community involvement in the amendment process. IBI Group is excited to see improvement to the modelling of fares in GTFS and the resulting benefits for customer information.

[matikin9]

+1 from LA Metro

Thanks for all the hard work everyone has put into this!

[evansiroky]

+1 from @cal-itp

Big thanks to everyone involved with this and notably to the Mobility Data team for all of their facilitation efforts.

[scmcca]

The voting period ended on 2022-05-16 at 23:59:59 UTC.

With 9 votes in favour and no votes against, the vote to extend GTFS with the base implementation of GTFS-Fares v2 passes! The votes came from:

• Transit App (@jsteelz)
• Interline (@drewda)
• Metropolitan Transportation Commission (MTC) (@jewel1965)
• Trillium (@westontrillium)
• Ito World (@davidlewis-ito)
• Maryland Transit Administration (@SteveGladstone)
• IBI Group (@philip-cline)
• LA Metro (@matikin9)
• Cal-ITP (@evansiroky)

It is great to see this pull request come to a close, and we are excited to see what is next for GTFS-Fares v2. We encourage you to continue the conversations on the future of this extension by opening GitHub issues as well as Pull Requests.

On our end, to keep momentum on the fares project, we are dedicating a workshop session at our International Mobility Data Summit in Montreal to discuss GTFS-Fares, and we look forward to seeing you there!