Fix locations in presense of nested trivia

[povik]

@MikePopoloski I want to check with you to: confirm there's a bug, and see about a preferred way of addressing it.

[povik]

This check fails, the returned location points to the start of the directive and doesn't count the nested trivia that precede it.

[MikePopoloski]

Hmm, I'm not sure if this is actually a bug or not. It's currently only used when printing the syntax tree in a way that tries to get back the original source text, in which case I think it doesn't actually matter. Are you trying to use it in a way where this matters?

[povik]

Yes, I am reconstructing the trivia locations in a way suggested by this comment

    // The vast majority of trivia lives alongside the token it's attached to, so if
    // you want to know its source location just walk backward from the parent location.
    // For certain cases though (the preprocessor inserted tokens) the trivia gets glued
    // together from different locations. In that case hasFullLocation will be true and
    // the union will point at a FullLocation structure.
    bool hasFullLocation = false;

With the location pointing to the start of the underlying directive syntax and not to the source text to which the trivia corresponds, I am not sure how to go about reconstructing the locations reliably. I guess you would need to run the syntax tree printer on trivia.syntax()->getFirstToken().trivia(), and then use the length of the returned text to offset the location (or something effectively the same without constructing the printed text).

Ideally, for the downstream use of slang I am working on, nested trivia wouldn't be possible and the trivia would be moved to the upmost token where it can live: that is, instead of there being any trivia in some_trivia.syntax()->getFirstToken().trivia(), the trivia would be moved up to belong to the token to which some_trivia corresponds. @MikePopoloski do you think such a change can be made consistently? Would you welcome it?

[MikePopoloski]

I think that would be fine; the existing code already assumes that if there is a trivia with an explicit location that all preceding trivia is implicitly relative to it, so I think it would just work. I'm not sure how annoying it would be to find all the places where this can occur, but it would at least be contained to the Preprocessor class. Feel free to attempt and let me know how it goes.

[povik]

OK, thanks

[MikePopoloski]

Closing this for now; feel free to reopen if you want to put up a fix for the test.