Simpler OCaml language grammar

[mnxn]

As mentioned in #91, I have been working on a simplified version of the OCaml language grammar used for syntax highlighting.

As of now, the existing language grammar has various inconsistencies and issues that make it hard to work with. My implementation uses less context-sensitive highlighting and is much simpler and shorter because of it (500 vs 2000 lines). I have also added comments describing each rule, using terminology from the OCaml manual, which will hopefully make it easier to maintain in the future.

I have followed the official OCaml manual as close as I can, so special syntax is highlighted, including:

• escaped character sequences in characters/strings
  • standard sequences (\\, \n, \t, etc.)
  • decimal/hexadecimal/octal ASCII codes
  • unicode escape sequence (in strings only)
• printf format strings (https://caml.inria.fr/pub/docs/manual-ocaml/libref/Printf.html)
  • including flags, width, precision, and type
• floating point literals
  • decimal with exponent
  • hexadecimal with exponent part
• integer literals (int/int32/int64/nativeint)
  • decimal
  • hexadecimal
  • octal
  • binary
• operators
  • infix symbols
  • prefix symbols
  • monadic let operators (let+, and+, etc.)

A limitation of the approach I took is that capital identifiers are all highlighted the same (constructors, exceptions, modules); type parameters ('a) and builtin types are the only highlighted types.

I also added a few context sensitive highlighting (see the bindings patterns) that seem to work consistently, unlike the existing grammar.

My intention for this syntax is that it will be a base for incremental improvements in the future. Due to the inherit complexities of the official OCaml grammar, it may be worthwhile to look at tree-sitter or LSP semantic highlighting (proposal: microsoft/vscode-languageserver-node#367) in the future.

Comparison:

Old	New

For an example of two existing problems that this fixes, I provide the following examples.
For the let operators in #45:

Old	New

For the Menhir highlighting problem I mentioned in #91:

Old	New

I know this is a lengthy PR, but hopefully I communicated my justifications clearly. If you have any questions, I'll be happy to answer.

[smorimoto]

The code looks good. I have the same thought as you, and the existing code was kind of a placeholder. So I really appreciate that you did this.

[rgrinberg]

Thank you for your contribution. This looks much better indeed.