Can ANTLR lexers/parsers be used for VS syntax highlighting and Error List?

[stevecooperorg]

Hi -- I'm using Antlr4cs and it's working beautifully; apologies if this is the wrong place to post!

I'm happily developing a language and I'd like to start integrating my language into an IDE. The natural place for me feels like integrating it into Visual Studio. It occurs to me that someone might have figured out a general way to plug an ANTLR lexer into the Visual Studio syntax highlighting system, or ANTLR parsers into an MSBuild task such that errors appear in the Visual Studio error list.

Is there any kind of starting point, base package, plugin system, or similar which lets you integrate an ANTLR4CS language with Visual Studio's language support mechanisms?

Alternatively, has anyone had any success building, say, a Sublime Text build system / language def automatically from their grammar?

[sharwell]

Hi @stevecooperorg,

Considering I originally joined the ANTLR project for the purpose of addressing this question, I would say it's quite relevant. Unfortunately, I don't know of a framework available to automate the process. While I've become quite efficient at integrating languages into an IDE, it's primarily due to the fact that I know exactly what parts to duplicate from other closed-source projects developed at companies I work for.

Note that if you implement your IDE support as an open source project and run into specific problems that you can point to in code, it would be much easier for me to guide you towards a solution.

[stevecooperorg]

Sam - thanks very much for the response!

I asked this on StackOverflow, too, and someone suggested that a single-file generator might be the way to go for a small, isolated language because it's easy to report errors during compilation. Does that seem sensible in your experience?

[sharwell]

That's probably not what you want. Single file generators are used when you want to add a file to a project which is written in a DSL, but produces code in the language of the project which gets compiled. For example, support for resource files could be implemented this way, where the .cs file is generated from the .resx file, and then the .cs file is compiled along with the C# project. These generators don't provide any editor functionality (of any kind). It's really two different problems.

[stevecooperorg]

Thanks, Sam. I'll have a look at some other options, then. Thanks for the guidance.

[timeraider4u]

Sorry for posting to an old thread, but I found this issue through Google search. The language server protocol (or LSP for short) might be relevant to deal with these kind of questions. VS and other IDEs have already support for it. One drawback is that syntax highlighting is not supported yet by LSP (but it is in the backlog for the next version, see microsoft/language-server-protocol#124).

[timeraider4u]

Once the mentioned issue is resolved, I will try to implement a language server for the ANTLR4 grammar itself as proposed in antlr4ide/antlr4ide#186

[ghost]

You can add syntax highlighting to a language using only a textMate file in VS Code. I am working on a language extension for a language for which there is an antlr2 parser, I don't know if there is a tool that allows converting the lexer rules in a .g file to a textMate file. As to error lists, I think the best way to implement it is to write a language server that interacts with the parser

[timeraider4u]

Yes, you are right. As https://www.eclipse.org/community/eclipse_newsletter/2017/may/article3.php mentions, most LSP plugins for Eclipse also use TextMate for syntax highlighting. As Antlr can already generate code for syntax highlighting, I am not happy that syntax highlighting is not already supported through LSP because if it were, all dependencies could just be on the server side and the plug-ins for IDEs would get plain simple to implement.

There currently is no converter for Antlr <-> TextMate grammars, as far as I know. There is a syntax highlighting grammar in TextMate for Antlr, but it seems to be an older version (Antlr2/Antlr3?): https://github.com/textmate/antlr.tmbundle

[timeraider4u]

Yes, you are right. As https://www.eclipse.org/community/eclipse_newsletter/2017/may/article3.php mentions, most LSP plugins for Eclipse also use TextMate for syntax highlighting. As Antlr can already generate code for syntax highlighting, I am not happy that syntax highlighting is not already supported through LSP because if it were, all dependencies could just be on the server side, the plug-ins for IDEs would get plain simple to implement and implementing syntax highlighting would not be much work.

There currently is no converter for Antlr <-> TextMate grammars, as far as I know. There is a syntax highlighting grammar in TextMate for Antlr, but it seems to be an older version (Antlr2/Antlr3?): https://github.com/textmate/antlr.tmbundle But even if it were Antlr version 4, it would require writing a converter anyway (and this is not something I would rather avoid if possible).