Add rouge as an alternative to pygments (Jekyll 2.x)

[kostrse]

Jekyll 2 supports https://github.com/jneen/rouge hightlighter as an alternative to pygments.
This change adds current version of rouge (1.5.1) to github-pages.

Backlog and known issues:

• Upgrade to Jekyll 2.1 #75 Upgrade to Jekyll 2.1
• Invalid highlight html with rouge and linenos jekyll/jekyll#2607 Invalid highlight html with rouge and linenos

[lkuczborski]

When can we expect Rouge to be included? It is currently the only one to support Swift :(

[kostrse]

Only after integration of Jekyll 2.x:
#75 Upgrade to Jekyll 2.1

[benbalter]

What's the advantage to Rouge over Pygments? From what I understand, it's pure Ruby (:+1:), but less feature complete (:-1:).

I'm hesitant to add Yet Another Syntax Highlighter (TM), lest we end up with Markdown interpreter-style code sprawl. It's crazy confusing for new users. (e.g., how do I know which one's "better"?)

As of right now I'm 👎 on including Rogue, unless A) it gains feature parity with Pygments, and B) Jekyll drops support for Pygments in favor of Rogue as a transparent upgrade from a user perspective.

[lkuczborski]

Rouge is supporting Swift already as I understand. This is important for many Swift bloggers out there. Currently no way to do this natively in Pygments, unless you create custom lexer yourself.

[parkr]

@lkuczborski Have you upgraded to pygments.rb 0.6.x? It is supposed to support rouge.

[lkuczborski]

@parkr How would I do that? I am using Bundler to see what is currently available on Github Pages and it uses pygments.rb 0.5.4.

[parkr]

@lkuczborski It will be bumped with the Jekyll 2.1 bump. Jekyll 1.x does not support Rouge.

[parkr]

So far GitHub Security is 👎 on Rouge, and @benbalter is likewise per the "Yet Another X" argument.

I see Jekyll eventually switching over to Rouge exclusively but until then, we will want to close this. Thanks!

[swanson]

Just a footnote - it was not clear to me that Rouge is not supported on GH Pages yet. I tried using it, only to find that the build would fail on GitHub - with a pretty unhelpful message (generic markdown syntax error). Reverting to pygments fixed the issue.

Could/should the jekyll proper docs be updated to be more explicit about what GH Pages supports?

[kleinfreund]

Basically, GitHub Pages and Jekyll are not the same thing, but having a note for so thightly coupled things would be helpful, yes.

I have bookmarked https://pages.github.com/versions/ to check what Pages is actually using currently.

[ronan-mch]

Hi, Just to note that this would be a useful change for those of us developing on the Windows platform. Jekyll cannot build successfully on Windows when pygments is used for highlighting (see this issue for details). The accepted solution is to use rouge instead, but this caused a build error when my site was pushed to GitHub presumably because Pages doesn't support rouge. This means that I will have to use rouge locally while ensuring that it isn't pushed to GitHub. A minor inconvenience but I thought you might like to hear about it. Cheers!

[parkr]

@ronan-mch FWIW, we plan to set Rouge as the default in Jekyll 3. Not sure about Pages adoption, but I'm going to ask for it very nicely. It being purely Ruby, Rouge provides a better experience for all, including those who run JRuby.

[mofosyne]

github pages default highlighter pygments just plain don't work well on windows.

My current process is quite cumbersome, in having to switch between rouge/pygments. So getting rouge enabled by default would be awesome. (at least until pygments get their build fixed for windows)

And yea I agree with you parkr that rouge seems to be a much less painful experience installing it.

[NickCraver]

Has there been any change on this? I'm currently maintaining my site on multiple OSes (Windows and OS X) and don't need python at all except to get syntax highlighting. It's a rather large pain to install an entire runtime (with security concerns of its own) for such a simple purpose. It's tremendously more complicated to setup and maintain (especially on Windows) whereas rouge took no time at all.

It's a negative enough experience on Windows that I've currently given up on syntax highlighting on GitHub itself. I'd love for it to work, but it's simply not worth the trouble to get pygments running smooth on Windows.

[parkr]

Hey Nick,

As you may know, Jekyll 3.0 will ship with Rouge as its default, rather than pygments.rb. As Pygments and Rouge are not 1-1 compatible, I'll be asking the Pages team to support Rouge instead of Pygments to provide the best experience to Pages users. That said, it's entirely their call so it's possible they may not choose to support Rouge.

[NickCraver]

@parkr Thanks. It'll be unfortunate if they don't even support the defaults from you guys. A note here about the security concerns would be helpful so they can at least be addressed and hopefully resolved.

I do think it would be worth nothing in the Jekyll docs that Rouge isn't a viable option for such a large use case.

Please do let us know what they say about Rouge support with 3.0. If they don't intend to support the defaults or even (IMO) reasonable setup, then I'd personally much rather do my compilation and pushing to hosting elsewhere.

[vmg]

Hi! Unfortunately we're not able to support Rouge in production right now (for the reasons previously stated in the thread). We're looking to migrate to another syntax highlighter, the one we use in production and which runs natively and without Python, but I have no ETA on that at the moment. Sorry for the hassle!

[NickCraver]

@vmg can you actually elaborate on the security reason? Unless I'm missing something, only 2 reasons were given in this thread:

1. It's "yet another highlighter"
2. Unknown security issue

Number 1 is going away as it will be the default. That means a default install of Jekyll will not behave as expected when pushed. I'd say that's also far more confusing for new users, which was another portion of the point.

Number 2 has no detail at all, it's just a relay of "github security doesn't like it". I'm not saying it's unfounded, but why not tell us what's wrong? We're software developers here, if security is the only reason left, let us actually fix it.

There's currently no decent way for Windows users to use syntax highlighting on GitHub, that's really the bottom line here. Given the source of the hosting, the audience using it, and he mechanism in which it works (a git push), it's far more than likely someone wants to use this with GitHub pages.

All I'm asking is this: Please let us help.

[vmg]

Hi again! We don't have a specific security issue in mind when looking at Rouge, but our concern is with the fact that it's a third party library which we've never used, audited nor reviewed before.

We've used Pygments in production for 5 years with solid results and audited it extensively, and we really trust it. We've just recently switched to a homegrown syntax highlighter which we've also audited inhouse and paid a (very expensive) third party audit for. This is software that runs under a sandbox but that it's used by millions of users every day; we take security very seriously, specially for the parts of our infrastructure that process arbitrary amounts of user content every day, and we cannot throw in new libraries in our stack just because Jekyll upstream decided to switch to them.

We could certainly spend our time reviewing and preparing Rouge to be run in production, but we'd rather work on open-sourcing our own syntax highlighter so people on all platforms can enjoy syntax highlighting at native speed without having to install other language runtimes. I'm 100% sure that once we release PrettyLights (our native highlighter), it will become the default in Jekyll, so wasting our time with a full audit and integration of Rouge can hardly be justified.

Again, I definitely feel your pain (syntax highlighting in Ruby is one of the many things that are inconvenient to do in Windows right now), and I appreciate your willingness to help, but there's not much you can do here -- it's just a matter of us not having enough hours on the day. But you can have my word we're prioritizing the issue, and that we really want to share our new highlighter with the world.

[thomasjo]

I'm 100% sure that once we release PrettyLights (our native highlighter), it will become the default in Jekyll

Awesome! ✨

[parkr]

I'm 100% sure that once we release PrettyLights (our native highlighter), it will become the default in Jekyll

Man, this must be the best syntax highlighter out there!

Whatever we support by default must be 100% compatible for Windows and keep C ext to an absolute minimum. The headaches we have had to deal with as a result of pygments and posix spawn on all sorts of systems make a pure-Ruby syntax highlighter far more appealing as a default. Hope it's that!

[NickCraver]

@vmg Thank you! That's a much better picture of the GitHub side I was looking for and it sounds like a great solution all around. Do you have any idea if it'll be released in time to be in Jekyll 3.0?

It seems like changing the default would be best in a major release on their side, and it'd be good if you guys reached out (if you haven't already!) as they may want to coordinate the release with it in mind.

[donmccurdy]

The recommendation here seems to be to keep using Pygments until PrettyLights is released, but I've started getting notices that redcarpet support is going away and I need to switch to Kramdown.

Kramdown supports Rouge or Coderay, but GH Pages does not. Pygment doesn't seem to work with Kramdown unless I rewrite everything in Liquid {% highlight %} tags. Is there another way to keep syntax highlighting after I switch to Kramdown?

---

EDIT: Apparently I misunderstood, or this thread is out of date. GH Pages seems to work fine with Rouge, according to both the Jekyll 3 announcement post and a quick test on my blog.