Skip to content

Project for benchmarking popular Json Web Token (JWT) frameworks for Java using JMH.

Notifications You must be signed in to change notification settings

skjolber/java-jwt-benchmark

Repository files navigation

java-jwt-benchmark

Project for benchmarking popular Json Web Token (JWT) frameworks for Java using JMH.

Supported frameworks:

License

Apache 2.0

Obtain

The project is based on Gradle.

Usage

Modify the build version to your current snapshot, then run

./gradlew --stop && ./gradlew clean jmhClasses jmh --refresh-dependencies --info

Note: Requires Java 8 to build, there is some classpath issue with Java 11 which I've yet to understand.

Disable CPU Boost

Temporarily disable boost (untill next reboot) for AMD:

echo "0" | sudo tee /sys/devices/system/cpu/cpufreq/boost

The JMH plugin seems to have trouble refreshing the project, so restart the Gradle deamon before running.

Benchmarks

  • Parse token (if possible)
  • Verify token
  • Verify token, get claim.

General performance observations

There is not much headroom for improving the performance of RSA-based signatures; most (> 90%) of the CPU time goes to signature verification.

This observation is based on two simple 'baseline' RSA-based JWT verifiers included in this project. They perform about 5% faster than the fastest JWT libraries, but then only does signature verification and no JSON parsing / field validation. So it seems likely this is the upper limit.

Latest results

Running for OpenJDK 11 build 11.0.18+10 and OpenJDK 17 build 17.0.6+10, on Fedora Linux 6.1.8-200, AMD Ryzen 5950.

Result visualizations:

Summary: For JWT verification fusionauth-jwt and java-jwt are tied for first place. fusionauth-jwt can parse faster (without verification).

Previous results

Note that the relative number matters, benchmarks are not necessarily run on the same machine, operating system or Java version.

Version 1.4.0:

Running for OpenJDK 11 build 11.0.11.0.9, Fedora Linux 5.13.4-200, AMD Ryzen 7 PRO 3700U laptop.

A visualization_v1.4.0 is available. In short, fusionauth-jwt is the fastest parser, but Java-jwt is very close.

Version 1.3.0:

Running for OpenJDK 1.8.0 build 275-b01, Fedora Linux 5.10.7-200, AMD 5950x.

A visualization_v1.3.0 is available, summery:

Framework Version Verify (op/s) Claim (op/s) Parse (op/s)
java-jwt 3.12.0/0.15.0 30.1k 29.9k 387k
fusionauth-jwt 4.0.1 32.6k 32.4k 1196k
jjwt 0.9.1 31.4k 30.7k
Nimbus-JOSE-JWT 9.4.1 19.7k 18.9k 945k
Okta JWT Verifier for Java 0.5.0 1.3k 1.3k

In short, fusionauth-jwt is the fastest parser. Parsing without validation is now faster than before for java-jwt, otherwise no improvements.

Version 1.2.0:

Running for OpenJDK 1.8.0 build 232, Linux 5.3.7.

A visualization_v1.2.0 is available, summery:

Framework Version Verify (op/s) Claim (op/s) Parse (op/s)
java-jwt 3.8.3/0.9.0 14.6k 14.6k 230.8k
jjwt 0.9.1 14.9k 14.5k
Okta JWT Verifier for Java 0.4.0 0.6k 0.6k
fusionauth-jwt 3.1.6 15.9k 15.7k 581.5k

In short, fusionauth-jwt is the new fastest parser. Parsing without validation is now faster than before for java-jwt, otherwise no improvements.

Version 1.1.0:

Running for OpenJDK 1.8.0 build 191.

A visualization_v1.1.0 is available, summery:

Framework Version Verify (op/s) Claim (op/s)
java-jwt 3.7.0/0.7.0 14.5k 14.6k
jjwt 0.9.1 14.7k 14.9k
Okta JWT Verifier for Java 0.4.0 0.61k 0.61k

In short, java-jwt did catch up with jjwt, they were essentially just as fast. The Okta JWT verifiser regressed into a total disaster.

Only java-jwt seems to parse without validation, at about 52.3k operations per second.

Version 1.0.0:

Running for Oracle JDK 1.8 build unknown-

A visualization_v1.0.0 is available, summery:

Framework Version Verify (op/s) Claim (op/s)
java-jwt 3.3.0/0.4.0 14.2k 13.9k
jjwt 0.9.0 19k 18.3k
Okta JWT Verifier for Java 0.3.0 17.1k 17.1k

Only java-jwt seems to parse without validation, at about 65.2k operations per second.

History

  • 1.x: Added baseline JWT verifiers
  • 1.5.0: Updated dependencies
  • 1.3.0: Updated dependencies, added Nimbus JOSE + JWT
  • 1.2.0: Added FusionAuth
  • 1.1.0: Bumped versions after accepted performance-enhancing PR#255 for java-jwt.
  • 1.0.0: Initial version

About

Project for benchmarking popular Json Web Token (JWT) frameworks for Java using JMH.

Topics

Resources

Stars

Watchers

Forks

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •