This repository contains my answer to How can I improve the performance of my xxd port? on reddit.
The result is a Go version of xxd that outperforms the native versions on OSX 10.10.1 / Ubuntu 14.04 (inside VirtualBox), see benchmarks below. However, that is not impressive, given that none of the usual xxd flags are supported.
What is interesting however, are the steps to get there:
- Make the code testable and compare against output of native xxd using test/quick: https://github.com/felixge/go-xxd/commit/90262b3dcdc518ca3eaec7171aa14d74d95f34b8
- Fix bugs: https://github.com/felixge/go-xxd/commit/e9ebeb0abdf78f6e7729fdbfc68842b3a86ee0a3, https://github.com/felixge/go-xxd/commit/120804574f12033999f23e6cf6a3b75961f14da1, https://github.com/felixge/go-xxd/commit/dab678ecf5dcb3eff345db8ac68ae6d7438f9d0e
- Buffer output to stdout: https://github.com/felixge/go-xxd/commit/69b5fe0cc7da80d374413d72892507d5e5ecaabc
- Implement a benchmark: https://github.com/felixge/go-xxd/commit/0bce954073ce92b72ed3fbcf36603c6e23852feb
- Use the benchmark + go pprof to get ideas for optimization
- Remove unneeded printf calls: https://github.com/felixge/go-xxd/commit/473330acd320e5318e896d6408fb3d64a5b8e10b
- Optimize hex encoding and avoid allocations: https://github.com/felixge/go-xxd/commit/dce3bca200ae499e1cf57994c7592a42f66694d5, https://github.com/felixge/go-xxd/commit/a48d892de3fb625bdb3e8e367337578c766a42f5, https://github.com/felixge/go-xxd/commit/d653d2f4eeb5fa41e907f260bd95c205bd8e1ff7, https://github.com/felixge/go-xxd/commit/0d3ae7f0be863a138fab3fd2dd89208073b61c7f
- Avoid type casts: https://github.com/felixge/go-xxd/commit/859ebc4489ce81edc0681881700b0f22754943f0
You can also follow along by looking at the commit history: https://github.com/felixge/go-xxd/commits/master
$ time xxd image.jpg > /dev/null
real 0m0.205s
user 0m0.202s
sys 0m0.003s
$ go build xxd.go && time ./xxd image.jpg > /dev/null
real 0m5.914s
user 0m3.598s
sys 0m2.318s
$ go build xxd.go && time ./xxd image.jpg > /dev/null
real 0m0.138s
user 0m0.133s
sys 0m0.004s
$ time xxd image.jpg > /dev/null
real 0m0.273s
user 0m0.017s
sys 0m0.231s
$ go build xxd.go && time ./xxd image.jpg > /dev/null
real 0m5.856s
user 0m3.517s
sys 0m1.897s
$ go build xxd.go && time ./xxd image.jpg > /dev/null
real 0m0.233s
user 0m0.021s
sys 0m0.207s