automated testing

[trentm]

Bunyan needs to use an automated testing system/service. Currently it relies on me running 'make testall' (which effectively runs 'make test' on a number of node versions I have installed). Doing this on a number of platforms is impractical for me.

Requirements (possibly reduced to nice-to-haves if I can't easily find a service that offers it):

• platforms: Mac, Linux, SmartOS, and maybe Windows
• node.js versions: 0.10, 0.12, 4.x, 5.x
• the dtrace-integration parts of the test suite require 'root', is that possible?
• browser testing: Bunyan is meant to work in the browser as well. It would be super helpful to automate testing that as well

I suspect this is a tall order. I haven't used any of the testing services for a long time. I have access to cloud resources at my job at Joyent, so a low cost home rolled system is possible. I'd like to avoid that if I can.

Eventually using this for PR checking via GitHub's webhook support would be great.

Question for others: Any experiences with the various testing services out there? Comments on which are available and how many of the above requirements they tick off would be helpful.

[glenjamin]

I'm just kicking off a new project an did a survey of available logging libs - bunyan still tops my list - so glad to hear there's still life in the project.

I've had great success with travis CI in the past, it's github integration is really simple and it is able to run on lots of node versions.

Running on windows, browsers dtrace supported systems is a bit trickier.

I can send a PR which runs all relevant tests on those node versions on linux via travis if you like. Could be a good starter for 10?

[trentm]

Note to self: A note on #311 that mocha can test under NW.js: mochajs/mocha#960
My hope is to move to tape for the test lib, FWIW.

[trentm]

I can send a PR which runs all relevant tests on those node versions on linux via travis if you like.

@glenjamin I think that would be a great start, if it wouldn't take you too much effort.

Could be a good starter for 10?

What do you mean by "10"?

[glenjamin]

What do you mean by "10"?

Ah, I forgot that it's a colloquial British phrase!

http://answers.google.com/answers/threadview/id/193772.html

[Srokap]

Travis CI is pretty mature CI service that's free for OSS projects.

It does provide osx and linux environments with sudo access. A lot of stuff is bundled nowadays, but you can go crazy and install custom packages if necessary.

It supports any node js version we could possibly need https://docs.travis-ci.com/user/languages/javascript-with-nodejs#Available-Versions

I was doing some headless browser testing in the past, should be no problem as well.

As far as I know, it doesnt support windows and smartOS is not on a road map at the moment travis-ci/travis-ci#1368