SmartOS support

[rkh]

As an alternative to Linux and OSX. I have never used SmartOS, so no idea how hard this would be. I guess we could use Solaris containers on a SmartOS host system.

[joshk]

We could add a Joyent adapter to travis-worker which would spin up SmartOS containers.

On 26/08/2013, at 4:14 PM, Konstantin Haase [email protected] wrote:

As an alternative to Linux and OSX. I have never used SmartOS, so no idea how hard this would be. I guess we could use Solaris containers on a SmartOS host system.

—
Reply to this email directly or view it on GitHub.

[StephanieSunshine]

So i'm the one that filed the bug: rubinius/rubinius#2866

I had to compile LLVM from scratch and I added clang and tried LLVM without clang. Every compile of LLVM worked after the bug patch here was applied: http://llvm.1065342.n5.nabble.com/Compiling-llvm-and-Clang-in-solaris-10-td58748.html

I also tried against LLVM 3.2 with no luck

From there, using Ruby193 or Ruby200 with LLVM 3.3 I would get the same error of _setjmp not being defined. However, simple googling show man pages that speak of it's existence.

http://www.manpages.spotlynx.com/solaris/man/_setjmp.3C

I can provide access to the zone I built this in if it helps the debug process. LLVM within Virtualbox on a i7 with clang about an hour to build.

[roidrage]

@rkh do you think this is something we should implement in the near future?

[joshk]

I have spoken to a guy at Joyent about SmartOS support, I can follow up on that next week.

[roidrage]

Sure, you can, but my question is whether this is something we'd want to tackle (or have the capacity to) in the coming weeks or months?

[roidrage]

I'm asking this question because SmartOS involves touching our workers, adjust travis-build to work with it and to add full support to our Chef cookbooks. Is that effort worth the while in the near future?

[rkh]

Unless we get it supplied from the outside (say, Joyent) I don't see us working on it any time soon. SmartOS is also very niche.

[joshk]

More updates on this soon.

[jquast]

Is it possible to purchase my own SmartOS instance, deploy a travis-ci agent, and integrate it with my project? It doesn't appear so -- but if it is possible, I'm all ears!

[rkh]

Is it possible to purchase my own SmartOS instance, deploy a travis-ci agent, and integrate it with my project? It doesn't appear so -- but if it is possible, I'm all ears!

It is not. The worker needs to have means of controlling the SmartOS VMs and we currently do not support third-party workers. That said, if the worker and the provisioning logic had SmartOS support, we could also run it for you.

[roidrage]

Closing this issue for now, as we have no immediate plans to add this feature.

Should we add it to the roadmap eventually, we'll make sure to update this ticket.

[kigster]

That's too bad :( SmartOS, while possibly a niche OS, is actually super nice for running server applications, in particular based on MRI ruby, Java, jruby, node.js, etc.

Not intending to start an OS war here, but just want to shed some light on why this "niche" OS is used by companies to run high scale ruby apps.

We (at Wanelo) are hosted on Joyent Public Cloud (JPC), and when I was working at another large online retailer, we also migrated their entire app infrastructure to Joyent without a hitch. Here are several reasons I think SmartOS (and JPC) is awesome for deploying custom (ruby) applications:

• SmartOS used lightweight containers for many years before people discovered that software-based virtualization actually works pretty well. You can (and could for years) dynamically resize your instances while they are running and continuing to serve requests. You can add/subtract more RAM and/or CPU cores, or disk space.
• SmartOS comes with a built-in service management framework called SMF. SMF is bulletproof at managing and watching over services, and their spawned child processes if needed, restarting the master process when something crashes, etc – all this behavior is extremely customizable.
• SmartOS is a dream OS for performance geeks (like myself). It comes with dTrace, and everything you install (that supports dTrace) compiles with dTrace enabled. This allows us to trace things like PostgreSQL's queries: their rate per second, average timing, slowest queries, etc. We also instrument our own ruby code with dTrace probes supported by ruby 2.0. One of the most comprehensive (but very long) resources on this, is this recently published book called "Systems Performance: Enterprise and the Cloud".
• SmartOS typically runs on ZFS, which comes with it's own amazing features, like copy on write, and file system snapshots and rollbacks. We are using this extensively, eg. for databases. By taking a snapshot every hour on the DB host, you have a solution when someone, say by accident, truncates or drops production tables. Without snapshots you have to run a replica with a delay, but unlike snapshots you do not get multiple time periods for recovery – you get as many as the number of replicas you've setup.
• SmartOS has a concept of a "project" which is a custom defined aggregation of processes. Our chef cookbooks often define a project for a set of related services, for example we group various related sidekiq worker processes into a single project. Majority of SmartOS system observation tools. like prstat, have CLI flags that show by-project breakdown. Here is an example breakdown (taken from production, reproduced with fake project names for confidentiality):

# -J = show by project
# -L = show threads instead of processes
# -n0,20 = show up to 20 project summaries and zero per-process stats

> prstat -J -n0,20 
PROJID    NPROC  SWAP   RSS MEMORY      TIME  CPU PROJECT
   113       13 5243M 4642M   3.5%  23:47:25 2.9% sidekiq-email
   124        4 2036M 1749M   1.3%  25:15:55 2.8% sidekiq-feeds
     0       36  569M  425M   0.3% 1377:06:2 2.6% system
   104        4 1481M 1347M   1.0%   7:26:22 1.6% sidekiq-database-updates
   106        1  315M  287M   0.2%   5:03:26 0.7% sidekiq-counter-caches
   118        1  322M  292M   0.2%   4:44:01 0.7% sidekiq-cache-expiration
   116        2  651M  585M   0.4%   3:32:04 0.6% sidekiq-solr
   126        1  324M  293M   0.2%   3:18:51 0.6% sidekiq-fibonacci
   108        3 1219M 1082M   0.8%   3:50:08 0.5% sidekiq-default
   105        1  322M  293M   0.2%   2:49:14 0.4% sidekiq-solr

You can see that this view shows how many processes, (or independent threads with -L switch) are within each project. I found this information invaluable in debugging performance problems with background jobs. For example, we now have a well defined process for identifying how many threads vs processes to configure each group of sidekiq workers based on their workload as shown above. We use prstat -mLc -j <project-id> which gives microstate accounting for a given project, presumably the one that consumes most of the resources.

TL;DR

Once again, not here to start an OS war, as many other options, such as FreeBSD and Linux, are also valid, and have their own benefits, where a much wider adoption is probably the biggest one. That said, I don't see many resources published on exactly why SmartOS is used by some folks, so this is my take on it, after running large scale (ie 50K-200K RPMs) ruby applications for two companies over the last 6 years.

[dago]

I'd be interested in support for third party adapters too. It would allow using Travis as industry standard for non-mainstream operating systems and allow to broadening the test coverage e.g. to big-endian systems. At OpenCSW we are providing packages for Solaris and it would be nice to benefit directly from Travis integration. We already use the Webhooks from Github to trigger builds on Solaris and feed the status back for commits and pull requests.

Best regards -- Dago