Erics Tech Blog » Ruby

Benchmarking in jRuby NYC.RB Talk

eric — Wed, 10 Nov 2010 16:32:01 +0000

Here are the slides from my presentation on jRuby during the NYC.rb talk on 11/9.

NYC.rb – Nov 2010 – Benchmarking jRuby

JSON Benchmarks in jRuby

JSON Benchmarks in jRuby

eric — Mon, 25 Oct 2010 07:45:12 +0000

I am in the process of switching a major application from MRI Ruby (specifically 1.8.7-p302) using many C extensions to jRuby (currently trying 1.5.3-master). In my application, performance is extremely important. It is so important in fact, that I will be writing about some of my experiences in troubleshooting the speed and getting those important milliseconds back. When I am trying to keep an entire transaction from start to finish under 40ms and just the decoding of a JSON object into a Ruby object in jRuby takes roughly 30ms using json_pure, we may have to explore other avenues.

Just to verify that JSON was indeed as slow as I thought it was, here is a quick benchmark of it:

require 'json/pure'
Benchmark.bmbm do |x|
x.report("parse:") { JSON.parse(json) }
end

JSON Pure Results:

Rehearsal ------------------------------------------
parse: 0.255000 0.000000 0.255000 ( 0.255000)
--------------------------------- total: 0.255000sec

user system total real
parse: 0.024000 0.000000 0.024000 ( 0.024000)
parse: 0.017000 0.000000 0.017000 ( 0.017000)
parse: 0.032000 0.000000 0.032000 ( 0.032000)
parse: 0.042000 0.000000 0.042000 ( 0.041000)

Rehearsal -------------------------------------------
encode: 0.036000 0.000000 0.036000 ( 0.035000)
---------------------------------- total: 0.036000sec

user system total real
encode: 0.010000 0.000000 0.010000 ( 0.010000)
encode: 0.007000 0.000000 0.007000 ( 0.006000)
encode: 0.006000 0.000000 0.006000 ( 0.006000)
encode: 0.017000 0.000000 0.017000 ( 0.016000)

That wasn’t so great. We’re looking at about 30ms on average to decode and an additional about 10ms to re-encode. If we want to keep our total transactions to 40ms and we haven’t even done anything with our data, we’re in pretty bad shape. Let’s try the same thing again, only this time with json-jruby.

require 'json/pure'
Benchmark.bmbm do |x|
x.report("parse:") { JSON.parse(json) }
end

JSON-jRuby Results:

Rehearsal ------------------------------------------
parse: 0.009000 0.000000 0.009000 ( 0.008000)
--------------------------------- total: 0.009000sec

user system total real
parse: 0.003000 0.000000 0.003000 ( 0.002000)
parse: 0.007000 0.000000 0.007000 ( 0.003000)
parse: 0.003000 0.000000 0.003000 ( 0.003000)
parse: 0.002000 0.000000 0.002000 ( 0.002000)

Rehearsal -------------------------------------------
encode: 0.009000 0.000000 0.009000 ( 0.009000)
---------------------------------- total: 0.009000sec

user system total real
encode: 0.001000 0.000000 0.001000 ( 0.001000)
encode: 0.001000 0.000000 0.001000 ( 0.001000)
encode: 0.001000 0.000000 0.001000 ( 0.001000)
encode: 0.001000 0.000000 0.001000 ( 0.001000)

Much better, right. We’re averaging 2.5ms to decode it and 1ms to encode it. That’s much more tolerable.

Just for poops and laughs, we should compare this to our MRI version with C bindings (the version we were using before), the super fast yajl-ruby gem.

require 'yajl'
parser = Yajl::Parser.new
Benchmark.bm do |x|
x.report("parse:") { parser.parse(json) }
end
parsed = parser.parse(json)
Benchmark.bm do |x|
x.report("encode:") { parsed.to_json }
end

And here are the MRI Yajl-ruby results:

Rehearsal ------------------------------------------
parse: 0.000000 0.000000 0.000000 ( 0.000159)
--------------------------------- total: 0.000000sec

user system total real
parse: 0.000000 0.000000 0.000000 ( 0.000101)
parse: 0.000000 0.000000 0.000000 ( 0.000093)
parse: 0.000000 0.000000 0.000000 ( 0.000098)
parse: 0.000000 0.000000 0.000000 ( 0.000093)

user system total real
encode: 0.000000 0.000000 0.000000 ( 0.000063)
encode: 0.000000 0.000000 0.000000 ( 0.000063)
encode: 0.000000 0.000000 0.000000 ( 0.000064)
encode: 0.000000 0.000000 0.000000 ( 0.000065)

These benchmarks are by no means exact, but they do give you a feel for the fact that jRuby is slower than MRI with C extensions. The comparison is essentially 3.5ms round trip in jRuby and about 1.5ms in MRI. It might not seem like a lot, but when every millisecond counts, there is a lot that can happen in those extra 2 milliseconds that could be available every transaction.

Note: I reran the code by hand and formatted the results by hand, so the code won’t exactly produce the results above. These benchmarks were all done on my 2008 MacBook Pro with 4G of RAM and an Intel 2.53GHz Core 2 Duo.

UPDATE: It has been pointed out to me that I didn’t take advantage of jRuby’s hotspot feature (Thanks NickSeiger and Charles Remes). Hotspot basically kicks in after it figures out the best ways to optimize the code. So here are some new results and it’s safe to note that json-jruby is so fast, that the computer can’t time it.

jRuby json_pure:
Rehearsal ------------------------------------------
parse: 0.001000 0.000000 0.001000 ( 0.001000)
--------------------------------- total: 0.001000sec

user system total real
parse: 0.001000 0.000000 0.001000 ( 0.000000)
Rehearsal -------------------------------------------
encode: 0.001000 0.000000 0.001000 ( 0.001000)
---------------------------------- total: 0.001000sec

user system total real
encode: 0.000000 0.000000 0.000000 ( 0.000000)

jRuby json-jruby:
Rehearsal ------------------------------------------
parse: 0.000000 0.000000 0.000000 ( 0.000000)
--------------------------------- total: 0.000000sec

user system total real
parse: 0.000000 0.000000 0.000000 ( 0.000000)
Rehearsal -------------------------------------------
encode: 0.000000 0.000000 0.000000 ( 0.000000)
---------------------------------- total: 0.000000sec

user system total real
encode: 0.000000 0.000000 0.000000 ( 0.000000)

Hash Autovivification in Ruby

eric — Wed, 06 Oct 2010 11:00:38 +0000

One of the features that I miss most from my Perl days (and to be honest, there isn’t a whole lot I miss from my Perl days) is autovivification. For more information on what it is, read the wikipedia page on it here.

I stumbled across a post that talks about mimicking Perl’s autovivification here. But I wanted a data structure (specifically a hash) that would be created on demand. So here is a class that creates a recursive hash.

class RecursiveHash < Hash
def initialize
recurse_hash = proc { |h,k| h[k] = Hash.new(&recurse_hash) }
super(&recurse_hash)
end
end

Interesting Object Methods in Ruby

eric — Mon, 27 Sep 2010 07:45:49 +0000

This little Rubyism is something that I use frequently for debugging my objects. I add a method to every object to show only the interesting methods. What do I mean by interesting methods?

In my .irbrc file, I have the following lines of code:

class Object
def i_methods
(self.methods - Object.new.methods).sort
end
end

Using the Ruby notion that classes are always open, it opens up the base class Object (of which almost everything in Ruby inherits) and adds a method called i_methods. This shows all methods available on the object minus the methods that come with an object by default. It’s in my .irbrc file so I can append this method at any time when I am working in IRB.

Other than that, it’s a handy little concept to remember even if you’re debugging. Just doing the following can help you deal with those pesky NoMethod errors on objects:

p (MyObject.methods - Object.new.methods).sort

Stopping Curb From Segfaulting

eric — Thu, 08 Jul 2010 11:30:00 +0000

While trying to get the curb gem up and running using Ruby 1.8.7p174, I kept getting segmentation faults. I Google’d around and really wasn’t able to come up with much other than lots of people saying not to use ports here. Since I don’t use Mac Ports, I use Homebrew, I figured this wasn’t an issue. I had also recently installed the latest XCode so I incorrectly assumed there was no issue there either. (Note: I am running on a freshly upgraded version of Mac OS X Snow Leopard).

Since everything built correctly, I just assumed it would work correctly. Here it is obviously not doing so.

ruby-1.8.7-p174 > require 'curb'
/Users/elubow/.rvm/gems/ruby-1.8.7-p174/gems/curb-0.7.7.1/lib/curb_core.bundle: [BUG] Segmentation fault
ruby 1.8.7 (2009-06-12 patchlevel 174) [i686-darwin10.4.0]
Abort trap

So I narrowed it down to what I believe is an issue with the curl headers. The solution is to install a new curl library. Since I wanted to do it the Homebrew way, and Homebrew doesn’t have curl by default, here is what I did.

Copy the Homebrew Formula code from http://github.com/adamv/homebrew/blob/duplicates/Library/Formula/curl.rb into a file named curl.rb in your formula directory (mine is /usr/local/Library/Formula/). You should then be able to:

elubow@beacon tmp$ brew install curl
==> Downloading http://curl.haxx.se/download/curl-7.21.0.tar.bz2
######################################################################## 100.0%
==> ./configure --prefix=/usr/local/Cellar/curl/7.21.0 --disable-debug --disable-dependency-track
==> make install
/usr/local/Cellar/curl/7.21.0: 72 files, 1.8M, built in 3.6 minutes

Don’t forget to logout and log back into the shell so the new version of curl is available:

$ curl --version
curl 7.21.0 (x86_64-apple-darwin10.4.0) libcurl/7.21.0 OpenSSL/0.9.8l zlib/1.2.3 libssh2/1.2.2
Protocols: dict file ftp ftps http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: IPv6 Largefile NTLM SSL libz

Now you have to uninstall and reinstall the curb gem. The easiest way that I found (since I have bundler installed) was to do bundle show curb to find out where the gem is located. I removed the directory and reran bundle install (or gem install curb if you don’t have bundler).

Because I am using rvm (Ruby Version Manager), I actually encountered this issue with 1.9.1p378 as well. This fixed the issue for both versions.

Creating Configuration Files With Ruby Templates

eric — Wed, 30 Jun 2010 11:45:58 +0000

I recently had a very repetitive configuration file that needed creating. There were approximately 50 config blocks of 10 lines each with only the host name changing with each block. So I decided to take a shortcut and do it in Ruby using ERB templates. This is so easy and literally save me hours worth of work.

I started out by creating a template for the rsyslog block I wanted to replicate. In the full version of the script, there is an additional section for the access.log file which you’ll see below. If you know Rails, then this template style should look very familiar.

rsyslog_block = %{#
# < %= virtual_host %>
#
$InputFileName /var/log/httpd/< %= virtual_host %>-error.log
$InputFileTag < %= virtual_host %>_error
$InputFileStateFile stat-< %= virtual_host %>-error.log
$InputFileSeverity info
$InputFileFacility local5
$InputRunFileMonitor

}

The below block of code creates a configuration block (above) for each of virtual hosts in the array and then prints it to the screen.

%w{
vhost.com
bar.myhost.com
}.each { |virtual_host| puts ERB.new( rsyslog_block ).result(binding) }

This is going to produce the following block which you can then pipe to a file:

...Removed for brevity...
#
# bar.myhost.com
#
$InputFileName /var/log/httpd/bar.myhost.com-error.log
$InputFileTag bar.myhost.com_error
$InputFileStateFile stat-bar.myhost.com-error.log
$InputFileSeverity info
$InputFileFacility local5
$InputRunFileMonitor

Here is the script in its entirety:

Multiple Input Locations From Bash Into Ruby

eric — Thu, 13 May 2010 12:00:59 +0000

I have been trying to figure out how, while using OptionParser to be able to check for files being input on the command line and if they don’t exist, check other input streams (like Bash). This initially wasn’t very easy since input streams are blocking. So with a little help from friends (thanks roberto), I was able to use his method of non-blocking IO and wrap it in a begin/rescue block. I also took a little advice given in this Stack Overflow question called Best Practices with STDIN in Ruby.

First we need to get the file list off the command line and assume that anything left are files.

@opts = OptionParser.new do |o|
...
end
@opts.parse!(args)
@files = args

Since we are using threads, I open up a thread for STDIN and killed it when we run out of input.

require 'fcntl'
threads = Array.new

# Set $stdin to be non-blocking
$stdin.fcntl(Fcntl::F_SETFL,Fcntl::O_NONBLOCK)

threads[0] = Thread.new {
begin
$stdin.each_line { |line| puts "STDIN: #{line}" }
rescue Errno::EAGAIN
threads.delete(0) # Remove this thread since we won't be reading from $stdin
end
}.run

Now its time for the files.

@files.each do |file|
threads.push Thread.new {
# do stuff with 'file'
}
end

# Put it all together and have the threads run
threads.each { |thread| thread.join }

Using these code snippets, you will be able to use input from both files on the command line and STDIN:

$ myscript.rb file1 file2
$ cat foo | myscript.rb file1 file2
$ myscript.rb file1 file2 < foo

Should I Mock Kernel#exit

eric — Mon, 26 Apr 2010 09:00:44 +0000

I don’t know what the right answer is, so any insight would be appreciated. From Googling around, I came across this thread on StackOverflow which talks about Validating exits and aborts in Rspec. I have a few issues with this, namely that I am using shoulda and I don’t want to validate the exit, I want to stop it from happening inside of the test.

I have figured out that rescueing a Kernel#exit with afford the same opportunity to test results so I am not sure if mocking it is necessary. Consider the following code which doesn’t involve mocking:

class Application
def thread_cleanup(threads)
threads.each do |thread|
thread.kill
end
$stderr.puts "Terminating..."
exit
end
end

# test cases
class ApplicationTest < Test::Unit::TestCase
should "kill all threads and exit cleanly" do
threads = Array.new
threads.push(Thread.new { sleep }, Thread.new { sleep })
begin
Application.thread_cleanup(threads)
rescue SystemExit => e
assert_equal 0, e.status
end
end
end

Here we rescue the exit and continue on with the testing. The tests don’t actually exit (which allows the suite to continue) and no mocking of core functionality is required. Is it the better way? I don’t know, but it works.

Multiple Input Locations From Bash Into Ruby

ColorTail Gem

eric — Tue, 20 Apr 2010 10:30:31 +0000

One of the best ways to learn something new is to find a project in that realm and do it. I wanted to learn Ruby on Rails, so I wrote a web site. I wanted to learn Python, so I did a project in it for work. Now I want to get better at Ruby, so I wrote a gem. It’s usually not as simple as just picking a project though. It’s much easier to pick and complete a project if you are filling a need. So that’s what I did, I filled a need.

Coming from an SA background, every time I write a script, I use lots of logging. It doesn’t matter whether that script is in Perl, Python, Ruby, Bash, or anything else I write. But what I do keep pretty consistent is that I specify the log levels on the log line. So I get things that look like this:

2010-04-19 11:36:39,734 DEBUG Getting all mailings for 2010-05-18
2010-04-19 11:36:39,735 INFO Handling 1 mailing for 2010-05-18
2010-04-19 11:37:16,547 WARNING Handling 0 mailings for 2010-05-18

When I tail this, it’s usually not a big deal to see what’s what because there aren’t things flying by. Every so often, things fly by and it would just be easier to know what I am looking at. Install the colortail:

elubow@beacon ~$ sudo gem install colortail
Successfully installed spruz-0.1.0
Successfully installed file-tail-1.0.5
Successfully installed colortail-0.1.3
3 gems installed
Installing ri documentation for spruz-0.1.0...
Installing ri documentation for file-tail-1.0.5...
Installing ri documentation for colortail-0.1.3...
Installing RDoc documentation for spruz-0.1.0...
Installing RDoc documentation for file-tail-1.0.5...
Installing RDoc documentation for colortail-0.1.3...

Add this setup to .colortailrc:

'syslog' => [
{ :match => /EMERGENCY/, :color => :red, :attribute => :reverse },
{ :match => /FATAL/, :color => :red, :attribute => :bright },
{ :match => /CRITICAL/, :color => :red },
{ :match => /DEBUG/, :color => :green },
{ :match => /ERROR/, :color => :green },
{ :match => /INFO/, :color => :none },
{ :match => /WARN/, :color => :yellow }
]

your syslog style log will look like this:

Colored Log Example

I have tried to keep the code flexible enough for you to mix and match combinations. It’s easy to setup and use. Rather than rewriting everything, just check out the repository here at Codaset. Also check out the example groupings wiki page for more ideas.

Full Disclosure: This is my first gem, so there may be some issues, please submit a ticket here so I can fix it. Or if you’d like additional features, I’d be happy to add them, just let me know.

Hope this is as useful for me as it is for you.

Capistrano Hangs on Mac OS X Leopard

eric — Mon, 05 Apr 2010 11:30:54 +0000

I use Capistrano to deploy my webapps and have been for a while. I also deploy right from my laptop quite frequently. So I was a little taken aback when I could deploy in one place and not in another. It turned out that Capistrano was having issues with deploying to multiple servers. In fact, it turned out that the problem had the appearance of being Capistrano, but in actuality, it was Net-SSH. The whole issue can be tracked via the Lighthouse ticket here. I’m going to show you how to diagnose if you have this problem faster and how to fix it.

However, the shorter version is as follows. It starts out by hanging while trying to parallelize multiple connections. Not only does it hang, but it should peg your CPU. You will be able to tell this if you hit the Terminal and type uptime and you see a high load average and it keeps climbing while your deploy is hung.

elubow@beacon capistrano$ uptime && sleep 5 && uptime
22:25 up 7:25 mins, 2 users, load averages: 2.58 0.61 0.47
22:25 up 7:25 mins, 2 users, load averages: 3.16 1.11 0.97
elubow@beacon capistrano$ ruby -v
ruby 1.8.6 (2009-03-31 patchlevel 368) [i686-darwin9.7.0]
elubow@beacon capistrano$ cap deploy:mail:modules
* executing `deploy:mail:modules'
* executing "sudo mkdir -p /usr/local/lib/perl5/STMail && sudo chmod g+w /usr/local/lib/perl5/STMail"
servers: ["10.123.125.82", "10.123.125.101", "10.123.125.107", "10.123.125.113"]

I am able to replicate the problem all the way back to Net-SSH 2.0.11 (although it may go back further). All you have to do is upgrade your version of Net-SSH. Check your version and then upgrade if necessary.

elubow@beacon capistrano$ gem list | grep ssh
net-ssh (2.0.21, 2.0.11)
elubow@beacon capistrano$ sudo gem install net-ssh
Successfully installed net-ssh-2.0.21
1 gem installed
Installing ri documentation for net-ssh-2.0.21...
Installing RDoc documentation for net-ssh-2.0.21...

Hope this helps.