Monday, December 8, 2008

Selenium Grid Update - Handling Dependencies

I set up a Selenium Grid configuration a while ago. It pretty much ran stuff, pointing back to the central grid computer for its server. This was nice, but boy the dependencies!

To run the test, look at all the things we had to do:
  1. update the source code
  2. start the server (script/server)
  3. start the hub
  4. start each of the remote controls and let them register with the hub
  5. finally! run the tests
This weekend I worked on scripting those dependencies (hey, the less I have to mess up, the better).  Here's what I wrote:
  • an init script for selenium-hub that starts the Ruby server and the hub
  • a new environment for the grid (not strictly necessary but it gave me a place for some URLs for third-party dependencies that change by environment in the same place)
  • a rake task to actually do the work of updating source code, etc.
So how does it all work now?
  1. Log in to the hub box
  2. set RAILS_ENV to the environment I'm interested in
  3. call the rake task
  4. Wait until it prompts me
  5. Start the remote controls
  6. Press enter and let the tests run
The trick is to make rake do all your prep work for you. Here's the interesting part of my task:


unless ENV.include?("platform") && @platforms.has_key?(ENV['platform'])
raise "Command line parameters incorrect! \nusage: rake test:run_browsers platform=[win|osx]"
else
platform = ENV['platform']
case platform
when 'win' then
browsers = @win_browsers
when 'osx' then
browsers = @osx_browsers
end

# Write out our environment
puts "Our environment is #{ENV['RAILS_ENV']}"

# Set up our db
puts "Migrating database and loading fixtures"
begin
Rake::Task["db:drop"].invoke
rescue
"Database does not exist"
end
Rake::Task["db:create"].invoke
Rake::Task["db:migrate"].invoke
Rake::Task["db:fixtures:load"].invoke

# Restart the environment
prepEnvironment(browsers)

# Run tests
puts "Running tests for the #{@platforms[platform]} platform"
browsers.each do |browser|
puts "Running tests in the #{browser} browser"
ENV["Browser"] = browser
ENV["TEST_DIR"] = 'test/selenium'
puts "RUN TESTS HERE"
Rake::Task["test:selenium"].invoke #run all the selenium tests
end

def prepEnvironment(browsers)
puts "Restarting Selenium Grid Hub and Ruby server"
system("/etc/init.d/selenium-hub restart")
sleep 5
print "\n"
print "\n"
print "\n"
print "Go start your Remote Controls. Then press enter to continue."
continue = STDIN.gets

host = "localhost"
port = "4444"
path = "/console"
data = checkSeleniumHub(host,port,path) # Hub is running on port 4444
checkNeededRCs(data,browsers) # All needed RCs for the platform are running
puts "Remote controls are running. Proceeding to test."
end

All I really did was take the rake task and make it do its own preparation. Saves time, and makes my tests run a lot more consistently. Hope it helps anyone else who is trying to get Selenium Grid running without human intervention.


Oh, and may favorite part of this is the STDIN.gets call  (translation: wait for the human to go do the part she hasn't automated yet!).

10 comments:

  1. Hey Catherine,

    We appear to be on the same Selenium-Grid-Rails trajectory!

    Could you post more of your rake files and/or email me offline?

    Thanks for your info so far... great posts!

    ReplyDelete
  2. Peter - good luck! This has been one of those projects that seemed simple but had vast numbers of devils in the details (my least favorite so far is Selenium's tendency to simply stop communicating between the hub and a remote control - I still haven't solved that one yet).

    So far I've only got the one rake file doing the work (plus the init script and whatnot, which are specific to my OS). I hope to have another update soon on this one. And thanks for the words of encouragement!

    ReplyDelete
  3. Thanks a lot,

    But I have a question. I've already written my .rsel file and it works. I call it like this rake test:acceptance my_file.rsel. I would like to pass to the file a parameter to precise wich product to test. How can I do. Thanks.

    ReplyDelete
  4. Anon Mar 20:

    Take a look in the rake file where I'm calling Rake::Task["test:selenium"]. You should be able to take that out and replace it with your method of calling the selenium tests.

    ReplyDelete
  5. I am getting an error, by using " STDIN.gets "

    ./lib/tasks/customization.rake:12:in `gets'
    ./lib/tasks/customization.rake:12
    ./lib/tasks/customization.rake:8:in `each'
    ./lib/tasks/customization.rake:8
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:617:in `call'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:617:in `execute'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:612:in `each'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:612:in `execute'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:578:in `invoke_with_call_chain'
    /usr/lib/ruby/1.8/monitor.rb:242:in `synchronize'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:571:in `invoke_with_call_chain'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:564:in `invoke'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:2019:in `invoke_task'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:1997:in `top_level'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:1997:in `each'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:1997:in `top_level'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:2036:in `standard_exception_handling'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:1991:in `top_level'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:1970:in `run'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:2036:in `standard_exception_handling'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake.rb:1967:in `run'
    /usr/lib/ruby/gems/1.8/gems/rake-0.8.3/bin/rake:31
    /usr/bin/rake:19:in `load'
    /usr/bin/rake:19

    ReplyDelete
  6. Anonymous: There are a lot of things that could be going on. Can you please post the complete error message (there should be a part above what you've posted) and let us know your OS, Ruby version, Rails version, Selenium version?

    ReplyDelete
  7. If we are using the same session for too long, selenium hub seems to stop communicating with the RCs. Did you find any solution for this?

    ReplyDelete
  8. Haroon,

    I haven't seen this when the session just stays open (although I've never left them open for more than a couple of hours). I have seen this when the test itself errors or fails. In this case the RC seems to lose connection to the hub (although it doesn't get unregistered). I was never able to figure out a way around that behavior; in fact, that is the number one reason I felt at the time that Selenium Grid was simply not ready for a large-scale deployment.

    In your situation I would take a couple of tacks:
    1. make your tests as short as possible. In general I think this is a good idea. Long tests GUI tests strung together tend to be rather brittle and also tend to show misleading errors (test failures caused by an unseen failure far earlier).
    2. Run a wireshark trace (or tcpdump or flavor of your choosing) on the hub and see if there's a timeout or something you're hitting. It's possible you've got network or pingback problems and it's worth looking.

    ReplyDelete
  9. Hi,

    Can anyone help with a rakefile for php selenium rc tests?

    I need to get my selenium rc tests in selenium grid.

    Thanks,

    Chris.

    ReplyDelete