Tuesday, January 6, 2009

Selenium Saga - Part III

This is the last in the series of posts loosely called "So you wanna run Selenium Grid and a Ruby app using VMWare...". I'm putting this all in one post, but I've linked to the older ones for reference.

This is how you do it:
  1. create Selenium tests in the manner of your choosing. How you get there is not important, as long as it responds to something like rake test:selenium.
  2. Go configure your machines, including the remote control and the hub.
  3. Set up your rake task to run the selenium test against each browser.
  4. Set up a runner script to update your code and call the task.
Create Selenium Tests
I'll leave this as an exercise to the reader. I use the selenium gem.

Configure Machines
There are a number of machines to configure: the test runner, the VM server, and the VM clients. Note that these instructions are for a Linux machine.
Configure the Test Runner
The test runner machine runs the Selenium Hub, the script that controls the overall grid run, and the project code.
  1. Designate a user that will run the grid scripts. This user is going to need privileges on other machines, so choose accordingly.
  2. Install Java (be sure it's the JDK). Confirm your installation by opening a terminal and ensuring that typing "java -v" gives you version information.
  3. Install Apache Ant (including setting JAVA_HOME).  Confirm your installation by opening a terminal and ensuring that typing "ant" gives you an error about missing build.xml and no other errors.
  4. Install Selenium Grid. Unzip it and place it in a logical location (/usr/local/bin is where I put it). Be sure that your user has execute privileges on this location. In these instructions, we'll call this ${GRID_HOME}
  5. To be sure everything is running correctly, open up a terminal, cd to ${GRID_HOME} and type: ant sanity-check. You should see text indicating that the build was successful.
  6. Generate an ssh key for your user. Make sure the key has an empty passphrase.
  7. Add the following line to your user's .bashrc so that the key is added when a base session is started: ssh-add ~/.ssh/id_rsa
  8. Create a folder to hold your grid artifacts. This can be anywhere on the system as long as the user has rwx privileges on this folder. I generally put it in the user's home directory (e.g., /home/user/grid). We'll call this ${ARTIFACTS}.
  9. Place the following things inside ${ARTIFACTS}:
  • run.rb
  • A folder called "code"
  • controls.yml
Configure the Remote Control Server
  1. Install the ssh public key for your user from the test runner. Do this by scping the id_rsa.pub key to the machine, and catting it to authorized_keys in that user's home directory.
  2. Ensure that you have permission to run vmware-vim-cmd (you may need sudo for this)
  3. Configure as many clients as you think you will need. Each client needs at least 512MB RAM (and preferably 1 GB) and at least 15GB of disk space beyond what the OS needs.
Configure the Remote Control Client
This should be done on each remote control client individually. Note that I'm assuming a Windows client.
  1. Install the OS and ensure that it is NOT a member of a domain.
  2. Install Java (be sure it's the JDK). Confirm your installation by opening a terminal and ensuring that typing "java -v" gives you version information.
  3. Install Apache Ant (including setting JAVA_HOME). Confirm your installation by opening a terminal and ensuring that typing "ant" gives you an error about missing build.xml and no other errors.
  4. Install Selenium Grid. Unzip it and place it in a logical location (c:\ is where I put it). Be sure that your user has execute privileges on this location. In these instructions, we'll call this ${GRID_HOME}
  5. Create a batch file that cds to the ${GRID_HOME} and runs:
    ant -Dhost=foo.example.com -Dport=5556 -Denvironment=*firefox DhubUrl=http://myhub.example.com:4444/ launch-remote-control
  6. Put that batch file in a folder on your computer (I put it in c:\logonscripts)
  7. Share the folder containing your batch file as NETLOGON (using Properties->Sharing... on the folder). Name matters here; you must use NETLOGON.
  8. Set up automatic script running during logon
    • Go to Start -> Run and type “gpedit.msc”
    • In the Group Policy Editor that opens, select Computer Configuration -> Administrative Templates -> System -> Logon
    • Double click on “Always wait for the network at computer startup and logon” and choose “Enabled”
    • Double click on “Run these Programs at user logon” and select Enabled.
    • In the same “Run these Programs…” dialog, click the “Show…” button, then “Add…”. Put in the full path to the batch file.
  9. Set up Windows to automatically log the user in on boot. Be sure the user that is logged in is the one that runs the batch file at logon. The total effect here is that when you start the machine it will automatically log in and start the Selenium Remote Control (cool, huh?). I use TweakUI to do this but there are several ways, so choose the one you like best.
Set Up Rake Task
Set up the rake task like you did in my earlier post. This will get you 90% of the way there.

For the last 10%, if you're using physical machines for your Remote Controls, well, you're going to have to solve that one yourself (hope you have controllable power!). If you're using VMWare (or Xen or the like), then just power the appropriate machine on from the back end. Here's my trick to do it.
  1. Create a file called controls.yml (remember this? We put it in our ${ARTIFACTS} directory. This should describe for each browser whether that browser is present. It should give the name of the VM (as it's called by the VM software), and it should give the platform that it's running; this is the platform you will feed the rake task. This example is a Windows machine running Firefox 2:
    one:
    vmname: foo
    OS: win
    Firefox 2: true
    Firefox 3: false
    IE 6: false
    IE 7: true
  2. Add this method to your rake task and call it to start your remote control. Note that this is for VMWare; your syntax will vary for other products.
    def startRC(platform,browser)
    vm_hash = YAML.load_file("controls.yml")
    # Get a list of VMs for this browser
    vm_hash.each { | key,value |
    if value["OS"] == platform && value[browser] == true
    # We found a machine that fits our criteria
    cmdBase = "ssh user@vm_server sudo vmware-vim-cmd -U user -P password"
    vmId = %x[#{cmdBase} vmsvc/getallvms | grep #{value["vmname"]} | awk {'print $1'}]
    isOn = %x[#{cmdBase} vmsvc/power.getstate #{vmId} | grep Powered] #get the state
    if /on/ =~ isOn
    #In use by someone else.
    else
    #Start it and give it a bit to come up and log in to the hub
    system("#{cmdBase} vmsvc/power.on #{vmId}")
    sleep(90)
    break
    end
    end
    }
    end
  3. Reverse the method to stop the VM when you're done with it.
Set up Runner Script
Update your code, start the server, restart the hub, and then call the rake task. This is going to be quite specific to your OS, source code repository, etc., so I'll leave this bit for you. 

Gotchas
There are about a million gotchas here - this isn't a small project to undertake. Some of the biggies are:
  • Make sure your SSH keys are on all the machines; there's a lot of ssh-ing back and forth.
  • If you're sudoing, make sure it does not prompt for a password.
  • Be careful of running this in parallel. It looks easy - just add "fork" - but there are problems particularly around changing test data used in multiple tests.
  • The Selenium Hub basically can't recover from Remote Controls hanging or otherwise "going away". So if you have a test run that fails you're going to want to restart the hub and all your remote controls before continuing.

All of this was culled from documentation, blog posts, forums, discussion threads, you name it; I'd like to send a big thank you to the internet at large for helping me get all of this going. 

And good luck with your project!


ETA: By the way, if you need help or have updates, please feel free to comment or contact me privately. I don't promise to know the answers, but maybe we can work through it together.

6 comments:

  1. Thanks a ton for sharing you hard work with us.

    Well, I am in the midst of learning Selenium & it really cant get any better than this =)

    ReplyDelete
  2. This is great: We've linked to it from the official Selenium blog. Thanks!

    ReplyDelete
  3. Thanks for the post. I have also added a post on Continuous integration with Selenium grid, Subversion and Hudson. Have a look...

    http://clearspace.openqa.org/thread/17727

    ReplyDelete
  4. The information is really very useful here. Thanks for sharing it and also for its coding too. I like this site. As I wanted to know about Selenium grid since from many days. This information will be useful to me. I will sure visit this site in future.

    ReplyDelete
  5. As per my knowledge it is easy to write or generate a significant number of automated in browser tests. The real challange is to keep maintenance costs low as the application matures,the test suit grows and time passes by.

    ReplyDelete
  6. Please be sure to use "-DhubURL" as the argument, notice the capitals...

    With that minor change it is working perfectly for me. So thanks a lot.

    ReplyDelete