Archive for category Testing

Book Review: Selenium 1.0 Testing Tools

The Book

Title: Selenium 1.0 Testing Tools (Beginner's Guide)

Author: David Burns

Publishers: Packt Publishing

Review

First half of the book is dedicated to Selenium IDE and the second half covers -- Selenium Remote Control (RC), Selenium Grid, and discusses the upcoming changes in Selenium 2.0.

This is a how-to book, a detailed step-by-step guide, with several screenshots. The book starts off with the installation and setup of Selenium and then proceeds to cover Locators and Pattern Matching in the subsequent chapters. If you could identify the elements on the web page that's a battle half won in the test automation. Various techniques are in display to locate the elements including XPath, CSS, element IDs, link text. Pattern matching covers finding elements by using regular expressions and globs.

There is a lot of emphasis on Selenium IDE. If you are someone who are not too comfortable writing test scripts using Selenium RC (API-driven) you can take the best advantage of the bulk of the book. Later on in the book, the author discusses how you can convert your IDE-based tests to Selenium RC (as you get more familiar with APIs).

The discussion on the Remote Control and Grid is adequate. Integration of Selenium with JUnit and TestNG is nice, very handy if you would like to run your tests in parallel.

Given that the Selenium 2.0 is going to be released very soon (in the next couple of months?), I'm not so sure why the author concentrated on 1.0 for the most part of the book. Last chapter of the book is dedicated to discuss Selenium 2.0 changes. If my understanding is correct, most of the impact from Selenium 2.0 is in the Remote Control side, merging WebDriver stuff. The book, in general, is organized very well. The only complaint I have in the organization is: in the earlier chapters, may be the author should have pointed out which APIs are changing or would get impacted because of Selenium 2.0.

This is a beginner's guide, as printed on the cover of the book. Nothing less, nothing more. If you are looking for more advanced techniques of functional/acceptance testing, and want to learn internals of Selenium then look elsewhere. But if you are starting out and have little or no familiarity with Selenium then this book can certainly help you to get upto speed real quick.

 

Tags: , ,

WebDriver (Selenium 2.0): First Impressions

I've been using Selenium and Watij for a while now (more Watij than Selenium). I've written about my prior experiences here and here. Watij -- there is no significant development happening there for the last year or two. While Selenium Remote Control was nice, its IE support (especially IE 7) was not that great when I last evaluated (it may have improved since then).

I guess on Twitter, I first heard about WebDriver, and accolades about its approach and its API design. So decided to give it a try. Best part is Selenium and WebDriver projects are joining forces with a merger in Selenium 2.0. Per my understanding, the plan is to provide both Selenium and WebDriver APIs under Selenium 2.0 umbrella. At the time of writing this post Selenium 2.0a1 (Alpha-release) was out.

So here are my first impressions:

  • Approach: At the outset WebDriver is similar to Selenium (Remote Control) and Watij -- developer-focused and API-driven. But the best part, for me at least, WebDriver doesn't take the approach of running as a Javascript application within the browser (like Selenium), rather it takes JNA approach interacting with the browser. That approach rules out same origin issue (a headache with Javascript approach).  Also this doesn't require native libraries installed and registered on the client (e.g: DLLs need to installed on the client for Watij).
  • API: API is intuitive. For example, following code snippet finds the element (text box, in this case) by name and simulate the typing by the sendKeys() method.
    driver.findElement(By.name("full_name")).sendKeys("Joe Schmo");
    

    Similarly,

    driver.findElement(By.name("Details")).click();
    

    finds the element (button, in this case) and clicks the button.

  • Multiple drivers: Unlike Watij, which only supports IE at this time, WebDriver supports -- IE, Firefox, Chrome, and it also has a HtmlUnit version to run headless.
  • Window/Frame handling: This is yet another area where WebDriver excels with a caveat (that I'll get into in a moment). When an operation like a button-click or a hyperlink-click opens a new pop-up browser, WebDriver provides an excellent way to get a handle to the new browser. Following call returns a set of all the window Ids.
    driver.getWindowHandles()
    

    You can iterate over the Set to find which one is your new window and switch to that window for all future operations.

    driver.switchTo().window(windowName)
    

    Windows' handling is quite solid compared to handling of the Frames. In most of the situations it worked except when there are iFrames which load dynamically the behavior is not that consistent.

  • Wait behavior: When a particular operation (like a button-click) is underway WebDriver waits for the operation to finish before it executes the next step of the test case. Although this sounds trivial, I've had issues with this while using other tools. In those cases I had to use Thread.sleep() to wait between the page load and the execution of the subsequent step.
  • Alert Dialog: Couldn't find a way yet to handle alert dialogs. I have to dig a bit more into the API and forums to see if they are supported at this time.
  • Mouse Events: I'd like to see more mouse events -- double click, hover over an element, page scrolling, right click, etc.

Are you using WebDriver? What are your thoughts?

driver.findElement(By.name("q"));

Tags: , ,

Test Automation Revisited

More than an year ago I published my experiences with Selenium and Watij. At that time I was actually not sure whether Selenium and Watij were ready for prime usage, especially for testing on IE7 (a requirement for my app). So fast forward to now, I'm lot more satisfied with what I'm seeing ..

Both the tools have got more recent versions out compared to what I used earlier. Started to work on Selenium RC. I was glad a lot of stuff that was a bit shaky in my earlier experience is more polished now. Just when I was thinking that I'm proceeding quite well with Selenium RC, hit with an issue where Selenium loses handle to the main window if the (child) popup window posts data and the main window refreshes based on that post. Logged a critical issue at Selenium's Jira, still awaiting for some action on that (I wish I had some time to look into that and provide a patch, if required).

After that, started to look for alternatives -- having used Watij earlier, prompted me to give it another try. I was pleasantly surprised by the ease at which I could proceed this time, including the issues that I was having earlier with modal dialogs. Here are some of the comparisons:

  • Watij's API is more Object-oriented than that of Selenium's, which is perhaps more functional in nature.
  • While using Selenium I had to use waitForXXX calls to wait till the page gets loaded, or for the popup to get loaded, etc. But with Watij the tool is "smart" enough to figure when the page load completed so as to proceed with the subsequent calls. Having said that there are still some areas where some sort of wait is required, but I believe that is based on the application functionality that is under test.
  • Selenium injects Javascript into the pages of application under test. It also requires Selenium server running (only proxyInjection mode worked for me). Watij on the other hand doesn't need a server. It requires a dll (yikes!, but that's not that bad) to be placed in your System32 folder of Windows. I found Watij to be a bit more robust and found that behavior was consistent when compared with Selenium.
  • A drawback for Watij is it works only for IE. That works fine for me as that is our primary requirement. If you have to test under different browsers than Selenium is perhaps the way to go.
  • With Selenium IDE, the tests can be recorded using a Firefox plugin. There is no such recording capability with Watij.
  • Watij doesn't have a Selenese type of "language" to write the test cases outside of using a programming language (like Java). Needless to say that this kind of notation would be useful for non-technical people. Having said that it is not that difficult to write your own version of Selenese kind of notation.

With all those observations, Watij was what I chose to proceed with. Combination of the robustness of Watij and the critical issue that I was hit with while using Selenium (described above) made me to move in the direction of Watij.

Tags: ,

Watij and Selenium: Are they ready for prime usage?

Note: Published an update here.

I have been looking out for some good open open source automated functional testing tools. After some bit of research and based on some recommendations I went on to try Watij and Selenium.

Watij:
Based on popular ruby based Watir, I was quite impressed by its API. It has got good little documentation on its site and on wiki, just enough for you get started and do some reasonable testing. You can write the tests using the popular JUnit framework, which is a big plus.

It all worked fine, in fact, I could automate quite a few screens in an application until I hit a roadblock :( Whenever it encounters modal dialog windows it just stops working. Some how it seems not recognize them, and waits for the screen to load even though it has already been loaded.

Wondering how other people working with Watij working around this limitation. It has been a while since they made a release after 3.1.0. I hope the support for modal dialog windows would be functional in the next release.

Selenium:
After that experience with Watij, I started looking at Selenium. For some reason, I did not have that much luck, not even as much compared with Watij.

After starting up the Selenium server and try to hit a web page I've received a 404 error on page. Based on the documentation on the site, I have started using *iehta instead of *iexplore and the 404 went away. The joy was short lived as I started getting windows dialog messages saying a "script is trying to run, do you want run it?" (not the exact verbage). I had to keep clicking Yes on it to move forward.

However, with bit more researching I found a way to resolve that. This time started the server in the mutiwindow mode (with -multiWindow option) the application started running on a different window than that of the Selenium browser.

That is as far as I can go with this at this point. As I start with a page on the application I see dozens of browser windows started popping for the same page. Each of this page shows up a confirmation popup saying that the web site you're on is trying to close the window, do you want to close the window? Not sure why Selenium wants to close the browser as soon as it opens this very first page! This was working some time ago when I was on IE 6, now that I'm on IE 7 I find this issue.

Conclusion:
All in all, I found more success with Watij than Selenium (on IE 7). I'm sure there are many people out there using both these APIs. I see there are many open source projects using Selenium. I'd be interested in knowing what they think about these two APIs. Which one they liked better and why? Are they ready for prime usage?

Tags: , , ,