Posts Tagged OpenSource

Open Source Software Model – A Positive Sum Game

Last week, I've attended Philly ETE conference, which I enjoyed very much. Michael Tiemann, Vice President of Open Source Affairs at Red Hat, delivered the key note on the first day. His presentation was titled Exonovation: Leveraging Innovation from the Edge.

The speaker focused on the advantages of open source -- positive sum game as opposed to zero-sum game. In a zero-sum situation a party cannot benefit unless the other party suffers a corresponding loss. In a positive-sum scenario the sum of wins and losses are greater than zero; competition is viewed as a complimentary thing.

Tiemann presented research summary and stats to emphasize why open source initiatives are thriving. One of the staggering statistics that was presented --- proprietary software having 50 - 150 times more defect density than the open source software. More than 90% of leading IT vendors failed to achieve "good" rating for value from at least 80% of their top customers (both defect density and satisfaction stats are measured 3 years in a row).

He cited his experiences on the discussions he had with Department of Defense on open source software model. One of his arguments against the proprietary model is --- you can't propose that something be a universal space and at the same time keep control of it (obviously, quite a few references to Microsoft; Vista is a favorite target, explaining how it inherited defects from XP along with its own issues).

It is not an understatement to say that we are already seeing open source software competing quite well with proprietary models in some areas, and perhaps even having a major pie in some areas. In that regard the speaker suggested an interesting paper [PDF] on MIT website. The paper is a case study (year 2000) on Apache web server development compared with proprietary alternatives.

Some points I felt are interesting from the study (authors warned that not to infer too much from one case study, but provided some hypothesis based on their research):

  • In open source model code is written with more care and creativity, because developers are working only on things for which they have real passion.
  • Participation of wider development community helped significantly in the defect repair (that explains the defect density claim mentioned above).
  • The amount of code produced by the top Apache developers versus the top developers in the commercial projects --- the Apache core developers appear to be very productive, given that Apache is a voluntary, part time activity and the relatively lean code of Apache.
  • Code ownership is more a matter of recognition of expertise than one of strictly enforced ability to make commits to partitions of the code base.
  • While the user perceived defect density of the Apache product is inferior to that of the commercial products, the defect density of the code before system test is much lower. This latter comparison may indicate that fewer defects are injected into the code, or that other defect-finding activities such as inspections are conducted more frequently or more effectively. It is also possible that the diversity of backgrounds of the developers participating in the OSS project have reduced the probability of defects.
  • In successful open source developments, a group larger by an order of magnitude than the core will repair defects, and a yet larger group (by another order of magnitude) will report problems.
  • OSS developments exhibit very rapid responses to the customer problems. This goes along with another hypothesis that in many OSS developments the developers are also the users of the software.

That was research done back in 2000, many of these are more true, now in year 2009. What do you think?

Tags: ,

Test Automation Revisited

More than an year ago I published my experiences with Selenium and Watij. At that time I was actually not sure whether Selenium and Watij were ready for prime usage, especially for testing on IE7 (a requirement for my app). So fast forward to now, I'm lot more satisfied with what I'm seeing ..

Both the tools have got more recent versions out compared to what I used earlier. Started to work on Selenium RC. I was glad a lot of stuff that was a bit shaky in my earlier experience is more polished now. Just when I was thinking that I'm proceeding quite well with Selenium RC, hit with an issue where Selenium loses handle to the main window if the (child) popup window posts data and the main window refreshes based on that post. Logged a critical issue at Selenium's Jira, still awaiting for some action on that (I wish I had some time to look into that and provide a patch, if required).

After that, started to look for alternatives -- having used Watij earlier, prompted me to give it another try. I was pleasantly surprised by the ease at which I could proceed this time, including the issues that I was having earlier with modal dialogs. Here are some of the comparisons:

  • Watij's API is more Object-oriented than that of Selenium's, which is perhaps more functional in nature.
  • While using Selenium I had to use waitForXXX calls to wait till the page gets loaded, or for the popup to get loaded, etc. But with Watij the tool is "smart" enough to figure when the page load completed so as to proceed with the subsequent calls. Having said that there are still some areas where some sort of wait is required, but I believe that is based on the application functionality that is under test.
  • Selenium injects Javascript into the pages of application under test. It also requires Selenium server running (only proxyInjection mode worked for me). Watij on the other hand doesn't need a server. It requires a dll (yikes!, but that's not that bad) to be placed in your System32 folder of Windows. I found Watij to be a bit more robust and found that behavior was consistent when compared with Selenium.
  • A drawback for Watij is it works only for IE. That works fine for me as that is our primary requirement. If you have to test under different browsers than Selenium is perhaps the way to go.
  • With Selenium IDE, the tests can be recorded using a Firefox plugin. There is no such recording capability with Watij.
  • Watij doesn't have a Selenese type of "language" to write the test cases outside of using a programming language (like Java). Needless to say that this kind of notation would be useful for non-technical people. Having said that it is not that difficult to write your own version of Selenese kind of notation.

With all those observations, Watij was what I chose to proceed with. Combination of the robustness of Watij and the critical issue that I was hit with while using Selenium (described above) made me to move in the direction of Watij.

Tags: ,

Watij and Selenium: Are they ready for prime usage?

Note: Published an update here.

I have been looking out for some good open open source automated functional testing tools. After some bit of research and based on some recommendations I went on to try Watij and Selenium.

Watij:
Based on popular ruby based Watir, I was quite impressed by its API. It has got good little documentation on its site and on wiki, just enough for you get started and do some reasonable testing. You can write the tests using the popular JUnit framework, which is a big plus.

It all worked fine, in fact, I could automate quite a few screens in an application until I hit a roadblock :( Whenever it encounters modal dialog windows it just stops working. Some how it seems not recognize them, and waits for the screen to load even though it has already been loaded.

Wondering how other people working with Watij working around this limitation. It has been a while since they made a release after 3.1.0. I hope the support for modal dialog windows would be functional in the next release.

Selenium:
After that experience with Watij, I started looking at Selenium. For some reason, I did not have that much luck, not even as much compared with Watij.

After starting up the Selenium server and try to hit a web page I've received a 404 error on page. Based on the documentation on the site, I have started using *iehta instead of *iexplore and the 404 went away. The joy was short lived as I started getting windows dialog messages saying a "script is trying to run, do you want run it?" (not the exact verbage). I had to keep clicking Yes on it to move forward.

However, with bit more researching I found a way to resolve that. This time started the server in the mutiwindow mode (with -multiWindow option) the application started running on a different window than that of the Selenium browser.

That is as far as I can go with this at this point. As I start with a page on the application I see dozens of browser windows started popping for the same page. Each of this page shows up a confirmation popup saying that the web site you're on is trying to close the window, do you want to close the window? Not sure why Selenium wants to close the browser as soon as it opens this very first page! This was working some time ago when I was on IE 6, now that I'm on IE 7 I find this issue.

Conclusion:
All in all, I found more success with Watij than Selenium (on IE 7). I'm sure there are many people out there using both these APIs. I see there are many open source projects using Selenium. I'd be interested in knowing what they think about these two APIs. Which one they liked better and why? Are they ready for prime usage?

Tags: , , ,