Category Archives: ruby

Green Bar in a Ruby IDE!

The Ruby Development Tool project team, who develop a plugin for Eclipse, have added some great features in their latest release. Most important (to me), is test::unit support, as well as some other features such as code-completion based on configurable templates, and a regular expression plugin.

I’ve been using RDT in Eclipse for a few months, and with this latest release, I’m very pleased to finally have a green bar in my Ruby development. Or more often in my case, a red bar.

Thanks to Glenn Parker for letting me know about this release.

Gerard Meszaros on Computer Assisted Testing

At last night’s Extreme Tuesday Club meeting, Gerard Meszaros had a great way of describing what I do using automated scripts with Ruby combined with exploratory testing. (See my blog post about this here: Test Automation as a Testing Tool.)

As I was describing using a Ruby script to automate some parts of the application I’m testing, and then taking over manually with exploratory testing, Gerard said:

Sounds like you are using your Ruby program to set up the test fixture that you will then use as a starting point for exploratory testing.

I hadn’t thought of what I was doing in this way before, and I like the idea. It is a great way to succinctly explain how to complement the strengths of a testing tool and a brain-engaged tester.

Check out Gerard’s Patterns of xUnit Test Automation site for more of his thoughts on testing.

Automating Tests at the Browser Layer

With the rise in popularity of agile development, there is much work being done with various kinds of testing in the software development process. Developers as well as testers are looking at creative solutions for test automation. With the popularity of Open Source xUnit test framework tools such as JUnit, NUnit, HTTPUnit, JWebUnit and others, testing when developing software can be fun. Getting a “green bar” (all automated unit tests in the harness passed) has become a game, and folks are getting more creative about bringing these test tools to new areas of applications.

One area of web applications that is difficult to test is the browser layer. We can easily use JUnit or NUnit at the code level, we can create a testable interface and some fixtures for FIT or FITnesse to drive the code with tabular data, and we can run tests at the HTTP layer using HTTPUnit or JWebUnit. Where do we go from there, particularly in a web application that relies on JavaScript and CSS?

Historically, the well-known “capture/replay” testing tools have owned this market. These are an option, but do have some drawbacks. Following the home brew test automation vein that many agile development projects use, there is another option using a scripting language and Internet Explorer.

IE can be controlled using its COM interface (also referred to as OLE or ActiveX) which allows a user to access the IE DOM. This means that all users of IE have an API that is tested, published, and quite stable. In short, the vendor supplies us with a testable interface. We can use a testable interface that is published with the product, and maintained by the vendor. This provides a more stable interface than building one at run-time against the objects in the GUI, and we can use any kind of language we want to drive the API. I’m part of a group that prefers the scripting language Ruby.

A Simple Example

How does it work? We can find methods for the IE COM On the MSDN web site, and use these to create tests. I’ll provide a simple example using Ruby. If you have Ruby installed on your machine, open up the command interpreter, the Interactive Ruby Shell. At the prompt, enter the following (after Brian Marick’s example in Bypassing the GUI, what we type is in bold, while the response from the interpreter is in regular font).

irb> require ‘win32ole’
=>true
irb> ie = WIN32OLE.new(‘InternetExplorer.Application’)
=>#<WIN32OLE:0x2b9caa8>
irb> ie.visible = true
#You should see a new Internet Explorer application appear. Now let’s direct our browser to Google:
irb> ie.navigate(“http://www.google.com”)
=>nil
#now that we are on the Google main page, let’s try a search:
irb> ie.document.all[“q”].value = “pickaxe”
=>”pickaxe”
irb> ie.document.all[“btnG”].click
=>nil

#You should now see a search returned, with “Programming Ruby” high up on the results page. If you click that link, you will be taken to the site with the excellent “Programming Ruby” book known as the “pickaxe” book.

Where do we go from here?

Driving tests this way through the Interactive Ruby Shell may look a little cryptic, and the tests aren’t in a test framework. However, it shows us we can develop tests using those methods, and is a useful tool for computer-assisted exploratory testing, or for trying out new script ideas.

This approach for testing web applications was pioneered by Chris Morris, and taught by Bret Pettichord. Building from both of those sources, Paul Rogers has developed a sophisticated library of Ruby methods for web application testing. An Open Source development group has grown up around this method of testing first known as “WTR” (Web Testing with Ruby). Bret Pettichord and Paul Rogers are spearheading the latest effort known as WATIR. Check out the WATIR details here, and the RubyForge project here. If this project interests you, join the mailing list and ask about contributing.

*This post was made with thanks to Paul Rogers for his review and corrections.

Visible Tests and Web Testing with Ruby

Announcing a new project and blogging my notes for the Scripting Web Tests Tutorial which I prepared for XP Agile Universe:

Visible Tests

Automated test results are often difficult to communicate to business stakeholders on a project. People frequently ask me about the visibility of tests on a project saying: “The customer doesn’t understand JUnit tests” or “The JUnit and FIT tests mean very little to the customer. The customer doesn’t usually think in terms of green bars, or table-driven tests like developers and technical testers do. They believe we are testing, but they can’t see the tests run in a way that they relate to. How do we raise the visibility of tests?”

The WTR IE Controller has an advantage in test visibility since the testable interface we use is about as close as we can get to interacting with a program the way an end-user would. Business stakeholders understand testing at the GUI layer because that’s the way they relate to the program. These tests can be played back to the business stakeholders so they can see the tests run in a way they relate to. Technical people on a project team relate to the program at various layers and sometimes we forget about the business problems we are solving. Techies look at the backend and the front-end, business users usually only see and understand the front-end of the application. The preferred method of interaction with the program can differ between the groups.

Business stakeholders can watch the tests play back on a computer which provides rich, visual feedback. If the test fails, they see it fail. If it passes, they see it pass while exercising the application in a way that they would themselves. While they have faith in the ability of the technical members of the team, and will accept the testing numbers and their word, nothing replaces the assurance they get from seeing it work, and from manipulating the product themselves. With IE Controller tests, the customer can see if the tests pass or fail by watching how the application works in the Internet Explorer browser. Tests can also be designed to mimic business interaction, and provide results logging that non-technical project stakeholders can understand. The tests will demo the business solutions that the customer needs the application for in the first place. These kinds of automated tests help provide more assurance in what the technical members of the team are doing.

At an end of iteration meeting, the application can be demonstrated using these tests if the group wanted to give a quick demonstration.

Announcing WATIR

The latest version of the Web Testing with Ruby project has begun under the WATIR project spearheaded by Paul Rogers and Bret Pettichord. I’m excited by the prospect of a more sophisticated Open Source web testing solution using Ruby. I’ve had success with the IE Controller tests I’ve written to date, and look forward to using a more complete solution.

A Testing Win Using Ruby

Brian Marick has written lately about Exploratory Testing using Ruby. I decided to try this out on a web application using Ruby and the WTR(Web Testing with Ruby) IE Controller. I’m not at the level with Ruby yet where I can test an application using the Ruby Command interpreter like Brian does, but I thought I could write a short script and run it in an Exploratory manner. I decided to automate steps that would be tedious, time consuming and as a result, very much prone to error if run by a human. After a few minutes, I felt I had a script that was testing the application in a way I hadn’t tested manually. I ran it and watched it play back on my monitor.

When I ran it for the first time, the application responded in a strange manner. I had seen this behavior a couple of days previously when testing it manually, but couldn’t repeat it. When I re-ran the script, the behavior occurred again. I edited the script to hone in on the problem area and was able to repeat the problem each time. I edited it again to narrow down the cause, and I talked to the developer about the behavior. He had a suggestion to check a similar action, so I edited the script to try out the developer’s idea. In the end, I probably spent about a half hour on script development.

In a short period of time, I was able to track down a defect using Ruby that I had been trying to replicate manually. I’m confident that I would have spent a much longer time replicating the problem manually, and probably would have narrowed it down much later in the release when I was more familiar with the product. The test script that capitalized on the computer’s strengths over my human abilities run in a “what would happen if” scenario really paid off.