This article is the first of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article explains our test automation goals. Then rationale is given for the tools we use to achieve those goals.
The second article covers installation and configuration of these tools. It is aptly titled EE QA: Install and Configure Selenium, Java, Eclipse, and TestNG
. We work in a Windows environment so the article is written from that perspective.
The third article, EE QA: Developing and Running Automated Tests For Our Website
, covers our approach to test case design and implementation, as well as our runtime environment. It contains some good design and implementation practices.
Our Test Automation Goals
In order to continually improve our member’s experience, our engineering teams focus on quickly delivering high quality, high value enhancements to our web site. To facilitate these activities we use the Agile Development
process. In Agile, testing is an ongoing activity of each sprint. Within a sprint our QA folks manually test new features and also write new automated test cases for those features.
Our automation goals are simple. Within a sprint, we need to be able to quickly and efficiently write meaningful automated tests that cover the new features. We also need to be able to easily add those test cases into the right level of our continuous integration test environment. Additionally, we need to have a body of reliable, automated regression test cases
that cover existing functionality. This allows us to spend current sprint time completing the sprint testing tasks. In other words, our automated tests need to handle the bulk of our functional
regression testing. The tools and practices described in these articles help us achieve these goals.
As mentioned in the goals section, we are testing a website so obviously that’s going to influence some of our tool choices. Another reason for using this toolset is some tools are also used by our developers. Using the same tools gives the whole engineering team a lot of shared experiences and allows us to help out each other with questions and concerns. In other words, it’s our own community, a mini version of what Experts Exchange offers our members.
encompasses a number of packages. We use Selenium WebDriver
and Selenium IDE
. WebDriver provides the ability to programmatically drive browser actions and investigate browser objects. WebDriver does just what its name says, it drives web browsers. In order to do this driving, WebDriver needs to be able to find page elements and perform acts on them.
The source of a web page can be a complicated beast. Inspecting the page source to identify a button to click or a text field to fill in can be difficult and error prone. Selenium IDE provides the ability to capture a ‘path signature’ of a web page element
When we are developing automated tests we will use the IDE to generate lines of java code that can be inserted into a test case. The process is simple. We go to a web page we want to write automated test cases for. Launch Selenium IDE (#1). Click on the page elements (#2). The IDE will generate java code that represents the element and an action on the element.
Here’s a simple example of java code created by IDE. This is for clicking on the Login button on our home page.
driver.findElement(By.cssSelector("a.login-box > span")).click()
We insert the code generated by IDE into our WebDriver test cases. Sometimes we have to tweak the IDE generated code to get what we want, but we still find IDE very helpful.
The WebDriver test cases mostly consist of a series of finding page elements, taking actions on them and then verifying the results. EE QA: Developing and Running Automated Tests For Our Website
gives more details on our process. For us Selenium’s WebDriver and IDE are a powerful combination for web site test automation.
Other reasons for choosing this combination include Selenium’s good track record, its many sponsors, and its use by many companies, as well as it being zero cost.
We use Java
as our programming language for a couple important reasons. It is supported by Selenium. Our experienced programmers know Java and, with all the online and in-house help available, it’s easy for our new automation programmers to learn it. Selenium does support other languages, but since our team is most proficient in Java, we use it.
We use Eclipse
for a lot of the same reasons we use Java. Our product and test developers are familiar with it. It supports Java. It has a plug-in for TestNG. It’s free and it works well for what we need to get done. Although these articles don't cover source code control, another reason for using Eclipse is it has a plug-in for Git.
We use TestNG
because we want an easy to define automation run time environment for our java test cases. TestNG gives us the ability to apply tags to each test case. The tags go in the test case source code. We can then tell TestNG to find and run all tests bearing a specific tag. This can be done without having to go in and muck with the test cases. We can do things like run all ‘critical’ tests found in <these> source files. TestNG has additional tagging features and a logging feature we make use of. Like our other tools, it is zero cost.
This is a powerful, zero cost, well supported, toolset that allows us to quickly write high-value automated test cases. It provides a good debugging environment. Our source code control is integrated into the development environment so it becomes second nature to practice good version control. We can dynamically define the runtime environment as well as the set of tests to be executed in that environment. This combination of tools facilitates the achievement of our automation goals and helps us deliver a high quality product for everyone who visits experts-exchange.com
Second Article: EE QA: Install and Configure Selenium, Java, Eclipse, and TestNG
Third Article: EE QA: Developing and Running Automated Tests for our Website