Parallel Selenium testing

Parallel Selenium testing

In this post we publish an English translation of Dima Yakubosky’s first talk on SeleniumCamp 2012 conference which took place in February in Kiev, Ukraine. Text precedes a slide it is related to. Here it is …

What I will talk about?

I will tell you about the standard instruments, with the help of which we can execute tests in parallel – TestNG and Selenium Grid. And about a real system that can launch tests in different browsers, parallel to each other.

I will show you some problems of executing the same test in parallel – and a few ways to fix it. Also we will talk about when it is appropriate to increase the amount of parallel threads you are using.
Read the rest of this entry »

Handy Selenium reports

Handy Selenium reports

In this post we publish an English translation of Dima Yakubosky’s second talk on SeleniumCamp 2012 conference which took place in February in Kiev, Ukraine. Text precedes a slide it is related to. Here it is …

It’s nice that sophisticated tools, like Selenium, allow us to quickly and effectively execute a large chunk of routine work. For example, let’s check if all of 20 important elements are on this page. But, all this speed and thoroughness would not be useful to anyone unless tests could inform you of their results.

The easier the report is to understand, the less time you need to spend on finding the cause of the error when it finally appears.

In the Selenium manual it is written that most testers will sooner or later end up developing their own reports instead of or in addition to those reports which are presented by frameworks. It would be nice to know if any of you use your own, personal reports.
Read the rest of this entry »

Selenium reports – our take
by Igor

Selenium reports - our take


We decided to improve the reports our Selenium tests generate. We wrote about our reports in this post and here is a sample report. Why did we begin to dislike the current design?

First – the mere sight of the report should be aesthetically pleasing, because we have to work with them every day.

Second – we would like to make it easier to view screenshots. In the current version they were just opened in a new tab in the browser, making it difficult to view and work with the report as a whole.

Well, thirdly – we finished Nerrvana and created another web application, which is extensively tested in Nerrvana on all environments it currently supports. By environment we mean the combination of an operating system and a browser of specific versions. For example, Win 7 + Firefox 9 and Win XP + Firefox 9 – are two environments. It turned out that viewing a report for each environment is uncomfortable. We want to generate reports in XML, and then combine them into one report. Then we can immediately see errors that occurred in all environments.

I made a few designs, which I want to share with you. Dima is preparing a presentation for the Selenium Camp 2012, where he will talk about how to build such hierarchical reports and parallel testing. We want to implement at least some of the ideas, that we think are most urgently needed, and be able to show them during Dima’s conference talk.
Read the rest of this entry »

Nerrvana launched

Nerrvana launched

It has been three years since we started working on Nerrvana and finally we are ready to launch it. While we are setting our first server in Dupont Fabros facility in Chicago, IL, USA please free to fill a short form and join our beta program.

We do not plan to have more than ten companies to participate as we would really like to have time and understand what needs to be changed to make Nerrvana a problem solver for your business.

We are just back from SVOD 2011 conference which was held in San Francisco 29-30 Nov 2011 where we were presenting Nerrvana. You can watch uncut version of a presentation on YouTube.

We also just launched a dedicated to Nerrvana blog.

Coverage matters by Igor

Coverage matters

The sky will collapse on the ground,
The grass will cease to grow —
He will come and silently fix everything,
The man from Kemerovo.

Boris Grebenshchikov,
“Man from Kemerovo”
(translated by Andrei Soroker)

Let me remind you that we are testing only one object of Recruiting application – Employment Websites. For clarity and convenience, I created a chart as an image map, and linked it with screenshots so that you do not need to go anywhere to understand what we are talking about. Now testing is executed in a sequence (omitting login and logout):

Login
Click tab 'Employment Websites'
click 'Save & New'
'Employment Websites' search results
Add 'Employment Website' page
click 'New' button
click a record
click 'Save' click 'Delete'
Show 'Employment Website'
click 'Save & New'
click 'Save' click 'Save'
click 'Edit' click 'Clone'
Clone 'Employment Website'
Edit 'Employment Website'
click 'Save & New'

Figure 1 – Employment Websites object (flowchart elements are clickable)

1. checkIsRecordSavable() – picture
2. checkSequence() – picture
3. checkAllElements() – picture

In an academic world such an article would probably have called ‘Some aspects of the sequence of software testing’, well, we just ask – Do you see something illogical or unusual in such an approach? Maybe something is missing or something is excessive? Can’t you see? Or do not want to think? Then click on the link below and read.
Read the rest of this entry »

Selenium and non standard Firefox profiles
by Dima

Custom Fox

One day you will need different browser profiles for testing. For example, you may want to run tests with JavaScript switched off or test a web site with a certain browser plug-in turned on.

User scripts developers (userscripts.org, Greasemonkey) could be interested in running Selenium tests with their script loaded into browser. If you did not work with user scripts – have a look. They can improve your experience with some sites or with browser in general. For example you can add a button which allows downloading YouTube video in different formats.

Another situation is when your application uses CAPTCHA for login. For example such famous Russian sites as dirty.ru and habrahabr.ru use CAPTCHA on login.
What can be done to pass CAPTCHA automatically? On some sites there is an option ‘Remember me’ which allows us to avoid CAPTCHA but Selenium, by default, creates an absolutely new profile in Firefox without any cookies.
Read the rest of this entry »

Creation – Day Two by Dima

Creation - day 2


A lot of time has passed since we talked about our first working version of the SalesForce tests, version 54. We don’t devote as much time as we would like for developing tests, but still the current version has gone a long way from version 54.

We did write about it one way or another in our blog, but now it is time to put some milestones in its’ way.
Read the rest of this entry »

Se-nimatograph by Dima


The Selenium Driver (for Java) has four functions to work with screenshots:

- captureScreenshot(filename)
- captureScreenshotToString()
- captureEntirePageScreenshot(filename, kwargs)
- captureEntirePageScreenshotToString(kwargs)

Let’s work out how they differ from each other, and in what cases they are best to use.

The function captureScreenshot(filename) makes a png format screenshot and saves it to the file. The big disadvantage of this function is that it saves a standard screenshot, you would get the same result by pressing “Print screen” button.

This means that if your browser window is minimized or not maximized then the page or a section of it will not be seen on the screenshot. Even if the browser is opened completely, but the page is too long (or too wide) and you have to scroll through it, again only the visible part of the page will be on the screenshot.
Read the rest of this entry »

Review – Without waiting by Dima

In the beginning, the short announcement – we started one more blog. Why? Details are here.

Our test, which really isn’t that big, works for about 5 minutes. Just to remind you, for now we are testing two basic Recruiting application objects – Websites and Accounts. Therefore, the Account object really doesn’t exist, we just bundle up functionality related to logging in and out into application.

For the record: our Selenium Grid is located in Ukraine, the SalesForce application is in the U.S.A, and our internet connection is – 2 Mb ADSL from Ukrtelecom.

Besides Website, the Recruiting application contains five objects which are more complex than Website. All our tests will eventually run in parallel, but still the test runs’ time could be unsatisfyingly long. Now is the time to think about cutting it down.

One of the ways to shorten the run time is not to wait for the whole page to load completely. Sometimes everything needed for the test loads quickly and the best part of the time goes towards loading some pictures.

The SalesForce login page demonstrates this very well. The login and password fields load extremely quickly but the flash to the right loads slowly.

We will use this page for our experiments.

Read the rest of this entry »

Questions to the reader by Alex

As you already should know, we are mastering the uneasy field of automatic testing with the use of the Selenium Grid. We report our progress regularly in our blog. One of the beautiful things that makes testers very happy is tests being run on time unattended. We also want a small piece of joy for ourselves and decided to make a sheduled launcher.

Doing something for yourself is a dream for any professional, and we are among them. The feeling of power puffs up and overshadows the fact that we have no idea how to build a proper interface for it. One thing is for sure the process of making the schedule must be very simple, intuitive and allow to deal with most of the real life scenarios easily. Because our experience in Selenium testing isn’t very extensive, we could make an interface which suites only us. On the other hand we became interested how the people who have used Selenium for a long time with real applications. Then we came up with an idea – ask the reader.

Read the rest of this entry »

Looking for something? Visit the archives.