Review – Without waiting by Dima

In the beginning, the short announcement – we started one more blog. Why? Details are here.

Our test, which really isn’t that big, works for about 5 minutes. Just to remind you, for now we are testing two basic Recruiting application objects – Websites and Accounts. Therefore, the Account object really doesn’t exist, we just bundle up functionality related to logging in and out into application.

For the record: our Selenium Grid is located in Ukraine, the SalesForce application is in the U.S.A, and our internet connection is – 2 Mb ADSL from Ukrtelecom.

Besides Website, the Recruiting application contains five objects which are more complex than Website. All our tests will eventually run in parallel, but still the test runs’ time could be unsatisfyingly long. Now is the time to think about cutting it down.

One of the ways to shorten the run time is not to wait for the whole page to load completely. Sometimes everything needed for the test loads quickly and the best part of the time goes towards loading some pictures.

The SalesForce login page demonstrates this very well. The login and password fields load extremely quickly but the flash to the right loads slowly.

We will use this page for our experiments.

Questions

If you have some experience in using a client side cache during testing – please share with us. It would be interesting to know if server or application can be configured to send special ‘testing mode’ headers forcing client to cache not participating in testing elements.

If we could start entering the login and password without waiting for the page to load completely we would save about 5 seconds per each login. This is an obvious plus. What disadvantages could be in this approach? First thing that comes to mind, we could end up having screenshots of a semi-loaded page. Second, some important JS, that dictates the pages behaviour, could end up not fully loaded. If you have a solution, don’t be shy, share it with us.

Question

From your point of view what other cons for test to start without full page load?

On the SF login page we haven’t found any important JS at the end of the page.

Anyway, negatives aside, it is a real deal. Let’s talk about an implementation.

Firstly, we are going to cut out Selenium functions which are waiting for the page to load. These are basically functions that end with ‘AndWait’, function waitForPageToLoad() and open(). In our tests we only use waitForPageToLoad() and open(). Open(), in fact, implicitly calls waitForPageToLoad() so we only have to replace waitForPageToLoad() which will wait for the needed elements to load.

How do we know that all the needed elements have loaded? The first and most obvious that comes to mind is to wait for the last element to load. For example, on the SF login page (Fig. 1) we only need two elements: Username and Password.

Figure 1

So we can run isElementPresent(“password”) until the element appears. Everything seems simple, but this method has a big disadvantage – what if SalesForce decides to switch the placement of the Username and Password fields? In this case the test will start entering the username before it has even loaded.

Question

What do you think? Is it necessary to test the elements position related to other elements?

We came to the conclusion, that the only proper solution is to wait for all required elements to appear on the page. With the login page it could be done in this fashion:

loaded = false;
while (!loaded) {
	loaded  = ( isElementPresent("password") &&  isElementPresent("username") );
}

But this approach isn’t that good:
-it is necessary to send a command (actually two commands!) to the RC and wait for an answer.
-if the elements don’t appear the cycle will never finish.

We could add a timeout and wait a certain time before every attempt, but we found a more interesting way to do it. Instead of us implementing waiting timer we moved it to JavaScript, which will run on the RC side.

Creators of Selenium thought about us and wrote the following function waitForCondition(condition, timeout) – thanks to them! Condition – is a small JS script. This script will be executed by the RC and if it will return true, waitForCondition() will return true too. If this script will not return true in a specified timeout, defined in milliseconds, the function will return false.

All that is left to prepare is the condition parameter for Javascript, which will return true if all the needed elements are on the page.

As a result, this function was added to the commonActions.java file:

public boolean waitForListOfElements(ArrayList <String> locators, String timeout){
	Boolean result = false;
 
	String tempScript = "var result = false; ";
	if (locators.size()>0) {
		tempScript = tempScript+ "result = selenium.isElementPresent('"+locators.get(0)+"');";
	}
	for (int i=1; i<locators.size(); i++){
		tempScript = tempScript+ "result = result && selenium.isElementPresent('"+locators.get(i)+"');";
	}
 
	Event event = startEvent("waitForListOfElements", tempScript);
 
	result = waitForCondition(tempScript, timeout);
 
	event.setRealValue(result.toString());
	closeEventOk(event);
	return result;
}

To check that the Username and Password fields are loaded, the function call will look like this:

ArrayList <String> locatorsList = new ArrayList <String>();		
locatorsList.add("username");
locatorsList.add("password");
waitForListOfElements(locatorsList, Settings.TIMEOUT);

The condition that will be checked out on the RC side will look as shown below:

View Code JAVASCRIPT
{
var result = false; 
result = selenium.isElementPresent('username');
result = result && selenium.isElementPresent('password');
}

Is that it? We thought so too. In fact we get to the login page with the help of Seleniums’ open() command, but, sadly, it already contains the waitForPageToLoad() call and nothing can be used instead of open(). That is, willing or not, open() waits to the victorious end.

So how do we force it to end faster, so we are able to use our new function?

As it has appeared, very simply. waitForPageToLoad() waits for the page load with a timeout milliseconds (driver property). It’s enough to give it a minimum timeout value – 1, for example, and the open() function will be a problem no more!

This is how it looks in commonActions.java:

seleniumInstance.setTimeout(Constants.MIN_SELENIUM_TIMEOUT);
openUrl("/");
seleniumInstance.setTimeout(Settings.TIMEOUT);

Of course, in this case the open() function will generate an exception, but, if you remember, in our tests all Selenium drivers’ functions are wrapped with exception handlers and that’s why nothing bad will happen.

This will be our small know-how exception. You might have noticed it in the report examples about testing Website object contained in this post and shown below in Fig. 2.



Figure 2 (Click picture to enlarge)



The result – every login saves us 5-8 seconds!

The only thing left to figure out is if there is a point for using this approach on pages without large and not used by tests’ objects. The login page is really the only page with such “heavy” element that tests meet in Recruiting application.

What do you think? Is there a point to include this method into our new code revision?

That’s how it is when things disintegrate
And I don’t know how much longer I can wait
Can’t Wait – Bob Dylan


/pre

Print this post | Home

6 comments

  1. Alexei Barantsev says:

    Почему бы не сделать отдельный профиль браузера с отключенным flash-плагином?

  2. Alexander says:

    loaded = false;
    while (!loaded) {
    loaded = ( isElementPresent(“password”) && isElementPresent(“password”) );
    }

    похоже на опечатку.

  3. bear says:

    Alexei Barantsev:
    В данном примере (только на одной странице flash и он нам не нужен) это, наверное, отличное решение, спасибо за идею.

    Кстати, у меня, наверное, что-то с руками, – я попробовал два способа отключения flash в firefox, и ни один никак не повлиял на баннеры salesforce:
    1) tools -> add-ons -> plugins -> shockwave flash -> disable.
    2) Flashblock.
    Очистил кэш, перезапустил – фиг.
    А вы как отключаете?

    У способа с отключением флэша есть ещё пара недостатков:
    1) Если мы решим запустить те же тесты на опере, IE, других браузерах – придётся создавать профили и для них тоже.
    2) Чтобы отключить “ускорение”, нужно менять профиль. Отсюда вытекает
    3) нельзя использовать “ускорение” избирательно для отдельных страниц. Если на какой-то странице вдруг понадобится скриншот с участием flash, придётся выбирать между скриншотом и ускорением. Хотя Flashblock, наверное, можно настроить на нужные страницы, но это не совсем то.

  4. bear says:

    Alexander:
    Да, действительно, второй раз должен быть “username”, а не “password”.
    Исправил, спасибо!

  5. Alexei Barantsev says:

    У меня в виртуалке флеш просто не установлен, поэтому нет необходимости его отключать :)

    Ещё пара вариантов ускорения:

    1) реализовать блокировку “тяжёлого” контента, например, фильтровать его плагином к браузеру (типа баннерорезалки AdBlock) или поставить прокси между браузером и тестируемой системой.

    2) использовать Selenium 2.0 — в нём WebDriverBackedSelenium, а внутри него HtmlUnitDriver — никакого лишнего контента вообще, даже картинок не будет :) За это придётся, конечно, заплатить определённую цену (те же упомянутые скриншоты и ещё многое другое), но зато ускорение будет огого!

    3) разместить грид в штатах, поближе к тестируемой системе, чтобы быстро грузилось всё (но тут надо мерять — может быть обмен данными с селениум-сервером активный, и тогда вместо выигрыша можно оказаться в минусе).

  6. bear says:

    Alexei Barantsev:
    Selenium 2.0 – очень интересный вариант. Думаю, мы с ним скоро познакомимся поближе.
    Алексей, а вы уже работали с ним?
    Какие изменения нужны в тестах для Selenium, чтобы они смогли работать с Selenium 2.0?