Test anatomy by Igor

Unfortunately for you, dear readers, we do our job much faster than we write about it. Where to find watches with sixteen hours dial which is used, from what we know, by all who write a lot about staff they do.

I am writing now and catch myself on a thought that I describe our understanding about three months ago. Have I already heard it elsewhere? It seems like it was in planetarium, in the childhood? About light of far stars which reaches us through millions of years …, and for some reason advertising slogan is on the tip of my tongue “Remain with us another three months to learn about our today’s understanding of testing, and you will not regret!”

Our task for today is to operate such an easy-uneasy organism of Recruiting society as ‘Employment Website’.

In the course of operation we will write the detailed instruction on testing of each part separately and otherwise. So let’s sharpen our scalpels, gentlemen, anesthetize our patient and start after pray!

Fig. 1 Patient

Read the rest of this entry »

Creation – Day 1 (early morning)
by Dima

As a morning exercise, we propose to run first workable revision of Java tests (Revision 54). We assume that you have installed environment (Selenium and Java). Our short guide on how to do this can be found on page ‘Environment installation’.

It is important to have Recruiting application created on SalesForce platform (in fact, you only need to create Recruiting Web Site object in it for now).

While creating Recruiting application it is good to follow book instructions precisely. Otherwise tests will bring many errors and it will be hard to figure out what was the cause, in this early version of Selenium tests on Java. Current version of the same tests will explain all errors in a user friendly way but we will discuss it later in this blog).

If you do not have Recruiting application, you will be able to look at code and resulting log file only (link to log file will be provided in next post). But if you are like us – determined and hardworking – and have application, you only need to configure tests and Selenium launchers. All information below is for you to do it quickly and painlessly.

Firstly, we extract revision 54 from SVN repository…

Fig. 1 http://deepshiftlabs.com/svn/tests/trunk/salesforce/java

Read the rest of this entry »

Creation – Day One by Dima

And I said, Let there be code: and there was code.

If The Creator of our blue planet completed his huge job in a week I would suppose he had had a very detailed specification.
Its existence is not reflected in canonical texts but it could not have been otherwise – it is hard to go and do something out of thin air.

Before we started writing code to test Recruiting application we spent awful deal of time by clarifying this task. Results we want to obtain, what exactly, in which order and how – all of these questions were answered and morphed from a jumble of ideas, a prebiotic soup, so to say, into a quite articulate system. Remaining was ‘next to nothing’ – come back to earth and write it down.

Let’s start with some basic things – functionality which will be used by higher level classes.

Figure 1

Read the rest of this entry »

Creation – Day One (evening)
by Dima

If you have following values set in Setting.java:

	public static final Boolean LOG_INFOS = true;
	public static final Boolean LOG_VERBOSE = true;

, practically any action performed by tests will be reflected in logs.

Some examples.

11:25:12,156 INFO pool-1-thread-1 utils:info:? - (V)Tab opening started
11:25:13,453 INFO pool-1-thread-1 utils:info:? - (V)Tab opened: title is Employment Websites: Home ~ Salesforce - Developer Edition

Read the rest of this entry »

A shameless plug by Igor

We finished a series of posts describing our first working version of tests on Java and needed a break to decide what to publish next.
At this stage our writings are memoirs – we describe what we have done using our internal forum and SVN. Very soon a publication process will go in sync with code writing. Please do not expect many posts from us. We are also working on our customers systems and we build our own bootstrapped web service but process will continue until all Recruiting application will be tested from start to finish.

If you would want to ask us about PHP version of test we would tell you that Alena lost interest and left Deep Shift Labs ‘scene’. As French say “c’est la vie”.

The PHP version in our public SVN repository is working and roughly functionally matches revision 54 in Java we described in “Creation – Day 1″. We will try to continue developing it further just because we love PHP.

So we are planning to publish following posts next.

Read the rest of this entry »

Review – What is a ‘fatal’ in tests?
by Dima

This post is answering a question “Which events should stop testing process?”.

This time we are giving our list at the beginning of the post and explain how it was built below.

Here it is a main list of fatal errors we compiled:

1. No Selenium server
2. Unsuccessful login
3. Missing element
4. Can’t save with known-good-values (KGV)
5. Wrong page

Most of these errors will occur at the very beginning of testing. If we will not face them – I think, tests will run just fine.
Despite of the fact that this list covers most of the situations when tests should be stopped I added two more types into additional list:

6. Unsuccessful attempt to assert a field value
7. Too many non fatal errors

Read the rest of this entry »

Review – Is ‘Login’ an object or not?
by Dima

From one side a “Login” is a Recruiting application object, but on the other hand objects need to use the “Login” object too. What do we do?

It will be more accurate to talk about the “Account” object, where the “Login” is its functional part. When we were building the Recruiting application we did not create the “Account” object and naturally we did not analyse this object at all. Well, we can say now that the “Account” object is a part of the SalesForce platform and we cannot write a full suite of tests for the “Account” object because many functionalities are beyond our exercise scope.

However, if we will include the “Account” object into our test suite as a separate object we can draw parallels between its actions and actions that other objects can perform.

Account registration is by nature an ‘add’ action similar to the Recruiting Web Site ‘New’ button action. Editing and deletion of an account is usually hidden in Preferences or Setting somewhere in UI.

Full suite of tests for the “Account” object in common sense should include – registration, login, logout, account modification (including password recovery) as well as account deactivation (deletion).

Read the rest of this entry »

Review – It is all about logs
by Dima

Now that the classes that can test SalesForce Recruiting application mostly fulfil their duties what are we missing?
Basically, we answered this question at the end of this post:

Creation – Day One (evening)

1) we have no fatal errors – no matter what happened tests do continue rolling new errors as a snow ball;
2) logging – we have all information in logs but in a very raw form which is hard to process further;
3) it is possible to get lost in a code jungles – comment it;
4) we did not use a possibility to run tests in parallel – this is because we have only one test

and even managed to discuss first problem.

It’s the perfect time to begin with the second question – collection and processing of information which is gained during the testing.

The question, how should reports look, and therefore, what do logs contain has worried us for quite a while.
It is very pleasant to see that your own tests have been doing a whole bunch of routine jobs on testing elements, but who will know of this? It would help if tests could create a report on the work they have completed and the amount of errors they have found, not a text file that looks like gibberish.

In version 54, which we have looked at, logs contained nearly all of the information which could be used to analyse errors. But it was presented as a text file, and honestly speaking, it would be more likely to scare than to excite. Other than text logs, screenshots were made, and before the test there would be an option to set certain level of messages that would be written down in the log (info, warn, error). You can read more about it here.

We are going to answer some questions, we had since, related to logs today.

Read the rest of this entry »

Questions to the reader by Alex

As you already should know, we are mastering the uneasy field of automatic testing with the use of the Selenium Grid. We report our progress regularly in our blog. One of the beautiful things that makes testers very happy is tests being run on time unattended. We also want a small piece of joy for ourselves and decided to make a sheduled launcher.

Doing something for yourself is a dream for any professional, and we are among them. The feeling of power puffs up and overshadows the fact that we have no idea how to build a proper interface for it. One thing is for sure the process of making the schedule must be very simple, intuitive and allow to deal with most of the real life scenarios easily. Because our experience in Selenium testing isn’t very extensive, we could make an interface which suites only us. On the other hand we became interested how the people who have used Selenium for a long time with real applications. Then we came up with an idea – ask the reader.

Read the rest of this entry »

Review – Without waiting by Dima

In the beginning, the short announcement – we started one more blog. Why? Details are here.

Our test, which really isn’t that big, works for about 5 minutes. Just to remind you, for now we are testing two basic Recruiting application objects – Websites and Accounts. Therefore, the Account object really doesn’t exist, we just bundle up functionality related to logging in and out into application.

For the record: our Selenium Grid is located in Ukraine, the SalesForce application is in the U.S.A, and our internet connection is – 2 Mb ADSL from Ukrtelecom.

Besides Website, the Recruiting application contains five objects which are more complex than Website. All our tests will eventually run in parallel, but still the test runs’ time could be unsatisfyingly long. Now is the time to think about cutting it down.

One of the ways to shorten the run time is not to wait for the whole page to load completely. Sometimes everything needed for the test loads quickly and the best part of the time goes towards loading some pictures.

The SalesForce login page demonstrates this very well. The login and password fields load extremely quickly but the flash to the right loads slowly.

We will use this page for our experiments.

Read the rest of this entry »

Looking for something? Visit the archives.