It’s the perfect time to begin with the second question – collection and processing of information which is gained during the testing.

The question, how should reports look, and therefore, what do logs contain has worried us for quite a while.
It is very pleasant to see that your own tests have been doing a whole bunch of routine jobs on testing elements, but who will know of this? It would help if tests could create a report on the work they have completed and the amount of errors they have found, not a text file that looks like gibberish.

In version 54, which we have looked at, logs contained nearly all of the information which could be used to analyse errors. But it was presented as a text file, and honestly speaking, it would be more likely to scare than to excite. Other than text logs, screenshots were made, and before the test there would be an option to set certain level of messages that would be written down in the log (info, warn, error). You can read more about it here.

We are going to answer some questions, we had since, related to logs today.

Read the rest of this entry »

From one side a “Login” is a Recruiting application object, but on the other hand objects need to use the “Login” object too. What do we do?

It will be more accurate to talk about the “Account” object, where the “Login” is its functional part. When we were building the Recruiting application we did not create the “Account” object and naturally we did not analyse this object at all. Well, we can say now that the “Account” object is a part of the SalesForce platform and we cannot write a full suite of tests for the “Account” object because many functionalities are beyond our exercise scope.

However, if we will include the “Account” object into our test suite as a separate object we can draw parallels between its actions and actions that other objects can perform.

Account registration is by nature an ‘add’ action similar to the Recruiting Web Site ‘New’ button action. Editing and deletion of an account is usually hidden in Preferences or Setting somewhere in UI.

Full suite of tests for the “Account” object in common sense should include – registration, login, logout, account modification (including password recovery) as well as account deactivation (deletion).

Read the rest of this entry »

This post is answering a question “Which events should stop testing process?”.

This time we are giving our list at the beginning of the post and explain how it was built below.

Here it is a main list of fatal errors we compiled:

1. No Selenium server
2. Unsuccessful login
3. Missing element
4. Can’t save with known-good-values (KGV)
5. Wrong page

Most of these errors will occur at the very beginning of testing. If we will not face them – I think, tests will run just fine.
Despite of the fact that this list covers most of the situations when tests should be stopped I added two more types into additional list:

6. Unsuccessful attempt to assert a field value
7. Too many non fatal errors

Read the rest of this entry »

We finished a series of posts describing our first working version of tests on Java and needed a break to decide what to publish next.
At this stage our writings are memoirs – we describe what we have done using our internal forum and SVN. Very soon a publication process will go in sync with code writing. Please do not expect many posts from us. We are also working on our customers systems and we build our own bootstrapped web service but process will continue until all Recruiting application will be tested from start to finish.

If you would want to ask us about PHP version of test we would tell you that Alena lost interest and left Deep Shift Labs ‘scene’. As French say “c’est la vie”.

The PHP version in our public SVN repository is working and roughly functionally matches revision 54 in Java we described in “Creation – Day 1″. We will try to continue developing it further just because we love PHP.

So we are planning to publish following posts next.

Read the rest of this entry »

If you have following values set in Setting.java:

	public static final Boolean LOG_INFOS = true;
	public static final Boolean LOG_VERBOSE = true;

, practically any action performed by tests will be reflected in logs.

Time of creation
Quite useless information
level (log4java)
Tells us – it is Verbose log
Preview
One day logs will look like this!
Class and method which called a method of class Logger log4java. Also quite useless information because logger methods are not called directly and do not know a real caller’s class and method
Fill element elementName with validValue – value.
Actions which are expected to complete with some results can return a message (OK) or (ERROR).
In addition, error messages are surrounded with
!!!— error_message —!!!
to meet the eye

Some examples.

11:25:12,156 INFO pool-1-thread-1 utils:info:? - (V)Tab opening started
11:25:13,453 INFO pool-1-thread-1 utils:info:? - (V)Tab opened: title is Employment Websites: Home ~ Salesforce - Developer Edition

Read the rest of this entry »

And I said, Let there be code: and there was code.

If The Creator of our blue planet completed his huge job in a week I would suppose he had had a very detailed specification.
Its existence is not reflected in canonical texts but it could not have been otherwise – it is hard to go and do something out of thin air.

Before we started writing code to test Recruiting application we spent awful deal of time by clarifying this task. Results we want to obtain, what exactly, in which order and how – all of these questions were answered and morphed from a jumble of ideas, a prebiotic soup, so to say, into a quite articulate system. Remaining was ‘next to nothing’ – come back to earth and write it down.

Let’s start with some basic things – functionality which will be used by higher level classes.

Common actions for other classes and particularly wrapper methods for Selenium driver.
They are needed to catch and process Selenium driver exceptions. Class also contains methods for logging with log4x
SF Recruiting application tests
For different parameters we may need to change along the way
Collection of constants used in different classes

Figure 1

Read the rest of this entry »

As a morning exercise, we propose to run first workable revision of Java tests (Revision 54). We assume that you have installed environment (Selenium and Java). Our short guide on how to do this can be found on page ‘Environment installation’.

It is important to have Recruiting application created on SalesForce platform (in fact, you only need to create Recruiting Web Site object in it for now).

While creating Recruiting application it is good to follow book instructions precisely. Otherwise tests will bring many errors and it will be hard to figure out what was the cause, in this early version of Selenium tests on Java. Current version of the same tests will explain all errors in a user friendly way but we will discuss it later in this blog).

If you do not have Recruiting application, you will be able to look at code and resulting log file only (link to log file will be provided in next post). But if you are like us – determined and hardworking – and have application, you only need to configure tests and Selenium launchers. All information below is for you to do it quickly and painlessly.

Firstly, we extract revision 54 from SVN repository…

Fig. 1 http://deepshiftlabs.com/svn/tests/trunk/salesforce/java

Read the rest of this entry »

Unfortunately for you, dear readers, we do our job much faster than we write about it. Where to find watches with sixteen hours dial which is used, from what we know, by all who write a lot about staff they do.

I am writing now and catch myself on a thought that I describe our understanding about three months ago. Have I already heard it elsewhere? It seems like it was in planetarium, in the childhood? About light of far stars which reaches us through millions of years …, and for some reason advertising slogan is on the tip of my tongue “Remain with us another three months to learn about our today’s understanding of testing, and you will not regret!”

Our task for today is to operate such an easy-uneasy organism of Recruiting society as ‘Employment Website’.

In the course of operation we will write the detailed instruction on testing of each part separately and otherwise. So let’s sharpen our scalpels, gentlemen, anesthetize our patient and start after pray!

Fig. 1 Patient

Read the rest of this entry »

Our ‘Analysis’ tree has bloomed and blossomed. QA gurus and wanna be testers stop and relax in its shadow. Here you go – another excursion from Junior Tester Club is approaching. Let’s listen…

‘Wind of Test’ circa 1999-2009 by SalesForce
Donated by ‘Deep Shift Labs Still Pictures’

Read the rest of this entry »

Life of web-tester is uneasy. We direct plays, write in a blog and also participate in the test creation process which is both unstoppable and exciting for us. Sooner or later we started on Stage 3 (with a realistic and conscious attitude). We continue to concentrate on it, and consider what to look at next. What can we see? We can see our application – Recruiting, assembled by manual. It is time now to X-ray it and proceed to Step 1 which was written once on a dark cold windy night on white board surface by a simple black marker.
Let’s begin.

Text

SalesForce allows specifying Text field size when it is created. It can be any size as long as it is within 1..255 characters It is easy to check – enter as many characters as specified for a particular Text field plus one more and test if it can not fit. We know that SalesForce (SF) validation for the length based on setting maxsize attribute for <input>.
For painters & hackers: Other applications (outside SF universe) may allow more characters and check size with JS and/or on server side but this is a completely different story.

Phone

Field Phone allows input up to 40 characters. That’s why it is enough to push 41 characters to test this requirement. Phone will accept ANY characters which looks quite strange. Even though we are surrounded by phone numbers like ‘1300 ORDER TESTER’ we did not see a case where such numbers were accepted on web forms (especially in business web apps domain).

URL

URL field may contain up to 255 characters so we can apply Text filed rules to test its size. If URL does not start with ‘http://’ or ‘https://’, or ‘ftp://’ it will be prefixed with familiar ‘http://’ when saved. The displayed URL is shown as an HTML link. Double quotes and ‘< ’, ‘>’ will not be shown. Adjoined pictures will give you some idea how we can test this element.

Read the rest of this entry »