Analysis by Alena & Dima

Life of web-tester is uneasy. We direct plays, write in a blog and also participate in the test creation process which is both unstoppable and exciting for us. Sooner or later we started on Stage 3 (with a realistic and conscious attitude). We continue to concentrate on it, and consider what to look at next. What can we see? We can see our application – Recruiting, assembled by manual. It is time now to X-ray it and proceed to Step 1 which was written once on a dark cold windy night on white board surface by a simple black marker.
Let’s begin.

Text

SalesForce allows specifying Text field size when it is created. It can be any size as long as it is within 1..255 characters :) It is easy to check – enter as many characters as specified for a particular Text field plus one more and test if it can not fit. We know that SalesForce (SF) validation for the length based on setting maxsize attribute for <input>.
For painters & hackers: Other applications (outside SF universe) may allow more characters and check size with JS and/or on server side but this is a completely different story.

Phone

Field Phone allows input up to 40 characters. That’s why it is enough to push 41 characters to test this requirement. Phone will accept ANY characters which looks quite strange. Even though we are surrounded by phone numbers like ‘1300 ORDER TESTER’ we did not see a case where such numbers were accepted on web forms (especially in business web apps domain).

URL

URL field may contain up to 255 characters so we can apply Text filed rules to test its size. If URL does not start with ‘http://’ or ‘https://’, or ‘ftp://’ it will be prefixed with familiar ‘http://’ when saved. The displayed URL is shown as an HTML link. Double quotes and ‘<’, ‘>’ will not be shown. Adjoined pictures will give you some idea how we can test this element.



Text Area (Long)

The Text Area Long (TAL) element will consume 32,000 characters. It has 3 visible lines. When it remains less than 50 characters to 32K limit you will see a message ‘XX remaining’. When the limit is exceeded, this message changes to ‘XX over limit’. If we will try to save a form containing ‘over limit’ entry we will get a message ‘Error: Value can not exceed 32,000 characters’. How do we test this? We are sure you also have some ideas on it.
For painters & hackers: It is quite strange to believe that 32,000 characters will be actually typed in a web form without even an auto-save feature. Most likely it will be pasted from text editor. From our point of view counting (32000-50) character to give a warning is quite unnecessary. It will be just enough to check length of TAL on submission and again on a server (for those who may try to POST programmatically trying to put your app down).

Number

The total length (‘Length’ on SF platform) and fractional part length (‘Decimal Place’s on SF platform) are specified at a time of field creation. If size of integral part will exceed total length you will get a message ‘Error: Number is too large’.

If fractional part length will exceed a specified one for a field – you will see a magic of rounding when saving it. Spaces and leading zeroes will be ignored. Comma (,) is ignored too but only if it is not a last character. Dot (.) is used as a separator between the integral and fractional part of decimal numeral. If integral part size is greater than three it will be grouped by three separated by comma. Now we enter the value and validate it, to see that we haven’t missed anything.

All input characters, except numbers, dot and comma will lead to a message ‘Error: Invalid number’. Same message was observed when number contained two or more dots.

If the integral part exceeds 19 characters, you will have an opportunity to observe a message ‘#Too Big!’ which will be displayed (wonder where?) Yep, we were also surprised. All elements display messages underneath but not this input type. This one modifies the input field, halting progress for the form.

For this element testing we enter restricted characters, numbers exceeding Length and/or Decimal Points. If all messages come up – element Number stamped as valid by us.

Currency

Field Currency – having a greater success in life twin brother of field ‘Number’. The only difference is … right! – the dollar character ($) showing in front when displayed and not restricted as first character when entered. We are going to test it based on relationship (as Number) and than based on income status ($ handling) :)

Date

Format for Date field – month/day/year
Month – one or two digits (1-12)
Day – one or two digits (1-31)
Year – 4 digits

This field ignores leading and trailing spaces as well as leading zeroes for each part of date. If you have a habit to use dots or dashes as separators – please use it! As soon as you will finish typing and switch to next field application will silently replace your separators with strict slashes. Also you will need to create a cheat sheet to remember that you can not use dot instead of first slash and dash instead of second slash as this will cause an error message. But you can use dash instead of first slash and dot instead of second slash – no worries! Some ideas for tests are shown near by.

Also we found that SF programmers regularly try to “simplify work” for those lazy folks who like to truncate and abbreviate many of the date type input.
Digits 1-9 after second slash will be converted to 200X after field focus will be lost.
10-59 —> 20xx
60-99 —> 19xx
100-899 —> 2xxx
900-999 —> (xxx + 100).
Occasionally such transformations do look logical but sometimes they do not, for example, an ability to enter any characters after year bit. This filled up the cap and after some consideration we decided that our tests will disagree with this and fail.

Apart of classified operations with date we found some more we can not describe using any alphabet we know. Such are:
3/222/009 —> 10/8/2009;
4/102/008 —> 7/11/2008.
We believe our analysis was very deep and all other ways will trigger message ‘Error: Invalid Date’.

Email

Email field must have following format – ‘user_name@server_name.top_level_domain’. This leads to a format requirement of having character ‘@’ and at least one dot following it.

The user_name part can not contain: ‘:, ;, ”, <, >, ,, \, •’, server_name top_level_domain can not contain underscore (_) character. Leading and trailing spaces ignored. Email field is displayed as a mailto HTML tag. Usage of restricted characters or incorrect format produces – ‘Error: Invalid Email Address’.

Picklist

Some common observations while creating this element: the Platform allows specifying sorting order as well as making first value a default value (‘Use first value as default value’). Otherwise default value will be ‘-None-’ which most likely translates to NULL on database level when saved. It is impossible to create a Picklist as a required element. However, by editing a layout you can make it required. In this case value ‘-None-‘ will be shown until you have related records with null values. If you will try to save such value after changing attribute to ‘required‘, you will get a message ‘Error: You must enter a value’.

Checkbox

It is Dima’s favourite creature. It can not be mandatory. Hard to disagree.

Lookup

If you enter a string into this field and press the ‘Save’ button – the application will search for matching values. If nothing is found, the user will see the message ‘Error: No matches found’. We wonder why we are seeing this as an error?

If more than one match is found, Lookup will evolve into Picklist with ‘-–None–-’ value. The message will be displayed below the element as shown. Again is it an error?

If a single match (perfect) found record will save with a blink of an eye.

Big job and enough silver grenades for our Selenium bazooka!

Some people may think it is good enough to relax for a couple of days but guys and girls from Deep Shift Labs are made from the Right Stuff. We can not leave this alone without more attention to some attribute and behaviour similarities and we immediately start planting not a tree, but a ‘SalesForce Element-ary Tree’ on the fertile ground of our Analysis.

DSL expresses gratitude to a Heaven Chancellery for a kindly given sapling from a “Tree of Knowledge”.

Please start crossing your fingers now!

Print this post | Home

2 comments

  1. Keith says:

    Hi,

    I’m starting to look into Selenium testing of a Salesforce app and wonder what pragamatic approach you guys used to deal with this problem:

    http://boards.developerforce.com/t5/Best-Practices-Discussion/Selenium-Testing-Best-Practices/m-p/144335

    Thanks,
    Keith

  2. bear says:

    Hi Keith,

    I should say that we do not create apps on SF platform and do not test them. When we started learning Selenium in 2008 we picked up SF Handbook which described how to create Recruiting application in SF and decided to create it and write tests for it.

    We did not move this project for one or two years now, so I don’t know what has changed in SF over the years, but back then we used following approach:

    we used xpath locators based on elements’ titles, like
    //label[text()='Employment Website Name']/following::input

    and corresponding locators for view mode (in this mode we were checking if text we entered is present ).

    Of course, we had a very simple SF application, so this approach possibly can not satisfy all of your needs.

    We have done a kind of framework for working with Salesforce elements and pages, see this post and others (please, remember that it was done only for training and was not updated several years).

    Also you may be interested in reports our framework generates (example) and our cloud – Nerrvana which automaticaly runs tests.