Coverage matters by Igor

Coverage matters

The sky will collapse on the ground,
The grass will cease to grow —
He will come and silently fix everything,
The man from Kemerovo.

Boris Grebenshchikov,
“Man from Kemerovo”
(translated by Andrei Soroker)

Let me remind you that we are testing only one object of Recruiting application – Employment Websites. For clarity and convenience, I created a chart as an image map, and linked it with screenshots so that you do not need to go anywhere to understand what we are talking about. Now testing is executed in a sequence (omitting login and logout):

Click tab 'Employment Websites'
click 'Save & New'
'Employment Websites' search results
Add 'Employment Website' page
click 'New' button
click a record
click 'Save' click 'Delete'
Show 'Employment Website'
click 'Save & New'
click 'Save' click 'Save'
click 'Edit' click 'Clone'
Clone 'Employment Website'
Edit 'Employment Website'
click 'Save & New'

Figure 1 – Employment Websites object (flowchart elements are clickable)

1. checkIsRecordSavable() – picture
2. checkSequence() – picture
3. checkAllElements() – picture

In an academic world such an article would probably have called ‘Some aspects of the sequence of software testing’, well, we just ask – Do you see something illogical or unusual in such an approach? Maybe something is missing or something is excessive? Can’t you see? Or do not want to think? Then click on the link below and read.

If we look closely at the sequence of testing, we can see that in the first phase we check whether we can add or delete an object (in this case ‘Employmemnt Website’) with simple and correct values, which we call KGV (Known Good Values). It seems that so far everything is logical. Go ahead. In the second step, checkSequence() is checking ‘Edit’ and ‘Clone’ pages. Our attention, at the time of writing the code, has been focused on creation of actual tests and this is why ‘Edit’ and ‘Clone’ are checked this way:

Figure 2 checkSequence()

- create new record (1-2-3)
- on page ‘Show’, where we end up, we push the button ‘Edit’ (4) that should be there
- without changing anything on ‘Edit’ page, click ‘Save & New’ (5). Sort of checking if ‘Save & New’ works
- after landing on the ‘Add’ page, press the button ‘Cancel’ (6)
- now we click on the button ‘Clone’ (7)
- ending up on the page ‘Clone’, we sort of checked if we can reach this page. Click ‘Cancel’ (8)
- finally, we are finding ourselves on the page ‘Show’ and click on the button ‘Delete’ (9)

Not everything is logical here, is it? If we save a record on ‘Edit’ page, then why not do the same on ‘Clone’ page? We have an explanation for this – after cloning, we will have two records, which in the end of checkSequence(), must be removed. When writing this code, it seemed superfluous. Now it’s time to do it all correctly. Firstly, I want to immediately remove from consideration functionality ‘Save & New’. Why? Read in the box below (point 2). If your application has such functionality, then, I think, it won’t be hard to add special tests for it. Secondly, I propose to take and consolidate what checkIsRecordSavable() and checkSequence() do, making it more logical and better.

1. Add a record and delete it.
2. Add a record, go to the edit page, save it, delete record.
3. Add a record, go to cloning page, save, delete both records.

How to name these new methods?
checkIsRecordAddable() – picture
checkIsRecordEditable() – picture
checkIsRecordCloneable() – picture

We do not need checkIsRecordDeleteable () – if it does not work, we will immediately find out.

These tests themselves can act as quick tests (smoke). All we have done – made sure that the whole application works. If you wish, after this step you can begin to torment the application with detailed tests.


I would like to say a few words about our guinea pig – ‘Employment Websites’ object.

1. In essence, we are testing the interface in the typical list UI. Any business application can have hundreds of them. If they are all stored in the same or similar tables and their interface is created by the same script, then, perhaps, we will not test all of them, and test only one. If application contains several types of lists (for more details about what we call ‘list types’ here), we will test thoroughly each type.

2. One of SalesForce features, not so common, is a button ‘Save & New’. But paradox that the values are added in the lists in batches very rarely. This will be true not only for recruiting sites list but also for position list and candidates list which are also part of Recruiting application we test. Maybe just before introducing the application to the business payroll staff will enter positions and human resources manager – recruiting sites your company uses. Only for them, we save two clicks by creating functionality behind the button ‘Save & New’. Without it the user, saving a new record, will end up on a page showing just saved record and will press tab ‘Employment Websites’ (first transition). After landing on search results (where all entries are shown line by line) he would have pressed the button ‘New’ (second transition) to add one more record. Therefore we force most of the users to make a choice which button to press. Such user will most likely have such self talk: “So. Do I still have recruiting sites to add?” – “There seems to be none” -  “Then I need to press … the button ‘Save’”. The user who was given only one button ‘Save’ will add two records at the same time, since he has fewer choices leading to faster decision making. That is our opinion – if you can handle task 90% of time with this – it will work better and faster than this.

3. Not all applications allow to save a record after editing if you have not changed anything. This is verified by JavaScript and/or on the server side and is done this way because such an operation often updates the last modification date and user ID. Why would we save if there were no changes? If your application follows this rule – then you will need a) take it into account b) test it. Similarly, cloning functionality – the list must have a key field (such as Web Site Name and Web Address for our object) and the clone must not be allowed to be saved until combination of these fields have not been changed to unique. Our object allows saving unchanged records on edit and 100% matching clones, but SalesForce platform allows to create such validation rules and therefore we have no complains about it. The important thing is such validation deviations will impact test logic.

Now let’s look at checkAllElements(), where we test thoroughly each of the four elements of our object.

Figure 3 checkAllElements()

Here we find that all elements are tested only via ‘Add’ page. As developers, we know that such testing does not guarantee that all will work on the ‘Edit’ page and the ‘Clone’ page. That is, we have missing tests for a third of the functionality and need to repeat all tests for the ‘Edit’ and ‘Clone’ pages. As mentioned above (see the comments section – paragraph 1), we do not need to test same list types and can test just one, but we can not afford to test the page ‘New’, skipping tests for ‘Edit’ and ‘Clone’ .

As a result, instead of checkAllElements() we will use:

checkAllElementsOnAdd() – picture
checkAllElementsOnEdit() – picture
checkAllElementsOnClone() – picture

‘Kemerovo man’ is truly happy now.

Print this post | Home

Comments are closed.