Pattern in Test Automation

Keyword driven

Keyword driven frame work was mostly popular initially when we had no testing framework like JUnit, TestNG.

One use to write their own testing framework where he use to define ‘Keyword’ on excel/ xml and invoke a corresponding method in code using reflection.

Keyword driven + Excel

Here one can define multiple keyword and map them to a method name.

Once can write a parse to iterate through the excel and pick the keywords and invoke the corresponding method and pass the corresponding argument.

e.g. if one want to perform search he will have a keyword ‘Search’ then automation framework will invoke method ‘SearchForTerm’ using reflection.

The SearchForTerm method will do its job of locating the text box element and setting the value of test box and clicking search button.

Pros

Simple to use

Adding scenarios without touching code base. Ie. Keep reusing same keywords in excel

Hence easy for people who have never done automation

Will be good if the application is very simple and less functionality and functionality  does not change frequently.

Cons

All keywords are in same file. So all its tough to maintain file as single class file is growing.

As more and more test are added in excel the execution becomes slower

If data or signature of method changes, it time consuming to update the whole excel. And is prone to mistakes.

You cannot track the changes in SVN since excel is a binary file. Its difficult to compare the excel

If two people are working on same excel then its tough to merge changes one has to do it manually

You have to have unique keywords each time so adding unique keyword is another task. One cannot take advantage of polymorphic feature of OOP.

One will have to check in complete excel file even if single line changes

Using IDE help a lot . Intellisense feature not available in excel which is quit challenging and increases effort.

2. Keyword driven + XML

Her again using xml one can create a automation framework.

Parser can parse xml,

Fetch multiple test suite to execute. \

From each test suite pick the individual test case

Read its attribute and invoke the corresponding journey

Pros

Solves all the cons of Keyword driven [with excel]

It can support Page Object Model

One can define own customize attribute

Is flexible enough to add as many xml one needs as per need and framework can handle it.

If xml generation is pain one can create a tool to generate xml suites

Integration with CI is much feasible

Cons

If anything new is to be added, say new data source one will have to add support for that. So extensibility is a challenge

Integration with IDE is another problem. One cannot execute test from IDE instantly. One will have to depend on its command line arguments

Report displaying in CI again a extra efforts

Data driven

Data driven framework are basically one that will execute same test multiple times for different data.

They can we be used do boundary value analysis.

e.g. I want to search for different terms

e.g Pune [location], Cancer [dieses] , Amithab Bachaan [individual] and depending upon my search term automatically if system is displaying me information based on my search term. I,e

Pune [location]: it gives me map on search result page

Cancer [dieses]: Its shows near by hospitals

Amithab Bacchan[individual] : It shows me images of an individual

Then I would prefer doing data driven testing where my validation will also be corresponding details form data.

Pros

If functionality is driven depending on data and other step are same then useful

No need to define multiple time same test case just for data change {limitiation of keyword framework}

Cons

If new test case is to be added then we need to change code

Mostly Data driven test are very less

Hybrid [Data + Keyword]

Hybrid framework is combination of both Data + Keyword

Pros

All the pros of Keyword and data driven framework

Its like adding a extra bit of data source in keyword driven framework

Cons

All the cons of keyword driven framework

One cannot change to a different data source has to stick to excel

Unit Testing Libraries : JUnit/TestNG

Instead of writing our own frameworks using reflection we can use ready made framework like JUnit and TestNG

Pros

No point in reinventing the wheel again and again

Has IDE plugins. Easy to use

Don’t need to remember the attributes, one will get autosuggest

Running from IDE is easy cause of plugins

Has inbuilt reporting mechanism

One can produce report in various formats html , json , xml

Easy to integrate with build tools (maven , sbt , gradle) which helps in injecting auto dependencies [maven repository ], so updating versions , porting to other machine

Integrating with CI becomes easy cause there are direct CI plugins (jenkins), or build tool are enable us to trigger it from command prompt its easy to integrate with CI tools Jenkins , Go , ...

Will support few data sources

Can understood by everyone, since is commonly used [custom framework will need some sort of education]

Various tools are build on top of these base framework e.g. cucumber is build using junit and to support BDD

Cons

You need invest your energy to learn more [which indeed will give returns in long term]

Page Object Pattern (POM)

This pattern has evolved to solve web application automation problem. The drawback with keyword driven is one keeps adding functionality in same file and it becomes cumbersome to maintain a huge file of reusable methods.

Page object pattern solves this by creating one object per page and hand over the responsibility of all features to same file. So if a functionality / locator of any element changes. One just need to go to respective page file and update it.

It is easy to maintain POM.

Pros

This saves the maintenance cost drastically

You can define your locators in same class and this will reduce the cost of maintenance of object repository. [or one to one mapping of object repository ]

The pages are reusable by test and will avoid duplicate code

One can use exiting page object implementation e.g fluentlenium, which will further help you to just concentrate on domain, details .ie.

If new page is added you don’t disturb the existing code you just add a new page object. i.e. easy to extend , Or new control is added you just add new method to handle it

Depict your domain so easy to understand by everyone. Once everyone knows POM its easy for new team mates to understand framework.

POM + BDD Tools [One step more]

Page object pattern can be enhanced more by using BDD tools like cucumber.  Cucumber is a BDD tools where one can express the automation in domain language i.e. in Jerkin language [simple English language]

Pros

The automation coverage can be easily understood by business. Since scenarios are written in Gerkin language. Or at time BA can write the scenarios

BDD frame uses JUnit so all the feature of JUnit are inherited Reporting, logging, test attributes [before , after , before test , so on]

Has customized reports. Good integration with CT [Jenkins has Cucumber plugin]

Is updated with new feature frequently

Does support data driven and that too without compromising domain language. Will run same scenario multiple times across different data set.

Reusability of domain feature less duplication of test.

Support use of domain objects which in turns reduces maintenance of test data and reusability, customization of test data

Use of library like fluentlenium can help by wrapping most of the common method that one will have to implement on selenium webdriver.

Customized POM + BDD Tools + [Two step]

Well further optimization can be done based on the product design.

Consider a component like header, which can appear in almost all the pages.

 Then it we can create a header page object and inherit/embed it all the pages were it would appear.

Similar things can be done with components that are common across pages can be treated as separate page and can be inherited/embedded in pages where they are used

Customized POM + BDD Tools + [Single page app]

Incase of single page application we can tweak the page model patter to section model pattern.

Cons

Can bring lot of noise to single page.

In that case we can still have same mode but other way round we can logically keep each page separate and have singlePage object inherited/embedded. To keep it simple.

Like wise we can always manger page object in our framework depending upon the application type, design and its actual implementation. We can always refer to development code to see how they are managing it and can thus try same thing [very subjective] in our framework design.

References

Image is downloaded from http://www.mosaicinc.com/mosaicinc/html/test_automation.html . I dont own the copyright.