Why XML DSL required?

The rationale for the given platform

The automation of the product testing is a high priority issue, which brings certain challenges:

  • When should the testing be applied?
  • How to save time on testing?
  • Which automation tools to use?
  • What approach is better for test generation?

As regards the testing time, it often depends on the scale and the complexity of a project. Ideal is to apply testing at all the development stages. Keeping this in mind, one should already start writing tests when working on the initial version of the product. The tests must take into account the specifics of the system, and, if necessary, be able applicable for lots of different instances of the system. For example, for testing of all simultaneously developed versions of the project. Test input parameters should be treated with accuracy in order one be able to run them on servers with various configurations and do not change the code. It is preferable to register the largest possible quantity of input parameters in separate files.

The experience of time saving shows that the quality of the product directly depends on the investments in testing. It is necessary to properly maintain the documentation and mark the link with a corresponding business case for each test. This will save a lot of time on testing support in the future.

There exist a wide range of automation specialized programs to choose from. For web applications, one can take advantage of Selenium. It includes a variety of products for different testing methods, including Selenium RC Server, Selenium IDE, etc.

Selecting an automation approach, one should take into account all the diversity of methods: special programs, which generate tests for common manipulations with browser; writing tests using procedural and objective-oriented programming; or creating them with the help of DSL (Domain Specific Language). Let's have a closer look at the variety of approaches and the fundamental differences between them.

Creating tests utilizing the programs, which generate tests for browser manipulations.

The plugin FireFox Selenium IDE is an example of such a program. To create a test, you should enable test logging in the plug-in, reproduce a case in the browser and your test is ready! Such work can be entrusted even to a QA engineer. They will be able to create many tests in a couple of days, and check a large part of the application. Moreover, tests can be delivered as a code in almost any programming language. Such tests can be launched in both the plug-in and other Selenium products. Note that the generated tests will be performed in a copy-paste style, unless some additional checks are inserted manually in the code. If it is necessary to change several steps in all the tests, you will have to edit all the files, maybe altering them completely. Having to exercise a large number of tests, it may seem very time-consuming. Large-scale projects may demand several hundreds of tests. In this case test maintenance becomes complicated, and the tests themselves lack flexibility. As a result, this approach is effective for fast test generation, but at the same time is void of other possibilities.

This method can be applied for small-scale projects where test generation speed is a crucial parameter.

The procedural style of test coding

To obtain the desired flexibility, tests should be coded in a programming language, rather than generated. To perform the tests one can use the following Selenium libraries: Selenium RC or WebDriver. They allow you to access a browser directly from the code. This task may require specific programming expertise. It shouldn't be delegated to inexperienced developers, as they risk to make common mistakes, such as copy-paste, hard-code and others. The doubtful quality of tests will make them lack all the specific properties for which the decision to write them manually was taken. It is advisable to adhere to the following rules when creating a test:

  • bring out input data from the code, put them in files, or pass as parameters
  • introduce logging in the tests
  • employ a comprehensive way of error handling, so that it would concisely indicate what kind of error occurred
  • apply the parameterization to reduce copy-paste and other ineffective approaches

Due to the manual coding of all the test stages, it will be more time-consuming to apply them.

Another disadvantage of this approach is that if tests are written as a sequence of actions, they lose their desired software capacity. Such code is difficult to modify since the "net" of commands may intertwine, and certain situations may even require copy-pasting. It is hard to replace or modify the steps without affecting other scripts, when dealing with a large number of tests. All this impedes creating stable tests that one could easily scale and adjust. More time will be required for each new change.

This method is applicable for large projects, but if testing is necessary for over one year, the tests' maintenance may cause significant problems.

Using the OOP model (PageObject Pattern)

The PageObject template is a perfect solution for attaining a complete flexibility of tests. The essential difference from the previous approach lies in the emphasis on architecture. This is a challenge for an experienced developer, who should be able to define and separate test logic the actions that could be applied on a web-page. Such approach assumes the description of tests in the POP style only. At first the POP model of the whole tested system is created; it describes each element (buttons, input fields, etc.) and presents all possible actions (keystrokes, text input, etc.). This model alone is not a test yet, it is more of a prototype of the system under test and an interface for future tests, which will allow to access the system. For web tests, the prototype is a view of all pages, the cookie cutters from page, web elements, etc. In this prototype, you can get any of the objects and run all the types of actions it can actually perform. For example, for a log-in page the actions are:

  • enter data in the field "log in"
  • enter data in the field “password”
  • press the "enter" button

The last action, in addition to the designated operation (pressing the button), would return the next page object. And for that object there will be available other operations, that are the only relevant to it.

The steps of business cases are not described in the POP model. The tests are already created in a side class structure, where the manipulation with the model occurs. The structure has a description of the test cases that should be automated. High test flexibility is the principal advantage of this approach. For example, adding a new field for data entry, it is necessary to change only one (!) class in the model, and not the classes in all the tests, where a new field appears. At first glance, this approach may look like a winning solution, but it has its own disadvantages:

  • a high expertise standard of a QA engineer
  • a slow speed of test generation
  • the complexity of tests (sometimes it is comparable with the architecture of the application under test)

This method is usually applied to large and durable projects where the speed of test creation is of minor importance, but there is a need for continuous testing of a constantly changing project.

Creation and use of DSL (domain specific language) XML-based language

The prime demand for testing is a maximum flexibility in a minimum time. Auto-tests are often performed by testers rather than developers. Generally they lack programming skills to create an effective POP testing model. And there is simply no time for their education. In this case, persons with no programming background can use a tool which allows to create tests in a comprehensible way. This tool is the XML2Selenium framework. It is based on WebDriver. There is no need to write a complex code, it is enough to submit a test in a DSL language based on XML. Simply create an XML file and describe the sequence of tags. Each tag stands for an action in the test. It allows to generate illustrative and easily adjustable tests in short time. To run a test, you need only the framework, a browser, and the test described in an XML format. To maintain test flexibility, it is possible to implement inheritance for the tags, which will minimize the repetition of identical steps in different tests.

Platform distinctive features:

  • possibility to easily add a new plug-in for the realization of necessary actions
  • simple API for plug-in creation
  • integration with Junit and Jenkins (but full independence from them and also from Maven, with the ability to create proper runners)
  • remote debugging on a server
  • absence of dependence from selenium, possibility to use another tool
  • data driven support (ability to describe test results in a comprehensible manner, understandable even to someone who didn't create the test)
  • the possibility to self-test the behavior, i.e. to write tests for the framework in the same language
  • smart logging, the ability to create snapshots and screenshots
  • the minimum of programming - jaxb
  • eclipse plug-in - a simple editor for creating new tests, doesn’t requires XML skills
  • test cases validation
  • data randomization
  • thread saved, the ability to run any number of the core versions, data record to different directories (load testing)
  • user friendly exception processing
  • reports for a business-user in a bdd style of any format
  • if/for tags support

Using the platform, you can rapidly create quality tests, and get plenty of opportunities for their convenient application. If necessary, create your own application-specific framework extensions.