Automated Testing for the Web

As the web grows more complex and dynamic, the need for testing grows as well. However, testing often suffers from a lack of prioritization, accountability, and enforcement – frequently thrown by the wayside due to factors such as timeline and budgetary constraints.

Testing is a crucial part of any web development and maintenance workflow and is a broad field ranging from manual to automated. While manual testing is still important, automated testing tools are becoming ever more powerful and ubiquitous.

What is automated testing?

Automated testing differs from manual testing, being a series of checks that are run programmatically. Usually it works by running scripts that check your site for certain visual and/or behavioral outputs and compares it against an expected result. Automated testing can replicate human interaction with a site across numerous browsers and devices. How these tests are implemented, the level of technical knowledge needed, and the tools used can vary greatly.

Why is automated testing important?

Websites can be complex, leaving plenty of room where bugs can hide. While humans are much better than computers at detecting contextual errors, humans struggle with highly repetitive tasks. Luckily, computers are well suited to this type of work.

Automated testing allows us to decrease the cognitive load of tasks at which humans are innately less skilled, freeing up mental capacity to find other, more abstract errors.

One of the primary examples of this is cross browser testing. It takes a lot of time and effort for a person to look at the same thing across several browsers and devices. On the other hand, computers can quickly run through numerous tests across many browsers and viewports to ensure that your site looks and behaves as expected.

Automated testing isn’t necessarily meant to save time; it’s meant to improve the quality of your product.

It can be a laborious process to plan and create all the tests necessary to get enough test coverage of your site, especially with the additional time needed to continually maintain them. However, the quality of your code will improve as these tests reduce the risk of breaking changes when adding new features or content to your codebase.

Types of automated testing methods

The size and scope of your tests, automated or not, can be categorized into several levels, most commonly including unit, component, and integration testing. These levels of tests are designed to target code sets of different sizes, from small to overarching pieces of code. To learn more about these types of tests, please refer to our Overview of Web Testing article.  

There are several types of automated testing methods that can be used to carry out your unit, component, and integration testing. The most common are: visual regression testing, record and playback testing, and scripting testing.

Visual Regression Testing

Visual regression testing, also called screenshot testing, may sound complicated, but it is simply a way to visually test your site by comparing before and after screenshots. Usually a test will run every time you make changes to your site. A screenshot of the page with the new changes is compared against a baseline image for you to review. This method allows you to test your UI by verifying that only the desired changes have been made.

For example, say you want to change the color of the buttons across your site. Although it seems like a simple enough change, perhaps due to shared CSS styles, some header text has also changed color. This type of unintended change can be easily missed by a manual tester, but will be highlighted with visual regression testing.

Especially with more complex sites, code changes can sometimes affect unintended elements. This is why the ZingSoft team here at PINT uses visual regression for their integration tests as one way to ensure the continual quality of their site and product.

The setup of this type of tool can require more technical expertise, however, as it is often integrated into your deployment pipeline. This means that your site needs to have some type of CI/CD pipeline already set up. Once set up though, it can be a powerful tool for regression testing by making sure that any new changes haven’t affected other parts of your site.

As the name suggests, this method is only effective for checking visual changes and bugs and is not designed to test site functionality or backend implementation.

Record and Playback Testing

Record and playback testing allows you to record your actions and turn them into a test. This method requires only basic technical knowledge and is therefore a good option for those who are not familiar with programming.

A common example is the testing of a contact form. First, you would open your record and playback tool to record yourself filling out the form. The recording tool then converts your actions into a series of scripts that can then be repeated with the click of a button. Once ready to run the test, it will replicate your actions, including opening a browser, clicking on links, and typing in input fields.

The tests themselves are usually depicted as a list of block-like elements, each one containing a command, a target element on the webpage, and the expected value of that element. When you run the test, it will go through all the commands, compare the expected and actual results, and then provide you with a pass or fail response.

Some additional editing of the scripts is usually needed. After you finish recording your actions, you will need to include assertions (the expected values) against which the script will compare. A basic understanding of CSS classes and IDs is necessary to create these assertions, as that is how most record and playback testing tools target specific elements.  

Since record and playback testing is action-based, it is ideal for testing user flows and functionality, and can be used to test visual elements as well.

Scripting Testing

The most advanced automation method is to actually write your own tests. This covers many types of tests, including unit, integration, and component tests. These can be written in a variety of languages, using a multitude of libraries and frameworks. More so than any other, this method of creating tests requires a high level of programming skill and is labor intensive. Despite the difficulties, it is the most powerful and specialized tool you can have at your disposal.

When researching automated testing, you’ll come across an endless list of different products, like Mocha, Chai, Cucumber, Cypress, TestNG, Karma, and many, many more. The most common tool for testing web sites and applications from the user’s point of view is Selenium, which is a suite of tools that drive browser automation. Due to the wide variety of options, it is essential to identify your team’s technical strengths and needs to help drive your decision as to which set of tools will work best for you.

Before even writing your tests though, it is important to outline what specifically needs to be tested and formalize a series of test cases. A spreadsheet is often used to document these cases, listing out items such as steps to reproduce, expected result, and actual results. This list can then be used to help drive the creation of your tests.

Especially for groups that work on a singular product, it is common to have a team of QA engineers creating and maintaining libraries of automated tests. For example, the ZingSoft team has written about 3,000 unit tests in Mocha and Chai (a JavaScript testing framework and assertion library respectively), using Karma (a tool similar to Selenium) to run the tests.

Although the most difficult and time-consuming, scripting testing is the best option for serious, robust automated testing.

Conclusion

Automated testing is a large field and is rapidly becoming more relied upon. There are many forms of automated testing from which to choose including visual regression testing, record and playback testing, and scripting testing. These are all incredibly useful tools to help improve the quality of your site or product.

As automated testing can be quite labor intensive, it is essential to assess your team’s technical knowledge, testing needs of your product, and any budgetary constraints. Deciding what method of automated test creation is best for your team is highly dependent on the types of project(s) you work on and the technical skill level of those on your team. Automated testing can be viewed as a long-term investment; and although there is a high start-up cost, it can help improve the consistency and quality of your site in the long run.

Chat with PINT experts about your next project.