Introduction to Automated Testing

What is an automated test?

An automated test is a program that automates the task of testing an app. It interacts with the app, performs actions, and compares the actual result with the expected result we have previously defined.

Like manual tests, automated tests don’t guarantee whether an app works. Rather, they give us a level of confidence that it should. The more functionality that the tests cover, the more confidence we have.

Why write automated tests?

If an app doesn’t work, users will be frustrated. It doesn’t matter how great the UI is or how fast the app performs. Users will lose confidence in the app, and that confidence is hard to restore.

We can manually test the app with manual test scripts, but people are prone to error and may misinterpret test scripts or overlook steps. The more steps in a manual test, the more likely we are to make an error. Automating test scripts removes the risk of them being incorrectly executed.

Testing the app manually also takes time and slows the development process down. Running tests automatically when code is pushed to a code repository gives developers valuable feedback about whether the changes have broken something. The shorter the time between writing code and receiving feedback, the more efficient taking that action is. If we receive a bug long after we’ve finished writing the code, we are less likely to remember the intricacies of that code. Not to mention, the developer may no longer be working on the project. By automating tests, we don’t have to rely on specific developers or human memory.

It is common for multiple developers to work on an app working on different branches. When those branches are merged into the main branch, manually testing for breakages becomes very expensive and grinds development to a halt. Automated tests can be added to a build pipeline to give the team confidence that the app isn’t broken.

Automated tests are a fundamental part of continuous integration and continuous delivery systems, which help us quickly get valuable features to end users. They are an investment and more expensive to set up than manual tests, but they eventually improve the software development process’s speed and quality.

Note: It is important to note that automated tests don’t guarantee that an app isn’t broken. They reduce the risk of it being broken and increase our confidence in releasing versions of the app.

It is also important to note that automated tests aren’t a replacement for all manual testing. There is no replacement for a person’s first impression of the user experience of a feature. Humans can understand how the app fits into the end user’s activities and feedback on how well a feature fits into those activities. While people should always perform exploratory testing, any test that needs to be repeated over and over again is an ideal automated test.

Different types of automated tests

Automated testing is challenging. A test relies on the thing being tested to be in a consistent state at the start of the test. If a feature we are testing depends on something in a database, that test relies on some of the data being in a particular state. Other tests will depend on the data being in a slightly different state. This is why test automation is particularly challenging with databases.

Often, our app may rely on a third party service we don’t control, like an email service. How can we ensure a service we don’t have control of is in a consistent state? We may also get billed by the service based on the amount of usage. Do we really want our automated tests to increase the size of that bill?

Automated tests that exercise things like a database or a third-party service are generally much slower than exercising a unit of pure logic within our code. Executing tens of thousands of automated tests can take a long time to run, but we want to give developers quick feedback on whether something is broken.

Unfortunately, there isn’t one perfect automated test. Over the years, different kinds of tests have emerged due to the challenges described above. Different types of tests serve different purposes.

End-to-end tests interact with a fully running app as an end user would. They are the slowest type of tests to execute, but they give us the most confidence that our app is functioning properly. These are the most expensive tests to write because all the dependencies will be relevant for a feature we test, and we need to make sure these are in a consistent state.

Unit tests are the opposite of end-to-end tests. They interact with a particular aspect of an app. They are fast to execute but give us the least confidence that the app isn’t broken. Generally, these are the cheapest tests to write. The exception is when there are dependencies to deal with that need to be replaced with something that gives us consistent results.

Integration tests are the middle ground between unit and end-to-end tests. They don’t exercise a fully running app but do exercise multiple parts and layers of it. In recent times, these tests have become more popular because they execute fairly fast, give us a reasonable degree of confidence, and are pretty cheap to write.

A comprehensive automated test suite will include all these different types of tests. However, it is wasted effort and wasted execution time to cover the same functionality in different types of tests.

In this course, we’ll focus on front-end unit integration tests in a React app.