Automation Testing Introduction

What is Automation Testing?

Automation Testing is the use of software tools and scripts to execute test cases automatically, compare actual outcomes with expected results, and report findings — without human intervention at execution time.

Rather than a tester manually clicking through an application, an automation script performs the same steps programmatically, often in seconds. This makes it possible to run thousands of tests in the time it would take a human to run dozens.

When to Automate

Not every test is a good candidate for automation. Here is a practical guide:

Popular Automation Tools

The right tool depends on what you are testing — web, mobile, API, or desktop. Here are the three tools I have used most extensively in my career.

Selenium

Selenium is the industry standard for web browser automation. It supports multiple programming languages (Python, Java, JavaScript, C#) and all major browsers (Chrome, Firefox, Safari, Edge).

Best for

• Web application functional testing
• Cross-browser compatibility testing
• Integration with CI/CD pipelines (Jenkins, GitHub Actions)

Key components

• Selenium WebDriver — Drives the browser directly via browser-specific drivers
• Selenium Grid — Runs tests in parallel across multiple machines/browsers
• Selenium IDE — Record-and-playback tool for quick test creation

Appium

Appium is an open-source tool for automating native, hybrid, and mobile web apps on Android, iOS, and Fire OS. It uses the WebDriver protocol, so the API feels familiar if you already know Selenium.

Why Appium stands out

• Single framework for both Android and iOS
• Tests real devices and emulators/simulators
• No app modification required — tests the actual app package
• Supports Python, Java, JavaScript, and more

Best for

• Mobile app regression testing
• Cross-platform mobile test coverage
• IoT device app testing (Fire TV, Echo, Alexa apps)

Robot Framework

Robot Framework is a keyword-driven test automation framework that uses plain English syntax. It is particularly popular in teams where both technical and non-technical stakeholders contribute to test design.

Best for

• Acceptance Testing / BDD-style testing
• Teams using a mix of technical and non-technical testers
• API testing (via RequestsLibrary)
• Web testing (via SeleniumLibrary)

During my MSc in DevOps at ATU Letterkenny, Robot Framework was part of our CI/CD pipeline module — integrated with Jenkins for automated test reporting on every build.

Sample Appium Script (Python)

Below is a basic Appium test for an Android app — the kind of script I wrote regularly at Virtusa. It launches the app, finds an element, and verifies text.

from appium import webdriver
from appium.webdriver.common.appiumby import AppiumBy
import unittest

class AndroidLoginTest(unittest.TestCase):

    def setUp(self):
        desired_caps = {
            "platformName": "Android",
            "deviceName": "emulator-5554",
            "appPackage": "com.example.myapp",
            "appActivity": ".MainActivity",
            "automationName": "UIAutomator2",
            "noReset": True
        }
        self.driver = webdriver.Remote(
            "http://localhost:4723/wd/hub",
            desired_caps
        )
        self.driver.implicitly_wait(10)

    def test_login_with_valid_credentials(self):
        # Enter username
        username_field = self.driver.find_element(
            AppiumBy.ID, "com.example.myapp:id/username"
        )
        username_field.clear()
        username_field.send_keys("testuser@example.com")

        # Enter password
        password_field = self.driver.find_element(
            AppiumBy.ID, "com.example.myapp:id/password"
        )
        password_field.clear()
        password_field.send_keys("SecurePass123")

        # Tap login button
        login_button = self.driver.find_element(
            AppiumBy.ID, "com.example.myapp:id/loginBtn"
        )
        login_button.click()

        # Verify dashboard is displayed
        dashboard_title = self.driver.find_element(
            AppiumBy.ID, "com.example.myapp:id/dashboardTitle"
        )
        self.assertEqual(dashboard_title.text, "Welcome, testuser!")

    def tearDown(self):
        self.driver.quit()

if __name__ == "__main__":
    unittest.main()

Key points from this script

• desired_caps — Tells Appium which device and app to launch
• AppiumBy.ID — Most reliable locator strategy for Android resource IDs
• implicitly_wait(10) — Gives elements up to 10 seconds to appear before failing
• tearDown — Always closes the driver session, even if the test fails

Sample Selenium Script (Python)

A simple Selenium test for web login using Chrome:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import unittest

class WebLoginTest(unittest.TestCase):

    def setUp(self):
        self.driver = webdriver.Chrome()
        self.driver.maximize_window()
        self.wait = WebDriverWait(self.driver, 10)

    def test_login_success(self):
        self.driver.get("https://example.com/login")

        # Enter credentials
        self.driver.find_element(By.ID, "email").send_keys("user@example.com")
        self.driver.find_element(By.ID, "password").send_keys("Password123")
        self.driver.find_element(By.ID, "loginBtn").click()

        # Wait for dashboard to load and verify
        dashboard = self.wait.until(
            EC.presence_of_element_located((By.ID, "dashboard-header"))
        )
        self.assertIn("Dashboard", dashboard.text)

    def test_login_invalid_password(self):
        self.driver.get("https://example.com/login")

        self.driver.find_element(By.ID, "email").send_keys("user@example.com")
        self.driver.find_element(By.ID, "password").send_keys("WrongPassword")
        self.driver.find_element(By.ID, "loginBtn").click()

        # Verify error message appears
        error = self.wait.until(
            EC.presence_of_element_located((By.CLASS_NAME, "error-message"))
        )
        self.assertEqual(error.text, "Invalid email or password.")

    def tearDown(self):
        self.driver.quit()

if __name__ == "__main__":
    unittest.main()

Pros & Cons of Automation Testing

Setting Up a Basic Automation Framework

A bare Selenium or Appium script is not a framework — it is a script. A framework adds structure, reusability, and maintainability. Here is the minimal structure I use when starting a new automation project:

project/
├── tests/
│   ├── test_login.py
│   ├── test_checkout.py
│   └── test_search.py
├── pages/                  # Page Object Model classes
│   ├── login_page.py
│   ├── home_page.py
│   └── checkout_page.py
├── utils/
│   ├── driver_factory.py   # WebDriver setup/teardown
│   └── config.py           # URLs, credentials, timeouts
├── test_data/
│   └── users.json          # External test data
├── reports/                # HTML test reports (auto-generated)
├── requirements.txt
└── pytest.ini

Page Object Model (POM)

The Page Object Model is the single most important pattern in test automation. Each page of your application is represented as a Python class. Locators and actions for that page live inside the class — not scattered across test files.

# pages/login_page.py
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

class LoginPage:
    URL = "https://example.com/login"

    EMAIL_INPUT    = (By.ID, "email")
    PASSWORD_INPUT = (By.ID, "password")
    LOGIN_BUTTON   = (By.ID, "loginBtn")
    ERROR_MESSAGE  = (By.CLASS_NAME, "error-message")

    def __init__(self, driver):
        self.driver = driver
        self.wait = WebDriverWait(driver, 10)

    def open(self):
        self.driver.get(self.URL)

    def login(self, email, password):
        self.wait.until(EC.presence_of_element_located(self.EMAIL_INPUT))
        self.driver.find_element(*self.EMAIL_INPUT).send_keys(email)
        self.driver.find_element(*self.PASSWORD_INPUT).send_keys(password)
        self.driver.find_element(*self.LOGIN_BUTTON).click()

    def get_error_text(self):
        return self.wait.until(
            EC.presence_of_element_located(self.ERROR_MESSAGE)
        ).text

With POM, when the login button's ID changes, you update it in one place — login_page.py — instead of hunting through dozens of test files. This is the difference between a maintainable suite and a fragile one.

CI/CD Integration

Automation tests only deliver full value when they run automatically on every code change. Here is a minimal GitHub Actions workflow that runs your Selenium suite on every push:

# .github/workflows/test.yml
name: Run Selenium Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Install Chrome
        uses: browser-actions/setup-chrome@latest

      - name: Run tests
        run: pytest tests/ --html=reports/report.html --self-contained-html

      - name: Upload test report
        uses: actions/upload-artifact@v3
        if: always()
        with:
          name: test-report
          path: reports/report.html

This runs headlessly on Ubuntu with Chrome, generates an HTML report, and uploads it as an artifact — even if tests fail. The if: always() ensures you always get the report regardless of pass/fail.

Managing Test Flakiness

Flaky tests — tests that pass sometimes and fail other times without code changes — are the biggest threat to automation suite credibility. Once the team starts ignoring red builds because "it's probably just flaky," the entire CI safety net collapses.

Root causes of flaky tests

• Timing issues — Using time.sleep() instead of explicit waits. Elements appear at different speeds depending on network and device load.
• Test order dependency — Tests that rely on data created by a previous test. If that test fails or runs in a different order, downstream tests break.
• Shared state — Tests that don't clean up after themselves (leftover users, records, session state) pollute the environment for subsequent tests.
• Element locator fragility — Locators based on XPath that include positional indices (//div[3]/span[2]) break when the page layout changes slightly.
• Network variability — Tests that depend on external APIs without timeouts or retries will fail intermittently under slow network conditions.

Fixes

• Replace all time.sleep() with WebDriverWait + expected conditions
• Use ID, data-testid, or aria-label locators over XPath
• Make each test fully self-contained — create its own data, clean up in tearDown
• Isolate external API calls with mocks or a VCR-style cassette library in unit/integration tests
• Track flaky tests in your bug tracker with the same priority as real defects

Common Automation Anti-Patterns to Avoid

Most automation projects fail not because of bad tooling, but because of bad engineering habits that compound over time. These are the anti-patterns I see most often — and the ones that consistently turn promising automation suites into maintenance nightmares.

The common thread running through every anti-pattern above is the same: short-term convenience traded for long-term cost. Each shortcut feels harmless when you first write it — and the debt accumulates silently until the suite is too slow to run, too flaky to trust, and too brittle to maintain. The discipline to avoid these patterns from day one is what separates an automation suite that lasts from one that gets thrown away and rewritten six months later.