Selenium WebDriver with Python — Complete Guide

What is Selenium WebDriver?

Selenium is an open-source browser automation framework that allows you to programmatically control web browsers — clicking buttons, filling forms, navigating pages, and verifying results — exactly as a human user would. Originally created by Jason Huggins in 2004 as an internal ThoughtWorks tool, Selenium has grown into the industry-standard choice for web UI automation and is maintained by the Selenium project under the Software Freedom Conservancy.

At its core, Selenium WebDriver is a W3C-standardised protocol that allows programming languages to send commands to a browser. This means your Python (or Java, JavaScript, C#, Ruby) code communicates with a browser-specific driver — ChromeDriver for Chrome, GeckoDriver for Firefox, EdgeDriver for Edge — which in turn controls the actual browser. The beauty of this architecture is language agnosticism: the same testing concepts translate across all supported languages.

Selenium supports all major browsers: Google Chrome, Mozilla Firefox, Microsoft Edge, Apple Safari, and Internet Explorer (legacy). For the vast majority of projects today, Chrome is the default choice given its developer tooling, ChromeDriver reliability, and market share. Firefox with GeckoDriver is an excellent secondary choice for cross-browser validation.

Selenium 4, released in October 2021 and actively developed since, brought several important improvements over the older Selenium 3. The most significant change was adopting the W3C WebDriver standard as the only supported protocol, dropping the legacy JSON Wire Protocol. This means Selenium 4 is more standards-compliant and has better interoperability. Selenium 4 also introduced native Chrome DevTools Protocol (CDP) integration, giving you direct access to browser internals — network interception, console log capture, geolocation spoofing — without any third-party plugins. Relative locators (finding elements relative to other elements on the page) arrived in Selenium 4, making complex DOM navigation far more readable. Selenium Grid 4 was completely rewritten from scratch to use a fully distributed architecture, replacing the old hub-and-node model with a more scalable event-driven design.

I have used Selenium extensively across different roles. At Virtusa, I helped build a Python-based Selenium regression suite for a large e-commerce client's checkout and account management flows. The suite grew from a handful of smoke tests to over 400 automated scenarios covering product search, cart operations, payment flow, and order tracking. Getting the architecture right from the start — specifically choosing the Page Object Model and standardising on explicit waits — was what made it possible to maintain that suite as the product evolved rapidly.

How Selenium Works (Architecture)

Understanding the Selenium architecture is not just academic — it directly informs why certain approaches (like implicit waits) are problematic and why network latency matters when running tests against remote browsers. The communication flow in a Selenium test involves at least three distinct layers, and knowing where your command is at any moment helps you debug failures much more effectively.

When your Python script calls driver.find_element(By.ID, "submit"), it is not directly reaching into the browser. Instead, the Selenium Python client library serialises that command into an HTTP request formatted according to the W3C WebDriver specification. This request is sent to a locally running process — the ChromeDriver binary — which acts as the intermediary. ChromeDriver understands both the WebDriver HTTP protocol and Chrome's internal debugging protocol, and it translates your high-level command into the low-level instructions Chrome can execute. Chrome then performs the action, returns the result to ChromeDriver, which serialises it back into an HTTP response and returns it to your Python client.

This multi-layer architecture explains several things. First, every Selenium command is an HTTP round trip — there is network overhead even on localhost. A test that makes 50 Selenium calls (find element, send keys, click, wait for element, get text…) is making 50+ HTTP requests. This is why keeping your test interactions focused and using the minimum number of WebDriver calls to verify what you need is important for speed. Second, because ChromeDriver is a separate process, it can become desynchronised from the browser under heavy load — this is one of the root causes of test flakiness, and it is precisely why explicit waits exist. Third, since the protocol is HTTP-based, you can point your Python client at a remote ChromeDriver just as easily as a local one — this is how Selenium Grid works.

It is also worth understanding what Selenium does not do. Selenium cannot interact with native desktop application windows outside the browser. It cannot handle browser certificate dialogs or OS-level file upload dialogs without workarounds. For those scenarios, you need complementary tools like pyautogui or OS-level automation. For API testing, there is no reason to drive a browser at all — use requests or httpx directly.

Installation & Setup

Setting up Selenium with Python correctly from the start will save you hours of troubleshooting later. The most common mistake beginners make is downloading ChromeDriver manually and then being surprised when it breaks after Chrome auto-updates. The correct approach is to use webdriver-manager, which automatically downloads the correct driver version for your installed browser.

Start by creating a dedicated virtual environment for your project. This isolates your test dependencies and prevents version conflicts with other Python projects on your machine.

# Create the project directory and virtual environment
mkdir selenium-tests && cd selenium-tests
python -m venv venv

# Activate virtual environment
# On macOS/Linux:
source venv/bin/activate
# On Windows:
venv\Scripts\activate

# Install Selenium and supporting packages
pip install selenium
pip install webdriver-manager
pip install pytest
pip install pytest-html

# Freeze dependencies for reproducible installs
pip freeze > requirements.txt

The selenium package is the core WebDriver library. webdriver-manager is a utility that detects your installed Chrome/Firefox version and downloads the matching driver binary automatically — no more manual driver management. pytest is the test runner we will use throughout this guide, preferred over unittest for its more readable syntax and rich plugin ecosystem. pytest-html generates a self-contained HTML report after each test run, invaluable for sharing results with stakeholders.

Your project structure should follow a clear layout from the start:

selenium-tests/
├── conftest.py          # pytest fixtures (shared driver setup)
├── pages/               # Page Object classes
│   ├── __init__.py
│   ├── login_page.py
│   └── checkout_page.py
├── tests/               # Test files
│   ├── __init__.py
│   ├── test_login.py
│   └── test_checkout.py
├── requirements.txt
└── pytest.ini           # pytest configuration

Create a pytest.ini at the root to configure default options:

[pytest]
testpaths = tests
addopts = --html=reports/report.html --self-contained-html
log_cli = true
log_cli_level = INFO

Your First Selenium Test

Before introducing page objects and fixtures, let us write a direct, self-contained Selenium script to verify that your environment is working correctly. This script opens Google, performs a search, and prints the resulting page title.

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from webdriver_manager.chrome import ChromeDriverManager
import time

# webdriver-manager downloads and caches the correct ChromeDriver version
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
driver.maximize_window()

# Navigate to Google
driver.get("https://www.google.com")

# Find the search input by its name attribute
search_box = driver.find_element(By.NAME, "q")

# Type search query and submit the form
search_box.send_keys("Selenium Python tutorial")
search_box.submit()

# Brief pause — only for this demo; real tests use explicit waits
time.sleep(2)

print("Title:", driver.title)

# Always quit the driver to close the browser and free resources
driver.quit()

Let us go through each line. WebDriverManager().install() checks your Chrome version and downloads a matching ChromeDriver binary to a local cache directory. The next time you run the script, it uses the cached binary if Chrome has not updated. driver.maximize_window() ensures consistent viewport sizing — many elements are only visible at certain screen widths, and running tests in a small default window causes failures that do not reproduce in the browser. driver.get(url) navigates to a URL and waits until the page fires the load event (similar to when a browser's loading spinner stops).

By.NAME is one of eight locator strategies — we use it here because Google's search box has had the name "q" for decades, making it a reliable identifier. send_keys() types the provided string into the element character by character. submit() submits the form that contains the element, equivalent to pressing Enter. Finally, driver.quit() closes the browser window AND terminates the ChromeDriver process. If you use driver.close() instead, the ChromeDriver process stays running, eventually consuming all your system's ports.

The time.sleep(2) call is present only to make this demo visible. In real test code, sleeping for a fixed duration is a serious anti-pattern: it either wastes time (if the page loads in 0.5 seconds) or causes failures (if the network is slow and the page takes 3 seconds). The correct approach — explicit waits — is covered in a dedicated section below.

Locator Strategies

Choosing the right locator strategy is one of the most consequential decisions you make when writing Selenium tests. A good locator is unique, stable (does not change when the UI is redesigned), and readable. A bad locator breaks tests constantly and forces you to spend more time maintaining tests than it would take to do manual testing.

CSS selectors and XPath are the two most powerful and most commonly used strategies, so they deserve a detailed comparison. CSS selectors are faster — browsers have highly optimised CSS engines that evaluate selectors as part of rendering, so Selenium's use of CSS selectors benefits from the same optimisation. CSS selectors are also more readable to anyone familiar with front-end development. However, CSS selectors cannot traverse upward in the DOM (you cannot select a parent element) and cannot match by text content.

XPath is more powerful for complex conditions — you can navigate to a parent, match by partial text, combine multiple attribute conditions, and use functions like contains(), starts-with(), and normalize-space(). The cost is verbosity and slower evaluation in large DOMs. The single most important rule about XPath: never use positional XPath like //div[3]/table/tr[2]/td[1]. These expressions break the moment anyone adds or reorders an element in the page — and someone always will.

# ID - most reliable locator, always prefer when available
driver.find_element(By.ID, "submit-btn")

# CSS Selector - attribute match
driver.find_element(By.CSS_SELECTOR, "input[type='email']")

# CSS Selector - class and descendant
driver.find_element(By.CSS_SELECTOR, ".login-form input.email")

# CSS Selector - data-testid (best practice with developer cooperation)
driver.find_element(By.CSS_SELECTOR, "[data-testid='login-button']")

# XPath - text content match (CSS cannot do this)
driver.find_element(By.XPATH, "//button[text()='Log In']")

# XPath - contains for partial text
driver.find_element(By.XPATH, "//button[contains(text(), 'Log')]")

# XPath - multiple attribute conditions
driver.find_element(By.XPATH, "//button[contains(@class,'primary') and @type='submit']")

# Relative Locators (Selenium 4 feature)
from selenium.webdriver.support.relative_locator import locate_with

# Find the password field that is directly below the email field
email_field = driver.find_element(By.ID, "email")
password_field = driver.find_element(
    locate_with(By.TAG_NAME, "input").below(email_field)
)

The Selenium 4 relative locators (below, above, to_left_of, to_right_of, near) are useful when elements lack good IDs or CSS selectors but have a stable visual relationship to a neighbouring element. They use JavaScript's getBoundingClientRect() to calculate proximity, so they depend on visual layout rather than DOM structure — useful for form fields that are always rendered in the same visual arrangement.

When working with development teams, advocate for adding data-testid attributes to interactive elements. These attributes have no visual or functional effect on the UI but provide stable, test-specific handles that survive redesigns. A CSS selector like [data-testid="checkout-button"] will still work after the button's text, class, and position have all changed.

Explicit vs Implicit Waits

Timing is the single most common source of test flakiness in Selenium. Modern web applications are asynchronous — JavaScript frameworks like React, Vue, and Angular update the DOM continuously in response to user actions and API responses. An element may not exist in the DOM the instant your Python code tries to find it, even if it appears immediately from a human user's perspective. The solution is waits — but not all wait strategies are equal.

An implicit wait sets a global timeout that tells WebDriver to poll for an element for up to N seconds before throwing a NoSuchElementException. It sounds convenient, but it has a critical flaw: implicit waits interact badly with explicit waits and cause unpredictable timeouts. If you set an implicit wait of 10 seconds and use an explicit wait with a 5-second timeout, Selenium may wait up to 10 seconds for a "not present" condition because the implicit wait overrides it. The recommendation from the Selenium team itself is to never use implicit waits in production test code.

The correct approach is explicit waits using WebDriverWait and expected_conditions. An explicit wait polls a specific condition on a specific element at a specified frequency until the condition is true or the timeout is reached. This gives you precise control and makes your intent explicit in the code.

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import NoSuchElementException, TimeoutException

wait = WebDriverWait(driver, 10)  # Wait up to 10 seconds

# Wait for element to be present in the DOM
element = wait.until(EC.presence_of_element_located((By.ID, "loading-spinner")))

# Wait for element to be visible AND clickable
login_btn = wait.until(EC.element_to_be_clickable((By.ID, "loginBtn")))
login_btn.click()

# Wait for specific text to appear in an element
wait.until(EC.text_to_be_present_in_element((By.ID, "status-msg"), "Success"))

# Wait for URL to change after navigation
wait.until(EC.url_contains("/dashboard"))

# Wait for URL to match exactly
wait.until(EC.url_to_be("https://app.example.com/dashboard"))

# Wait for an element to disappear (e.g., loading overlay)
wait.until(EC.invisibility_of_element_located((By.CLASS_NAME, "loading-overlay")))

# Wait for number of windows/tabs to change
wait.until(EC.number_of_windows_to_be(2))

A Fluent Wait is an advanced form of explicit wait that gives you finer control over polling frequency and which exceptions to suppress during polling. Use it when the default 500ms polling interval is too slow (e.g., for very fast UI transitions) or when you want to ignore specific exceptions like StaleElementReferenceException that occur when the DOM updates while you are waiting.

from selenium.webdriver.support.wait import WebDriverWait
from selenium.common.exceptions import NoSuchElementException, StaleElementReferenceException

# Poll every 500ms, ignore NoSuchElement and StaleElement during the wait
fluent_wait = WebDriverWait(
    driver,
    timeout=20,
    poll_frequency=0.5,
    ignored_exceptions=[NoSuchElementException, StaleElementReferenceException]
)

element = fluent_wait.until(
    EC.presence_of_element_located((By.ID, "dynamic-result"))
)
print(element.text)

A practical pattern I use everywhere is creating a helper function that wraps element retrieval with a built-in explicit wait, so every element access in page objects automatically waits for the element to be interactable:

def wait_and_find(driver, locator, timeout=10):
    """Returns an element only after it is both present and visible."""
    return WebDriverWait(driver, timeout).until(
        EC.visibility_of_element_located(locator)
    )

Page Object Model (POM)

The Page Object Model is a design pattern, not a Selenium feature. It is a way of organising your test code so that each web page (or major component of a page) is represented by a dedicated Python class. This class encapsulates all locators and interaction methods for that page. Tests then call methods on these page objects instead of using WebDriver commands directly.

The key benefit of POM is that when a UI element changes — for example, the login button's ID is renamed from "loginBtn" to "login-submit" — you make that change in exactly one place (the LoginPage class), and all tests that use that class immediately get the fix. Without POM, you would need to hunt through every test file for every occurrence of that locator. In a 400-test suite, that hunt is painful.

Here is a complete, production-ready page object for a login page:

# pages/login_page.py
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

class LoginPage:
    URL = "https://app.example.com/login"

    # Locators defined as class-level tuples — easy to update in one place
    EMAIL     = (By.ID, "email")
    PASSWORD  = (By.ID, "password")
    LOGIN_BTN = (By.ID, "loginBtn")
    ERROR_MSG = (By.CSS_SELECTOR, ".error-message")
    WELCOME   = (By.CSS_SELECTOR, "h1.welcome")

    def __init__(self, driver):
        self.driver = driver
        self.wait = WebDriverWait(driver, 10)

    def open(self):
        """Navigate to the login page and wait until it is ready."""
        self.driver.get(self.URL)
        self.wait.until(EC.presence_of_element_located(self.EMAIL))
        return self  # Enables method chaining: LoginPage(driver).open().login(...)

    def login(self, email, password):
        """Enter credentials and submit the login form."""
        email_field = self.wait.until(EC.visibility_of_element_located(self.EMAIL))
        email_field.clear()
        email_field.send_keys(email)

        password_field = self.driver.find_element(*self.PASSWORD)
        password_field.clear()
        password_field.send_keys(password)

        self.driver.find_element(*self.LOGIN_BTN).click()

    def get_error(self):
        """Return the text of the validation error message."""
        return self.wait.until(
            EC.visibility_of_element_located(self.ERROR_MSG)
        ).text

    def is_logged_in(self):
        """Check whether login succeeded by verifying dashboard URL."""
        try:
            self.wait.until(EC.url_contains("/dashboard"))
            return True
        except Exception:
            return False

    def get_welcome_message(self):
        """Return the welcome heading text after successful login."""
        return self.wait.until(
            EC.visibility_of_element_located(self.WELCOME)
        ).text

Notice that locators are stored as tuples of (By.strategy, "value"). When you use them with find_element(), you unpack them with the asterisk operator: self.driver.find_element(*self.EMAIL). This is a Pythonic pattern that keeps locator definitions clean and allows you to pass them directly to WebDriverWait.until() as well.

pytest Integration

pytest transforms Selenium tests from standalone scripts into a maintainable, organised test suite. The key mechanism is fixtures — functions decorated with @pytest.fixture that provide shared resources (like a WebDriver instance) to tests that declare them as parameters. Fixtures handle setup and teardown automatically and can be scoped to function, class, module, or session level.

The most important fixture in any Selenium project is the browser fixture, defined in conftest.py:

# conftest.py
import pytest
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager

@pytest.fixture(scope="function")
def driver():
    """Provides a fresh WebDriver instance for each test function."""
    options = Options()
    options.add_argument("--no-sandbox")           # Required in Docker/CI
    options.add_argument("--disable-dev-shm-usage") # Prevents crashes on Linux
    options.add_argument("--disable-gpu")           # Stability improvement in CI
    options.add_argument("--window-size=1920,1080") # Consistent viewport

    driver = webdriver.Chrome(
        service=Service(ChromeDriverManager().install()),
        options=options
    )
    driver.implicitly_wait(0)  # Explicitly disable implicit waits
    driver.set_page_load_timeout(30)  # Fail fast if page takes too long

    yield driver  # Test runs here

    # Teardown: always quit, even if test fails
    driver.quit()

# tests/test_login.py
from pages.login_page import LoginPage

class TestLogin:

    def test_successful_login(self, driver):
        """A valid user with correct credentials should reach the dashboard."""
        page = LoginPage(driver)
        page.open()
        page.login("user@example.com", "Password123")
        assert page.is_logged_in(), "Expected dashboard URL after successful login"

    def test_invalid_password_shows_error(self, driver):
        """Wrong password should display an appropriate error message."""
        page = LoginPage(driver)
        page.open()
        page.login("user@example.com", "wrongpassword")
        error = page.get_error()
        assert "Invalid" in error, f"Expected 'Invalid' in error, got: '{error}'"

    def test_blank_email_shows_required_error(self, driver):
        """Submitting without email should show a required field error."""
        page = LoginPage(driver)
        page.open()
        page.login("", "Password123")
        error = page.get_error()
        assert "required" in error.lower(), f"Expected required field error, got: '{error}'"

Using scope="function" on the driver fixture means each test gets a completely fresh browser instance. This is the safest choice because it guarantees test isolation — a logged-in session from one test cannot affect the next. The trade-off is speed: launching Chrome for every test adds several seconds each. For suites where startup time is a bottleneck, you can use scope="class" and share the driver across all tests in a class, as long as each test resets the application state (navigating back to the starting URL, logging out, etc.) in a setup method.

Run your tests and generate an HTML report with a single command:

# Run all tests with HTML report
pytest tests/ -v

# Run a specific test file
pytest tests/test_login.py -v

# Run tests matching a keyword
pytest tests/ -k "login" -v

# Run and generate HTML report explicitly
pytest tests/ --html=reports/report.html --self-contained-html

Data-Driven Testing

Data-driven testing means running the same test logic with multiple sets of inputs, each set being its own test case. This is essential for validating form validation rules, boundary conditions, and multiple user types. In pytest, @pytest.mark.parametrize is the built-in mechanism for this pattern.

import pytest
from pages.login_page import LoginPage

class TestLoginValidation:

    @pytest.mark.parametrize("email,password,expected_error", [
        ("",                "Password123",  "Email is required"),
        ("not-an-email",    "Password123",  "Please enter a valid email"),
        ("user@test.com",   "",             "Password is required"),
        ("user@test.com",   "short",        "Password must be at least 8 characters"),
        ("user@test.com",   "NOLOWER1!",    "Password must contain a lowercase letter"),
    ])
    def test_login_validation_errors(self, driver, email, password, expected_error):
        """Validate that appropriate error messages appear for invalid input combinations."""
        page = LoginPage(driver)
        page.open()
        page.login(email, password)
        actual_error = page.get_error()
        assert expected_error in actual_error, (
            f"Input: email='{email}', password='{password}'\n"
            f"Expected error containing: '{expected_error}'\n"
            f"Actual error: '{actual_error}'"
        )

Each tuple in the parametrize list becomes a separate test case with its own name derived from the parameter values. pytest will report test_login_validation_errors[not-an-email-Password123-Please enter a valid email] as a distinct test in the output, making it easy to identify exactly which input combination failed.

For larger datasets — say, testing a search engine with 50 different query terms or validating address forms across 30 countries — defining parameters inline becomes unwieldy. A cleaner approach is to read from a CSV or JSON file:

import csv
import json
import pytest
from pages.login_page import LoginPage

def load_login_scenarios():
    """Load test data from CSV file."""
    scenarios = []
    with open("test_data/login_scenarios.csv") as f:
        reader = csv.DictReader(f)
        for row in reader:
            scenarios.append((row["email"], row["password"], row["expected_error"]))
    return scenarios

@pytest.mark.parametrize("email,password,expected_error", load_login_scenarios())
def test_login_from_csv(driver, email, password, expected_error):
    page = LoginPage(driver)
    page.open()
    page.login(email, password)
    assert expected_error in page.get_error()

The CSV file (test_data/login_scenarios.csv) would have headers: email,password,expected_error. This pattern scales well — non-technical team members or business analysts can add new test scenarios by editing the CSV file without touching any Python code.

Running Tests Headless

Headless mode runs Chrome (or Firefox) without rendering a visible browser window. The browser process still exists and executes JavaScript, CSS, and all the normal browser behaviour — it simply does not open a GUI window. This makes tests faster (no rendering overhead) and possible to run in environments without a display, such as Linux CI servers and Docker containers.

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager

options = Options()
# Use --headless=new for Chrome 112+ (the improved headless implementation)
options.add_argument("--headless=new")
options.add_argument("--window-size=1920,1080")  # Set viewport since there's no screen
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")
options.add_argument("--disable-gpu")

driver = webdriver.Chrome(
    service=Service(ChromeDriverManager().install()),
    options=options
)
driver.get("https://www.example.com")
print("Title:", driver.title)
driver.quit()

Note the distinction between --headless (the old mode, deprecated since Chrome 112) and --headless=new (the modern implementation introduced in Chrome 112). The new headless mode uses the full Chrome renderer rather than a separate stripped-down implementation, which means it behaves much more like a real browser and has fewer quirks — particularly around JavaScript execution and rendering of complex CSS.

For Firefox headless, the equivalent is:

from selenium.webdriver.firefox.options import Options as FirefoxOptions

firefox_options = FirefoxOptions()
firefox_options.add_argument("--headless")
driver = webdriver.Firefox(options=firefox_options)

In CI environments, always run headless. In your local development workflow, run with a visible browser when debugging failures — seeing the browser interact with the page in real time is invaluable for diagnosing why a test is failing.

Parallel Testing with pytest-xdist

Running a 400-test suite sequentially can take 30–60 minutes if each test averages 5–10 seconds. Parallel execution multiplies your throughput by running multiple tests simultaneously. The easiest way to parallelise a pytest Selenium suite is with pytest-xdist.

# Install the plugin
pip install pytest-xdist

# Run on 4 CPU cores in parallel
pytest tests/ -n 4

# Auto-detect the number of available CPUs and use all of them
pytest tests/ -n auto

# Run in parallel with HTML report
pytest tests/ -n auto --html=reports/report.html --self-contained-html

For parallel execution to work correctly, your tests must be independent — no test should rely on state created by another test, and no two tests should share a browser instance. The function-scoped driver fixture described earlier in the pytest section satisfies both requirements: each test worker gets its own Chrome process and its own WebDriver session.

For scaling beyond a single machine — for example, running 50 tests simultaneously across multiple physical computers or cloud VMs — Selenium Grid is the right tool. Selenium Grid 4 uses a fully distributed architecture where a Router, Distributor, Session Map, and Node components work together. The simplest way to start Grid locally is with Docker Compose:

version: '3'
services:
  selenium-hub:
    image: selenium/hub:4.20.0
    ports:
      - "4444:4444"
      - "4442:4442"
      - "4443:4443"

  chrome-node:
    image: selenium/node-chrome:4.20.0
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
    deploy:
      replicas: 4  # Start 4 Chrome nodes

Run with docker-compose up -d, then point your WebDriver to the Grid hub:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
driver = webdriver.Remote(
    command_executor="http://localhost:4444/wd/hub",
    options=options
)
driver.get("https://www.example.com")
driver.quit()

CI/CD with GitHub Actions

Integrating your Selenium suite into GitHub Actions means tests run automatically on every pull request, catching regressions before they reach the main branch. The following workflow installs Python, installs dependencies, sets up Chrome, runs tests, and uploads the HTML report as a build artifact — all on a clean Ubuntu runner.

name: Selenium Tests

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Set up Python 3.12
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'
          cache: 'pip'

      - name: Install Python dependencies
        run: pip install -r requirements.txt

      - name: Set up Chrome browser
        uses: browser-actions/setup-chrome@latest

      - name: Run Selenium tests (headless, parallel)
        run: pytest tests/ --html=report.html --self-contained-html -n auto -v
        env:
          APP_URL: ${{ secrets.APP_URL }}
          TEST_USER_EMAIL: ${{ secrets.TEST_USER_EMAIL }}
          TEST_USER_PASSWORD: ${{ secrets.TEST_USER_PASSWORD }}

      - name: Upload HTML test report
        uses: actions/upload-artifact@v4
        if: always()  # Upload even if tests fail
        with:
          name: selenium-test-report
          path: report.html
          retention-days: 30

The if: always() on the upload step is critical — it ensures the HTML report is uploaded even when tests fail, which is exactly when you need it most. Sensitive data like application URLs and test credentials are stored in GitHub repository secrets and injected as environment variables at runtime, never hardcoded in the workflow file.

In your test code, read these environment variables with os.environ.get():

import os

APP_URL = os.environ.get("APP_URL", "https://staging.example.com")
TEST_EMAIL = os.environ.get("TEST_USER_EMAIL", "test@example.com")
TEST_PASSWORD = os.environ.get("TEST_USER_PASSWORD", "testpassword")

Best Practices

After years of building and maintaining Selenium frameworks at Virtusa and beyond, here are the ten practices that separate stable, maintainable test suites from fragile, high-maintenance ones.

1. Always use explicit waits — never time.sleep() in production tests. Fixed sleeps make tests slow when the application is fast and flaky when it is slow. Explicit waits adapt to actual application state. This single change will reduce flakiness by more than any other optimisation.

2. Implement Page Object Model from day one. The POM investment pays for itself the first time a locator changes. In a properly structured POM, updating one locator in one file fixes every test that uses that element. Without POM, locator changes become a painful search-and-replace across hundreds of test files.

3. Use data-testid attributes in cooperation with developers. If you have the ability to add attributes to the application you are testing, add data-testid attributes to all interactive elements. These attributes are test-specific handles that survive UI redesigns because no developer will change them without QA sign-off.

4. Run tests against a stable, dedicated test environment — not production. Tests create data, click buttons, and may trigger external integrations. A stable staging environment that closely mirrors production but has no real users is the correct target for automation.

5. Keep tests independent and idempotent. Each test should set up its own preconditions, run its scenario, and clean up after itself. A test that depends on another test having run first is a time bomb — it will fail mysteriously when test order changes during parallelisation.

6. Write test names that describe the scenario, not the implementation. test_user_with_expired_subscription_sees_upgrade_prompt is infinitely more useful than test_login_scenario_3 when you are reading a CI failure report at 2 AM.

7. Prefer CSS selectors and IDs over XPath where possible. CSS selectors are faster, more readable, and familiar to anyone who knows CSS. Reserve XPath for situations where CSS genuinely cannot express the condition — text content matching being the most common case.

8. Take a screenshot on test failure. Add a pytest hook in conftest.py that captures a screenshot when any test fails and saves it with the test name and timestamp. This single change transforms debugging from guesswork into a 5-second diagnosis.

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    report = outcome.get_result()
    if report.when == "call" and report.failed:
        driver = item.funcargs.get("driver")
        if driver:
            import os, time
            os.makedirs("screenshots", exist_ok=True)
            filename = f"screenshots/{item.name}_{int(time.time())}.png"
            driver.save_screenshot(filename)
            print(f"\nScreenshot saved: {filename}")

9. Use environment variables for all sensitive and environment-specific data. URLs, usernames, passwords, API keys — none of these should be hardcoded in test files or committed to version control. Use environment variables in CI and a local .env file (git-ignored) for development.

10. Track and fix flaky tests immediately — do not skip or ignore them. A flaky test that sometimes passes and sometimes fails is worse than a consistently failing test because it erodes team trust in the entire suite. When a test is flaky, investigate the root cause (usually a missing wait or state leak), fix it, and add a comment explaining what was causing the flakiness so it does not regress.