How to handle an AJAX call in Selenium WebDriver

AJAXAsynchronous JavaScript & XML is a technique that allows a web page to retrieve small chunks of data from a web server in an asynchronous manner without having to reload the entire page every time.

Using AJAX, you can create dynamic, fast, and highly interactive web pages based on asynchronous calls exchanged in the background.

The use of this technique has grown substantially in recent years, some practical applications are as follows:

Chat and instant messaging
Autocomplete and autosuggest features
Instant login systems

Selenium WebDriver is a free, open-source, cross-platform framework that provides a common application programming interface (API) to create browser-based regression automation suites and tests.

One of the major challenges when handling AJAX calls in Selenium WebDriver is to know the loading time for the web page. Since this time varies, Selenium uses dynamic waits (WebDriverWait in combination with ExpectedCondition). It waits for an element while checking every second for a preset condition to be met and then, once this condition is satisfied, it continues to the next script command.

After this introduction, let us find out how to handle AJAX calls in Selenium WebDriver.

Example

This example illustrates how to deal with the Yahoo auto-suggestion widget backed by the AJAX technique while using Selenium WebDriver.

import os,time,logging
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions
os.environ['WDM_PROGRESS_BAR'] = str(0)
chromeOptions = webdriver.ChromeOptions()
chromeOptions.add_argument('--no-sandbox')
chromeOptions.add_argument('--disable-gpu')
chromeOptions.add_argument('--headless')
chromeOptions.add_argument('--disable-dev-shm-usage')
chromeOptions.add_argument("--start-maximized")
driver = webdriver.Chrome(executable_path=ChromeDriverManager().install(), chrome_options=chromeOptions)
driver.get("https://www.yahoo.com/")
time.sleep(10)
driver.find_element(By.XPATH, "//input[@name='p']").send_keys("educative.io")
time.sleep(10)
output_path = '/usercode/output'
driver.get_screenshot_as_file(os.path.join(output_path,'screenshot.png'))
auto_suggest_list_xpath = "//ul[@role='listbox']//li"
WebDriverWait(driver,20).until(expected_conditions.visibility_of_element_located((By.XPATH, auto_suggest_list_xpath)))
elements = driver.find_elements(By.XPATH,"//ul[@role='listbox']//li")
print("Count of Auto Suggestion Elements = ",len(elements))
for ele in elements:
    print("<br>",ele.text)
driver.close()

Explanation

Lines 1–7: Import the required Python libraries.
Line 9: We suppress the WebDriver default logging.
Lines 11–16: We initialize the Chrome WebDriver with the appropriate parameters.
Line 18–19: We launch the browser and fetch the URL targeting "yahoo.com".
Line 20: We suspend the execution for 10 seconds awaiting the loading of the specified URL.
Line 21: We find the search field and enter "educative.io" as the search criteria.
Line 22: We wait for 10 seconds.
Line 23–24: We grab a screenshot of the loaded page and output it.
Line 26: We specify the xpath of the auto-suggest list to search for.
Line 27: We call the WebDriverWait method for 20 seconds until the auto-suggest list is found in the web page.
Line 29: We seize the auto-suggest list.
Line 31: We display the count of elements fetched in the auto-suggest list.
Line 32–33: We iterate over the elements fetched and print each of them.
Line 35: We close the browser window opened by the Selenium WebDriver.

Free Resources

Learn in-demand tech skills in half the time

PRODUCTS

Mock Interview

New

Courses

Skill Paths

Projects

Assessments