Timing races in Selenium 2: implicit waits vs explicit waits

Jan 14, 2012   //   by Daniel Kranowski   //   Algorithms  //  4 comments

The most common debugging experience in automated Selenium testing is the ubiquitous timing race, where your code is rushing to access a WebElement that hasn’t been fully loaded into the driver yet. If you’re stepping through the test one line at a time it will work just fine, but when you let it run in batch mode at its own speed, then it crashes with a complaint like NoSuchElementException. When you’ve got a suite of a hundred or more Selenium testcases you definitely don’t want to step through them one line at a time, it needs to be automated.

The timing race stems from the fact that WebDriver.get(), driver.navigate().to(), driver.navigate().refresh() are nonblocking invocations that start a page load and return immediately, before the load thread is done. Your followup call to driver.findElement(…) will throw NoSuchElementException if the load thread hasn’t populated the desired element yet. There are two ways to wait for the element to be ready:

  1. Explicit waitWebDriverWait.until(condition-that-finds-the-element)
  2. Implicit waitdriver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);

With the explicit wait you have to insert a WebDriverWait.until(condition) before each findElement(). With the implicit wait it’s done for you, globally, after you make the driver timeouts setting. In general I like implicit wait, because the explicit WebDriverWait calls turn into clutter. I can’t think of a good reason why implicit wait is not the default.

Here’s what the explicit wait looks like:

public void testExplicitWait(WebDriver driver) {
   // Assume we're already on a fully loaded webpage.
   driver.findElement(By.id("myForm")).submit();
   // Now the next page starts loading.

   // Here's the explicit wait.
   WebDriverWait wdw = new WebDriverWait(driver, 10);
   ExpectedCondition<Boolean> condition = new ExpectedCondition<Boolean>() {
      @Override
      public Boolean apply(WebDriver d) {
         WebElement result = d.findElement(By.className("myResult"));
         return "The Next Page".equals(result.getText());
         // Returns true as soon as an element of class 'myResult' is found
         // where the element's text value is "The Next Page".
     }
   };
   wdw.until(condition); // Won't get past here til timeout or element is found

   // It is safe to operate on the element now.
   WebElement result = driver.findElement(By.className("myResult"));
   Assert.assertEquals("The Next Page", result.getText());
}

Compare that to the implicit wait, which is much more concise:

public void testImplicitWait(WebDriver driver) {
   // This setup would be done once per driver execution.
   driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);

   driver.findElement(By.id("myForm")).submit();
   // Now the next page starts loading.

   // An attempt to find the element implicitly waits til it is ready.
   WebElement result = driver.findElement(By.className("myResult"));
   Assert.assertEquals("The Next Page", result.getText());
}

implicitlyWait only affects findElement, it does not affect other WebDriver methods like driver.getCurrentUrl() or driver.manage().deleteAllCookies(). So you can very easily and accidentally get back the old url and/or delete cookies on the old subdomain, not the new url/subdomain you were expecting… unless you call findElement first to force an implicit wait:

private void forceImplicitWait(WebDriver driver) {
   driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);

   driver.findElement(By.id("myForm")).submit();
   // Now the next page starts loading ... but it's not done yet.

   printCurrentUrl(driver); // Timing race!  Might print "/oldPath".

   // An attempt to find an element on the next page forces an implicit wait
   // til it is ready.  At that point the driver will have the new URL.
   driver.findElement(By.className("myResult"));

   printCurrentUrl(driver); // Safely prints "/newPath".
}

private void printCurrentUrl(WebDriver driver) {
   URL currentUrl = null;
   try {
      currentUrl = new URL(driver.getCurrentUrl());
   }
   catch (MalformedURLException e) {
      System.out.println("malformedUrl");
   }
   System.out.println(currentUrl.getPath());
}

How long to wait?

One question is “what element should I wait for?” It depends on what your test is trying to do, because if the next step is to operate on element X then of course just wait for findElement(By.id(“X”)). If you’re about to operate on X, Y, and Z then you could add three explicit waits, or just add one explicit wait for Z if you know it’s the last one to load, but trying to do “just one wait” will probably make your test more brittle. That’s why I like implicit waits, it lets Selenium do the polling for you, and it waits for exactly the element you want.

There’s a train of thought that sometimes comes up here, “let’s just wait for the whole page to load” and be done with it. Perhaps by hooking into the window.onLoad event. The theory is that you’ll intentionally lose some test performance to gain safety, robustness and reliability in your test suite. It’s a nice idea, but ultimately unhelpful, primarily because onLoad is only the beginning. onLoad is an event that countless webpages are already listening for, whether in <body onload="doSomething()"/>, or its jQuery equivalent $(document).ready(function() { doSomething(); }), and “doSomething()” often means “modify the DOM.” It’s better to just wait for the specific element on which your test operates, since it’s faster and more accurate.

There are more pitfalls here. Some webpages are so dynamic that an element identified by selector X will render multiple times. A classic example is the login page for Yahoo! Mail, where the username/password form is initialized from HTML, then the page’s JavaScript checks your cookies and then may choose to rip out and replace the username/password form with a brand new element hierarchy. The script operates at human visible speed so you can watch the page adjusting itself. The username/password fields have the same CSS ids before and after the replacement, so there’s a timing race if you write Selenium code to try to automate this login. The test code can’t tell whether it has found the “before” element or the “after” element.

If the Selenium test calls sendKeys() on the Before element, it could get interrupted when the After element loads. It’s almost comical, you’ve written the first half of your username in Before and the second half in After. This of course breaks the test and breaks your automation. The solution is site-specific, not systematic: with intimate knowledge of the DOM and scripts on this page you can find other clues that tell you whether the element is Before or After. For example you’ll wait for the existence of an element that occurs only when After has loaded, or you’ll make cookie settings to avoid having After load in the first place. Unfortunately this kind of intimate knowledge leads to brittle tests. You’ll want to abstract out this knowledge in a PageObject.

4 Comments

  • Great article – it helped me with my understanding

  • Great post.

    Do you happen to know if there’s any negative effects to just waiting forever?

    I’m working on an interface that will tie into a web application front-end based on a card swipe and prepopulate a few fields (basically a systray app that will manipulate the DOM of Firefox). All the fields are on page 3 of the kiosk web-app, so I would just wait for one of them to appear. But the kiosk could potentially sit there with a FF window open for the weekend until Monday morning when someone comes in and swipes their card.

    Thanks!

    • Looks like all the WebDriverWait constructors have a timeout parameter (long timeOutInSeconds). You could set it to a very large number to avoid timeout over the weekend.

      It sounds like you are trying to use Selenium for core (non-test) functionality? You plan to have the card-swipe/systray app use Selenium to launch the browser, then wait for a swipe, then the app populates fields in the browser? Or the swipe unlocks a keyboard and the Selenium instance waits for the user key input? Just wondering.

  • This is a good article but it does a favor to Implicit wait. Here are the 2 advantages of using Explicit wait over implicit wait:

    1. Elements may require different load time. Let’s say one element can be loaded in 30 seconds and second element may take upto 2 minutes to load and a third one can take upto 3 minutes. We can set default timeout to 3 minutes, but is it advisable?

    2. Sometimes we have to wait for element to hide. This happens in AJAX when a ‘loading’ icon is visible on the top and we wait for that icon to hide before moving to next action. This is not achievable using Implicit wait because implicit wait checks for element to enable on DOM and in this scenario, DOM already has that element enabled.

    Warn Regards,
    Umesh

Please share your thoughts