com.smartgwt.client.docs

Interface AutomatedTesting

public interface AutomatedTesting

Automated Testing

Smart GWT supports automated testing with a variety of tools.

Selenium

Smart GWT includes a free, custom Selenium extension for robust record and playback of tests, including the ability to record on one browser and play back on others, support for Selenium Remote Control allowing tests to be written in a variety of programming languages and run as scripts, as well as Smart GWT-specific enhancements to the Selenium IDE.

These extensions can be found in the selenium/ directory and a user guide can be found here.

Selenium supports writing test code in any programming language via Seleniun RC. By writing Selenium RC test cases in Java, you can drive them from JUnit, hence creating automated tests that can be run from the command line or via Continuous Integration servers such as Hudson, allowing for running tests on checkins to source control or in overnight batch runs.

Services such as SauceLabs OnDemand allow you to run the actual browsers in the cloud, tunneling back to a private network via an encrypted channel, so that you do not need to set up Selenium RC servers with appropriate browsers installed.

For apps requiring load testing, also take a look at BrowserMob, which allows you to run Selenium tests with thousands of browsers at once against a test deployment.

JUnit + Selenium RC

Explore JUnit + Selenium RC, where we walk through a JUnit test built using Selenium IDE and targeting a Smart GWT Showcase example.

TestRunner

TestRunner is a system for automatically running a suite of Selenium tests, commiting the results to a database, and reporting any regressions (or fixes) via email.

GwtTestCase

GWT includes a way to run a GWT application under JUnit, running your GWT application in a "headless" browser. This is a very limited testing approach appropriate for certain unit tests only - it cannot replace events such as clicks, because by default it doesn't run in actual browser (instead it runs in a simulator called HtmlUnit).

Note that running tests under HtmlUnit can lead to false failures in a variety of areas, including network communication and XML processing, where HtmlUnit's behaviors do not correspond to any real browser. Note that, if you find that a test fails under HtmlUnit but would not fail in a real browser, this will not be regarded as a SmartGWT bug.

If you use GwtTestCase at all, Isomorphic recommends that the majority of your tests are executed using the runStyle option that allows GwtTestCase to run under a real browser via Selenium.

Also note, GwtTestCase has a bug where it does not run onModuleLoad() for included GWT modules. To make sure SmartGWT's onModuleLoad() runs, add a gwtSetUp() implementation like so:

    public class SgwtTest extends GWTTestCase {
       public void gwtSetUp() {
               new SmartGwtEntryPoint().onModuleLoad();                
       }
       ...
  

You may need to add similar manual calls for other GWT modules you inherit which expect to have their onModuleLoad() method called normally.

WebDriver / "Selenium 2"

WebDriver, which is now part of Selenium 2, uses a different basic architecture in which extensions are added to each browser in order to drive tests, instead of doing so from JavaScript.

Support for WebDriver-based testing for Smart GWT is now available with the same custom locator strategies and custom commands as we provide for Selenium 1.0. However, we continue to recommend Selenium 1.0 rather than WebDriver-based Selenium 2, because:

1. WebDriver is more complex to install: WebDriver requires installing support for each browser where you want to run tests, and in some cases multiple WebDriver plugins for multiple versions of the browser
2. WebDriver has version / browser support issues: Selenium 1.0 generally works with any standards-compliant browser. Because WebDriver requires deeper integration with the browser, new browser releases require updated WebDriver extensions. This is a particular issue with the rapid pace of new releases of Firefox, where the WebDriver extension becomes disabled by an update of Firefox, but WebDriver test will still run in a "non-native" mode that behaves erratically. Unfortunately, there is no way we can detect and warn users about this; this is a general issue with WebDriver and Firefox, not specific to Smart GWT.
3. Mobile testing issues: Mobile testing is supported only for certain devices, requires that an application be installed on the devices, doesn't run a normal browser (rather an embedded browser window inside an application), which can introduce spurious issues during playback. In contrast, while Selenium RC doesn't support mobile, with Selenium 1.0 you can use Selenium Core to test any mobile device that supports JavaScript without installation of an app. Both situations have drawbacks but we feel that Selenium 1.0 has an overall advantage over WebDriver here.
4. Java skills required: Tests created in Selenium IDE and stored in Selenese can be executed by a variety of tools without requiring Java skills, including our own TestRunner. Most ways of running WebDriver tests involve Java coding skills or at least the ability to work with a Java IDE. This tends to mean that all QA personnel must either have Java skills or drain the time of Java developers on repetitive tasks.

Ultimately, our current recommendation is to use Selenium 1.0 and Selenium RC exclusively or at least primarily. If there are critically important tests that you can only build via WebDriver (rare: the most common such case is testing file upload - see below), use WebDriver for those tests only, or use manual testing for those tests.

WebDriver Usage

When using WebDriver, we recommend using Selenum IDE to record tests, and storing tests in Selenese (as with Selenium RC / 1.0). WebDriver is not normally able to execute Selenese tests, but we provide a Java class SeleneseRunner that can be used to:

• execute Selenese directly from the command line
• execute Selenese from inside a Java program (eg, as part of a JUnit test)
• convert a Selenese test to Java code (as a JUnit test)

See the server-side JavaDoc for com.isomorphic.webdriver.SeleneseRunner for more information on how to use these features.

NOTE: Selenium IDE has an option to export tests as WebDriver-compatible code. Do not use this feature, it exports useless code that doesn't understand custom commands, custom locators, or other key features of Selenium IDE. Use SeleneseRunner instead.

WebDriver Classes overview

Storing and executing Selenese tests recorded in the Selenium IDE is recommended as the primary approach for using WebDriver. However, for certain rare tests it can make sense to use WebDriver Java support directly.

Smart GWT support for WebDriver is based around 3 different Java classes:

1. ByScLocator: This implements the ability to find WebElements or WebDriver "By" objects using Smart GWT Locator strings. See UsingSelenium for more background on Locator strings and how to obtain them. Given a locator String, example usage is:

     ByScLocator.scLocator("//ListGrid[ID=\"countryList\"]/body/row[countryCode=US||0]/col[fieldName=countryCode||0]")
     

2. SmartClientWebDriver: This is an abstract class which provides a number of different methods for interacting with the browser, such as:
   • open a browser at a particular URL
   • find the element or elements which match a given "By" object (either ByScLocator, or a standard WebDriver locator)
   • perform events and operations (click, drag, select etc)
   • perform custom Smart GWT validations / state checks, such as whether a grid has loaded data
   Three concrete implementations of SmartClientWebDriver are provided: SmartClientFireFoxDriver, SmartClientChromeDriver and SmartClientIEDriver. There is also a SmartClientRemoteWebdriver class which allows the injection of a manually configured RemoteWebDriver instance. This might be necessary, for example, for use with Selenium Grid.
3. ScAction: a Smart GWT-specific version of the standard WebDriver "Action" class, providing a builder pattern to create a sequence of operations which can then be perform()ed.

These classes are packaged in the library isomorphic_webdriver.jar, which can be found in the directory lib-WebDriverSupport (along with several 3rd-party supporting libraries).This directory can be found at the top level of the downloaded Smart GWT zip package.

General information regarding WebDriver can be found here. Setup for WebDriver is more complex than for classic Selenium: The basic Java package includes drivers for FireFox (subject to important version limitations as described above), but additional drivers must be downloaded for Google Chrome and Internet Explorer.

File Upload Example Test

As discussed above, one advantage which WebDriver does have over Classic Selenium is the ability to test file upload. It is still limited in that if a click is triggered on the file selection button an OS native file selection dialog will be triggered in which case the test will be suspended until the file is manually selected. To avoid this, the sendKeys() method can be used to enter the file location.

Sample code:

     /**
      * The following test runs against localhost and requires a small (< 5mb) image to be in /tmp/image.jpg
      */
     public void fileUploadGWT() throws Exception {
         final String basePath = "//VLayout[ID=\"isc_Showcase_1_0\"]/member[Class=HLayout||index=0||length=2|"
                                +"|classIndex=0||classLength=1]/member[Class=HLayout||index=0||length=2||classIndex=0|"
                                +"|classLength=1]/member[Class=Canvas||index=1||length=2||classIndex=0||classLength=1]"
                                +"/child[Class=TabSet||index=0||length=1||classIndex=0||classLength=1]/paneContainer/"
                                +"member[Class=VLayout||index=1||length=2||classIndex=0||classLength=1]/"
                                +"member[Class=VLayout||index=1||length=2||classIndex=0||classLength=1]/"
                                +"member[Class=HLayout||index=1||length=2||classIndex=0||classLength=1]/"
                                +"member[Class=HLayout||index=0||length=1||classIndex=0||classLength=1]/";
         final String formPath = basePath + "member[Class=DynamicForm||index=0||length=3||classIndex=0||classLength=1]";
         final String tilesPath = basePath + "member[Class=VLayout||index=2||length=3||classIndex=0||classLength=1]/"
                                           + "member[Class=TileGrid||index=2||length=4||classIndex=0||classLength=1]/tile";
         SmartClientWebDriver driver = new SmartClientFirefoxDriver();
         driver.setBaseUrl("http://localhost:8888/");
         driver.get("index.html#upload_sql", true);
 
         final int origSize = driver.findElements(ByScLocator.scLocator(tilesPath)).size();
         By uploadForm = ByScLocator.scLocator(formPath);
         WebElement form = driver.findElement(uploadForm);
       
         By titleInput = ByScLocator.scLocator(formPath + "/item[name=title||title=Title||index=0||Class=TextItem]/element");
         driver.click(titleInput);
         driver.sendKeys(titleInput, "test image: " + origSize);
         
         WebElement findElement = form.findElement(By.xpath("//input[@type='FILE']"));
         /*
          * The following causes a native dialog to be created which prevents further progress. Do NOT uncomment!
          * We just have to sendKeys() to it
          */
         //findElement.click(); 
         
         findElement.sendKeys("/tmp/image.jpg"); // A local file. Please change accordingly
 
         By saveButton = ByScLocator.scLocator(formPath + "/item[title=Save||index=2||Class=ButtonItem]/button/");
         driver.waitForElementClickable(saveButton);
         driver.click(saveButton);
         /*
          * Note the following fails once the grid contains more than 3 rows of data as the index becomes inconsistent
          * as tiles scrolled out of site are removed and the indices change
          */
         By tile = ByScLocator.scLocator(tilesPath + "[Class=SimpleTile||index="+(origSize)+"||length="+(origSize+1)
                                                       + "||classIndex="+(origSize)+"||classLength="+(origSize+1)+"]/");
         driver.waitForElementClickable(tile);
         WebElement tile1 = driver.findElement(tile);
         assertEquals("test image: " + origSize, tile1.getText());
         assertEquals(origSize + 1, driver.findElements(ByScLocator.scLocator(tilesPath)).size());
         driver.close();
         driver.quit();
     }
  

WebDriver Troubleshooting

There is a known issue that native events do not work with IE in Windows 8/8.1 that may manifest in WebDriver as clicks having no effect. One potential workaround is to disable native events:

     DesiredCapabilities caps = DesiredCapabilities.internetExplorer();
     caps.setCapability("nativeEvents",false);
     SmartClientWebDriver driver = new SmartClientIEDriver(caps);

It's also been reported that changing the second line above to:

     caps.setCapability("requireWindowFocus", true);

also resolves the issue, with the side effect that WebDriver then moves the mouse cursor.

In some versions of Internet Explorer, it's been reported that you must add the URL targeted by WebDriver to the "Trusted Sites" under Internet Options >> Security in order to allow the browser to communicate properly with Selenium. A discussion of the setup needed to use WebDriver's InternetExplorerDriver can be found here.

Other tools

Smart GWT supports a special JavaScript API to allow other test tools to integrate in the same manner as Selenium and WebDriver. This API allows the test tool to record an abstract "locator" string representing the logical name for an interactive DOM element, and then during test playback, retrieve a DOM element given a locator.

This is critical because, like many modern Ajax systems, Smart GWT generates different DOM elements in different browsers, in different skins, and in different versions of Smart GWT. Testing tools that try to directly record the generated Smart GWT DOM produce extremely brittle tests because they are effectively recording undocumented internals.

Using the "locator" API allows you to record or write tests that will run in any browser supported by Smart GWT, in any version of Smart GWT, and in any skin. It also makes tests more readable and easier to understand and maintain.

Different testing tools vary in how easily they can be configured to use the locator API, and in some older tools it can be a large effort. We highly recommend using our Selenium extensions - it often makes sense to use them even if you have to use them in parallel with another, older testing tool. If you are forced to use another tool exclusively:

• Refer to the SmartClient documentation for the AutoTest class (because it's a JavaScript API). It can be found here
• Read over the source code of our Selenium extensions to get a clear understanding of how the Selenium integration works, because this will be analogous to the work you'll need to do
• Search the forums for other developers who are trying to use the same test tool with Smart GWT, and share efforts