Run Katalon Studio tests with Automate

A step-by-step guide to running Katalon Studio tests with BrowserStack Automate

BrowserStack's AI-powered low-code testing

Checkout our AI-enabled and intuitive low-code recorder for scalable, reliable test execution

Try Low Code Automation

Katalon Studio helps you to automate your web and mobile testing. It is designed to create automated test scripts without coding. Integrating Katalon Studio with BrowserStack will help you to run Katalon Studio recorded test scripts on 3000+ real devices and desktop browsers.

This guide will help you:

• Integrate BrowserStack Automate with Katalon Studio
  • Introduction
  • Running Katalon tests on BrowserStack
  • Testing privately hosted websites
    • Establishing a Local Testing connection
    • Configuring tests to run with Local Testing connection
  • Running multiple tests in parallel
  • Organizing tests
  • Marking test status as passed or failed
  • Supported OS/Browsers

Prerequisites

• BrowserStack Username and Access key, which you can find in your account settings. If you have not created an account yet, you can sign up for a Free Trial or purchase a plan.
• Katalon Studio is installed.
• A recorded test using Katalon Studio’s recorder is available.

Running Katalon tests on BrowserStack

Katalon Studio recorded test scripts are executed on BrowserStack using the Remote option of Katalon Studio. To execute the test script remotely there is a prerequisite for specifying the capabilities provided by BrowserStack.

Follow the below steps to set the capabilities and execute Katalon test script on BrowserStack:

1. Open the Katalon Studio test script that you want to execute on BrowserStack.

2. Now, click on Project tab → Settings option:
   
   
   
   

   The Project Settings window will appear. To set the capabilities, click on the Desired Capabilities → Remote option, this will result in the following window.
   

   • In the Remote server URL field, add
     

      https://YOUR_USERNAME:YOUR_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub
     

     USERNAME and ACCESS_KEY are used for authentication and can be found on the BrowserStack Settings page under the Automate section.

   • In the Remote server type field, select Selenium to execute the test script on desktop browsers or Appium to execute the test script on mobile devices/browsers from the drop-down list.
   • In the case of Appium, the Appium driver must be specified based on Android (Android Driver) or iOS devices (iOS Driver):
     
     

3. Click on Add icon to add desired capabilities provided by BrowserStack:
   
   
   
   For example, to execute the test script on Chrome 78 on Windows 10 use the following capabilities in the form of Name and Value:
   
   

   • os: Windows (Name: os, Value: Windows)
   • os_version: 10 (Name: os_version, Value: 10)
   • browser: Chrome (Name: browser, Value: Chrome)
   • browser_version: 78 (Name: browser_version, Value: 78)

   You can refer to the Capabilities Generator section of the capabilities page to find out other possible Name and Value.
   
   

   Capabilities are set at the Project level, that is, all the test cases, test suites, and test suite collection within a single project will have the same set of capabilities.
   
   

   In order to execute the test cases, test suites, or test suite collection with a different set of capabilities, create another project with the new set of capabilities. Once all the capabilities are added, click on Apply → OK as shown in the following figure:
   
   

4. Now, execute the test script using the Remote option as shown in the following figure:
   
   

   If the test is executed successfully on BrowserStack, the Log Viewer tab in Katalon Studio displays the test script result as shown in the following figure:
   
   

   
   

   You can also verify the test script status on the BrowserStack Automate dashboard.

Testing privately hosted websites

You can test your websites that are hosted privately using the BrowserStack Local testing feature. This section will guide you through enabling Local Testing, and then using it to test websites on your work machines, CI/CD machines/nodes, and other private network configurations.

Enabling Local Testing with Automate is a simple two-step process:

1. Establishing a Local Testing connection.
2. Configuring Katalon test scripts so they run through the Local Testing connection.

Establishing a Local Testing connection

You can start a Local Testing connection through your command-line interface by following the steps below:

1. Download the appropriate binary for your system:
   • OS X (10.7 and above)
   • Linux 32-bit
   • Linux 64-bit
   • Windows (XP and above)
2. Unzip the binary to a folder/directory on your machine.
3. Open your command-line interface and navigate to the folder containing the Local binary.
4. Run the binary using the following command:

./BrowserStackLocal --key YOUR_ACCESS_KEY

BrowserStackLocal.exe --key YOUR_ACCESS_KEY

Configuring tests to run with Local Testing connection

After establishing the Local Testing connection, set browserstack.local capability to true.

Now, run the test script which is trying to access locally hosted URLs using the Remote option as shown in the above section (point number 4). If the test runs successfully on BrowserStack, the Log Viewer tab in Katalon Studio displays the test results as shown in the following figure.

You can also verify the test script status on the BrowserStack Automate dashboard.

Running multiple tests in parallel

On BrowserStack, you can run multiple test scripts at the same time across various browser, device and OS combinations, this is Parallel Testing. Parallel Testing gives you the same benefits as running a multi-threaded application.

With Parallel Testing, you can run the same test on different browser/device combinations, or run different tests on the same browser/device combination. Parallel Testing will help you reduce the run time of your test suites, resulting in faster build times and faster releases.

Parallel testing in Katalon Studio is done with the help of Test Suite Collection, which includes Test Suites, and in return, these Test Suites include the Test Cases.

Following are the steps to execute test scripts parallelly on BrowserStack:

1. Consider an example of executing two test cases/test scripts (TC1 and TC2) in parallel.
   
   

2. Create two Test suites (TS1 and TS2), add these test cases in the respective Test Suites. TS1:
   
   TS2:
   
   

3. Create a Test Suite Collection (TSC), and add both the Test Suites.
   
   When we add a test suite to a test suite collection, by default, the execution will be running on the Chrome browser as shown in the preceding figure.

4. To execute test scripts parallelly on BrowserStack, change the browser option to Remote by clicking on Run with column. It will display the execution environment selection dialog box as shown in the following figure:
   
   

5. Select the Remote option to execute the test scripts parallelly on BrowserStack.
6. Select the radio button Parallel in the Execution Information section.
7. Click on Execute to execute the test scripts parallelly on BrowserStack using the desired capabilities.

Organizing tests

You can execute your Katalon test scripts on BrowserStack in separate builds (test suite) with the help of listener function.

Test Listener file in Katalon allows you to add listener functions before/after TestCase/TestSuite. Listener functions will enable you to run your test scripts on BrowserStack with the Katalon test script name. This will lead to the execution of test scripts within the respective builds.

Following are the steps to implement Test Listeners in your Katalon test scripts:

1. Create a Test Listener file by clicking on Test Listeners > New > New Test Listener in the Tests Explorer.
2. In the following New Test Listener window, select all checkbox for Before/After Test Case/Test Suite:
   
   
3. In the Test Listener file, you will find Before/After Test Case/Test Suite listeners as shown below:

   @BeforeTestCase
   def sampleBeforeTestCase(TestCaseContext testCaseContext) {
    println testCaseContext.getTestCaseId()
    println testCaseContext.getTestCaseVariables()
   }
   @AfterTestCase
   def sampleAfterTestCase(TestCaseContext testCaseContext) {
        println testCaseContext.getTestCaseId()
        println testCaseContext.getTestCaseStatus()
   }
   @BeforeTestSuite
   def sampleBeforeTestSuite(TestSuiteContext testSuiteContext) {
    println testSuiteContext.getTestSuiteId()
   }
   @AfterTestSuite
   def sampleAfterTestSuite(TestSuiteContext testSuiteContext) {
    println testSuiteContext.getTestSuiteId()
   }
   

   Note: Ensure the WebUI.closeBrowser() code line is not present in the testscripts as it will be added in the @AfterTestCase listener function file.

4. To run your test scripts and test suite on BrowserStack with the Katalon test script and test suite name follow the below steps:
   
   • Add the below package in the Test Listener file:

      import com.kms.katalon.core.configuration.RunConfiguration
     

   • Add the below snippet in the @BeforeTestSuite listener function to run your test suite(build) on BrowserStack with the Katalon test suite name:

      @BeforeTestSuite
      def sampleBeforeTestSuite(TestSuiteContext testSuiteContext) {
           RunConfiguration.setDriverPreferencesProperty("Remote","build",testSuiteContext.getTestSuiteId())
      }
     

   • Add the below snippet in the @BeforeTestCase listener function to run your test scripts on BrowserStack with the Katalon test script name:

      @BeforeTestCase
      def sampleBeforeTestCase(TestCaseContext testCaseContext) {
           RunConfiguration.setDriverPreferencesProperty("Remote", "name",testCaseContext.getTestCaseId())
      }
     

5. Now run your test suite, you will find that the test scripts execute within the respective test suite(build) on the BrowserStack Automate dashboard.

Marking test status as passed or failed

BrowserStack does not know whether your test’s assertions have passed or failed because only the test script knows whether the assertions have passed. You can inform BrowserStack whether your test script has passed or failed by setting the test status to PASSED or FAILED.

Follow the below steps to set test status to PASSED or FAILED:

1. Create @AfterTestCase listener function as described in the Organizing tests section.
2. Add the below packages to Test Listener file:

    import org.openqa.selenium.JavascriptExecutor;
    import com.kms.katalon.core.webui.driver.DriverFactory;
    import org.openqa.selenium.WebDriver;
    import org.openqa.selenium.support.ui.WebDriverWait;
   

3. Add following code snippet in the @AfterTestCase listener function to set test status to PASSED or FAILED as shown in the following code snippet:

    @AfterTestCase
      def sampleAfterTestCase(TestCaseContext testCaseContext) {
     		String status = testCaseContext.getTestCaseStatus();
           String reason = testCaseContext.getTestCaseStatus();
     		def driver = DriverFactory.getWebDriver()
     		JavascriptExecutor jse = (JavascriptExecutor)driver;
     		jse.executeScript("browserstack_executor: {\"action\": \"setSessionStatus\", \"arguments\": {\"status\": \""+status+"\", \"reason\": \""+reason+"\"}}");
     		WebUI.closeBrowser()
      }
   

   In the above code snippet, you can pass the reason for test status in string datatype, the test status is obtained from the testCaseContext.getTestCaseStatus() method.

4. Execute the Test Case. You will find the Katalon test executed on BrowserStack with test status marked as PASSED or FAILED.

Supported OS/Browsers

The following table shows the supported browser versions on Windows and Mac:

The Katalon Studio test scripts are executed on BrowserStack with most of the browser and OS combinations, however, there are few exceptions at present: OS X Mavericks and OS X Catalina.

You should now be able to execute Katalon Studio tests on BrowserStack, if you need any further help, feel free to contact our Support team who are always ready to help you out!