Test Observability for Specflow
Quick start guide to integrate BrowserStack Test Observability with Specflow
Pre-requisites
- You have a C# NUnit, C# xUnit, or C# MSTest test suite.
- Ensure you have .Net v5.0+ and NUnit/xUnit/MSTest v3.9.0+.
- You may run your tests on BrowserStack Automate or even on any other cloud provider or even locally.
- Your tests can be unit/integration/functional or of any nature.
Integrate with Test Observability
You can use BrowserStack Test Observability both when you’re using BrowserStack’s devices and browsers to run your functional end-to-end tests and also if you’re running tests locally on your laptop/CI system or even when you’re using some other cloud provider.
Not only that, Test Observability is agnostic to the type of testing and hence you could also integrate it with your unit or integration test suite written using Specflow.
Please select your setup below to get started with an awesome debugging experience with Test Observability:
To start using BrowserStack Test Observability with your existing setup of Specflow tests running on BrowserStack Automate or App Automate, you’d need to integrate with the BrowserStack SDK. Follow one of the methods below to integrate the SDK and start using Test Observability:
If you’re an existing BrowserStack SDK user, you can skip the steps below. However, ensure that you’ve specified static names (names should not change across build runs) for projectName and buildName in the browserstack.yml file in your project. Also, restrict the characters in your projectName and buildName to alphanumeric characters (A-Z, a-z, 0-9), underscores (_), colons (:), and hyphens (-). Any other character will be replaced with a space.
Add BrowserStack SDK as NuGet package
Steps to install BrowserStack SDK as a NuGet package:
- On the Visual Studio toolbar, select Project > Manage NuGet Packages. A NuGet Package Manager Window opens.
- In the Browse tab, search for BrowserStack.TestAdapter, and click Install.
Create the browserstack.yml file
Ensure you create browserstack.yml file at the root of your directory with the following structure.
The projectName and buildName config must be static and not change across different runs of the same build. This is a deviation in approach as specified by BrowserStack Automate or App Automate as Test Observability will automatically identify different build runs.
Restrict the characters in your projectName and buildName to alphanumeric characters (A-Z, a-z, 0-9), underscores (_), colons (:), and hyphens (-). Any other character will be replaced with a space.
Enable Test Observability
Run the following commands to set Test Observability on in an environment variable. This is a onetime setup that you will need to do before running tests.
setx BROWSERSTACK_TEST_OBSERVABILITY="true"
set BROWSERSTACK_TEST_OBSERVABILITY="true"
dotnet build
export BROWSERSTACK_TEST_OBSERVABILITY="true"
dotnet build
Run your suite with Test Observability
The BrowserStack Test Adapter enables you to execute your test suite seamlessly, just as you did previously.
- On the Visual Studio toolbar, select Test > Test Explorer.
- Right-click on your test and click Run.
Post build run completion, you can see the build run report along with all necessary debugging information on Test Observability.
[Only for Macs with Apple silicon] Install dotnet x64 on MacOS
If you are using a Mac computer with Apple silicon chip (M1 or M2) architecture, use the given command.
-
<path>- Mention the absolute path to the directory where you want to save dotnet x64 -
<version>- Mention the dotnet version which you want to use to run tests
This command performs the following functions:
-
Installs dotnet x64
Installs the required version of dotnet x64 at an appropriate path -
Sets alias
Sets an alias for thedotnetinstallation location on confirmation (enteryoption)
The command sets the installation path in your~/.zshrc(zsh) or~/.bashrc(bash) profile and you can use this alias to run your tests later. If you choose not to set the alias, when you run the tests, you have to use the entire path to the installation location on every test run.
Update your browserstack.yml config file
After installing the SDK, a browserstack.yml config file will be created at the root level of your project. This file holds all the required capabilities to run tests on BrowserStack.
BrowserStack SDK is a very powerful tool that you can use to set the different browser/device combinations and parallelization. For more details, check out the Automate Integration guide.
Make sure you copy the contents of the below config file and set it in your project’s browserstack.yml file:
The projectName and buildName config must be static and not change across different runs of the same build. This is a deviation in approach as specified by BrowserStack Automate or App Automate as Test Observability will automatically identify different build runs.
Restrict the characters in your projectName and buildName to alphanumeric characters (A-Z, a-z, 0-9), underscores (_), colons (:), and hyphens (-). Any other character will be replaced with a space.
Run your test suite with Test Observability
Continue running your tests as you have been running previously. The command to run your tests using the BrowserStack SDK remains unchanged. For your reference, here’s a sample command:
Did not set the alias?
Use the absolute path to the dotnet installation to run your tests on Mac computers with Apple silicon chips:
</absolute/path/to/location/of/dotnet/>/dotnet test
Post build run completion, you’ll be able to see the build run report along with all necessary debugging information right on this dashboard.
Verify you have BrowserStack.TestAdapter installed
As you’re an existing BrowserStack SDK user, you must already be having a browserstack.yml config file at the root level of your project, as well as BrowserStack.TestAdapter as a dependency.
Check if you have the following entry in your .csproj file of your project.
Alternatively, you can run dotnet list package and ensure BrowserStack.TestAdapter is present.
[Only for Macs with Apple silicon] Install dotnet x64 on MacOS
If you are using a Mac computer with Apple silicon chip (M1 or M2) architecture, use the given command.
-
<path>- Mention the absolute path to the directory where you want to save dotnet x64 -
<version>- Mention the dotnet version which you want to use to run tests
This command performs the following functions:
-
Installs dotnet x64
Installs the required version of dotnet x64 at an appropriate path -
Sets alias
Sets an alias for thedotnetinstallation location on confirmation (enteryoption)
The command sets the installation path in your~/.zshrc(zsh) or~/.bashrc(bash) profile and you can use this alias to run your tests later. If you choose not to set the alias, when you run the tests, you have to use the entire path to the installation location on every test run.
Verify your browserstack.yml file
Test Observability mandatorily needs the following four configurations in the file. You need not change anything but you have to ensure that the values of the buildName and projectName variables are not changing across different build runs.
The projectName and buildName config must be static and not change across different runs of the same build. This is a deviation in approach as specified by BrowserStack Automate or App Automate as Test Observability will automatically identify different build runs.
Restrict the characters in your projectName and buildName to alphanumeric characters (A-Z, a-z, 0-9), underscores (_), colons (:), and hyphens (-). Any other character will be replaced with a space.
Enable Test Observability
Run the following commands to set Test Observability on in an environment variable. This is a onetime setup that you will need to do before running tests.
setx BROWSERSTACK_TEST_OBSERVABILITY="true"
set BROWSERSTACK_TEST_OBSERVABILITY="true"
dotnet build
export BROWSERSTACK_TEST_OBSERVABILITY="true"
dotnet build
Run your test suite with Test Observability
Continue running your tests as you have been running previously. The command to run your tests using the BrowserStack SDK remains unchanged. For your reference, here’s a sample command:
Did not set the alias?
Use the absolute path to the dotnet installation to run your tests on Mac computers with Apple silicon chips:
</absolute/path/to/location/of/dotnet/>/dotnet test
Post build run completion, you can see the build run report along with all necessary debugging information on Test Observability.
To start using BrowserStack Test Observability with your existing setup of Specflow tests running on your local laptop or CI or even on any other cloud provider, you’d need to integrate using the BrowserStack SDK. Follow one of these methods to integrate the SDK and start using Test Observability:
BrowserStack Test Observability works with any kind of automation tests. Use it with your unit or integration test suites and also your end-to-end functional test suite.
Add BrowserStack SDK as NuGet package
Steps to install BrowserStack SDK as a NuGet package:
- On the Visual Studio toolbar, select Project > Manage NuGet Packages. A NuGet Package Manager Window opens.
- In the Browse tab, search for BrowserStack.TestAdapter, and click Install.
Create the browserstack.yml file in your project root
Ensure you create browserstack.yml file at the root of your directory with the following structure.
The projectName and buildName config must be static and not change across different runs of the same build. This is a deviation in approach as specified by BrowserStack Automate or App Automate as Test Observability will automatically identify different build runs.
Restrict the characters in your projectName and buildName to alphanumeric characters (A-Z, a-z, 0-9), underscores (_), colons (:), and hyphens (-). Any other character will be replaced with a space.
Enable Test Observability
Run the following commands to set Test Observability on in an environment variable. This is a onetime setup that you will need to do before running tests.
setx BROWSERSTACK_TEST_OBSERVABILITY="true"
set BROWSERSTACK_TEST_OBSERVABILITY="true"
dotnet build
export BROWSERSTACK_TEST_OBSERVABILITY="true"
dotnet build
Run your suite with Test Observability
The BrowserStack Test Adapter enables you to execute your test suite seamlessly, just as you did previously.
- On the Visual Studio toolbar, select Test > Test Explorer.
- Right-click on your test and click Run.
After the build run completes, you can see the build run report along with all necessary debugging information on Test Observability.
Install BrowserStack SDK
Run the following command to install the BrowserStack SDK and create a browserstack.yml file in the root directory of your project.
-
The
browserstack.ymlconfig file holds all the required capabilities to run tests on BrowserStack. -
The
BrowserStack.TestAdapterworks on top of your framework test adapter to run parallel tests on BrowserStack across multiple platforms mentioned on thebrowserstack.ymlfile.
[Only for Macs with Apple silicon] Install dotnet x64 on MacOS
If you are using a Mac computer with Apple silicon chip (M1 or M2) architecture, use the given command.
-
<path>- Mention the absolute path to the directory where you want to save dotnet x64. -
<version>- Mention the dotnet version which you want to use to run tests.
This command performs the following functions:
-
Installs dotnet x64
Installs the required version of dotnet x64 at an appropriate path. -
Sets alias
Sets an alias for thedotnetinstallation location on confirmation (enteryoption).
The command sets the installation path in your~/.zshrc(zsh) or~/.bashrc(bash) profile and you can use this alias to run your tests later. If you choose not to set the alias, when you run the tests, you have to use the entire path to the installation location on every test run.
Update your browserstack.yml config file
After installing the SDK, a browserstack.yml config file is created at the root level of your project. This file holds all the required capabilities to run tests on BrowserStack.
BrowserStack SDK is a very powerful tool that you can use to set the different browser/device combinations and parallelization. For more details, check out the Automate Integration guide.
Copy the contents of the following config file and set it in your project’s browserstack.yml file:
The projectName and buildName config must be static and not change across different runs of the same build. This is a deviation in approach as specified by BrowserStack Automate or App Automate as Test Observability will automatically identify different build runs.
Restrict the characters in your projectName and buildName to alphanumeric characters (A-Z, a-z, 0-9), underscores (_), colons (:), and hyphens (-). Any other character will be replaced with a space.
Run your test suite with Test Observability
Continue running your tests as you have been running previously. The command to run your tests using the BrowserStack SDK remains unchanged. For your reference, here’s a sample command:
Did not set the alias?
Use the absolute path to the dotnet installation to run your tests on Mac computers with Apple silicon chips:
</absolute/path/to/location/of/dotnet/>/dotnet test
Post build run completion, you’ll be able to see the build run report along with all necessary debugging information on Test Observability.
We're sorry to hear that. Please share your feedback so we can do better
Contact our Support team for immediate help while we work on improving our docs.
We're continuously improving our docs. We'd love to know what you liked
We're sorry to hear that. Please share your feedback so we can do better
Contact our Support team for immediate help while we work on improving our docs.
We're continuously improving our docs. We'd love to know what you liked
Thank you for your valuable feedback!