Koen about .Net

February 15, 2011

Comparison of Web UI Testing toolkits

Filed under: Development, dotnetmag, Testing — Tags: , , — koenwillemse @ 16:00

I wanted to started using web UI tests because the current project I’m working on I lacking this and for my personal web shop application I also need it. So I had to make a choice for a testing framework. I already knew of Coded UI Tests in Visual Studio 2010 and Selenium. Another colleague at my current project also mentioned Watin, so I decided to do a quick test of the three and see if it matches my expectations.

Coded UI Tests in Visual Studio

This was my first choice, since it’s integrated in Visual Studio which makes it easier when testing my deployed applications from a build server. So I started by creating the test in de Visual Studio IDE and I clicked some pages. Then added a few assertions and done. Generated the coded test and I looked at the coded that was generated. And that what a bit of a shock Sad smile. It was very unreadable code, which actually shouldn’t matter since it’s generated code, but when you would like to tweak or edit it a bit (like I want) it’s not very nice to do that. I wanted to edit the tests to be able to use some parameters and stuff to make the tests more robust.
Now there is also the option of created tests using the Test Manager in Visual Studio 2010, but I haven’t tried this. That’s something I still want to do, to see if this makes it easier or better. For now, I’m a bit disappointed in using the coded UI tests. When I’ve got some time left, I’m going to check how the creation of coded UI tests is when using the Visual Studio 2010 Test Manager


Selenium is a tool which I’ve heard of several times, so I wanted to give it a try now myself. First you’ll have to install some stuff. I installed the following:

After installing all stuff (which is more work than I wanted Winking smile) I started creating my first test. Clicking the test and verifications is also pretty easy with the integration in Firefox. Then creating the code. There are a few formatters available, one of them being for c#, so I generated the code, but unfortunately it was based on NUnit and there was some stuff in it I didn’t like. But before I gave up, I looked a bit better in Selenium IDE and I saw that there is also the possibility to create your own formatter, so I decided to give that a try. It took some time with some mistakes that I made, but eventually I had a working formatter which created the C# code the way I wanted it. So I started the tests in visual studio (after starting the java application for the selenium server) and it worked Smile.


Because my colleague mentioned this framework, I wanted to give it a try. Unfortunately there was no IDE or something to create the tests. Since I was already more convinced of both other frameworks, I didn’t want to spend more time on this one, since it was also more difficult to work with it in code only mode.


So a small comparison

  Coded UI Tests Selenium Watin
  • Integrated in Visual Studio, so no installation required
  • Control over the generated code
  • Code easy to modify / extend
  • Ugly generated code
  • IDE is a bit buggy
  • Custom installation required
  • No IDE
  • Difficult to work with

So eventually I chose selenium for now, since some other people on my current project are also trying it out and were positive about it. I’ll write another blog post on the custom formatter I’ve been working on. It generates code for the MSTest framework and makes use of the FluentAssertions to make the tests more readable.



  1. […] This post was mentioned on Twitter by Dennis Doomen, Koen Willemse. Koen Willemse said: Comparison of Web UI Testing toolkits http://wp.me/plwkW-3X […]

    Pingback by Tweets that mention Comparison of Web UI Testing toolkits « Koen about .Net -- Topsy.com — February 15, 2011 @ 17:04

  2. […] my previous post I mentioned that I had been working on a C# formatter for selenium which uses MSTest as test […]

    Pingback by Custom C# formatter for Selenium « Koen about .Net — February 17, 2011 @ 07:47

  3. CUITe (Coded UI Test enhanced) Framework (http://cuite.codeplex.com/) is something that you can have look at.

    It allows you to define a much simpler Object Repository (== UIMap) like this:

    public class SATHome : Common
    public CUITe_HtmlHyperlink lnkTermsOfUse = new CUITe_HtmlHyperlink(“InnerText=Terms of Use”);
    public CUITe_HtmlInputButton btnNext = new CUITe_HtmlInputButton(“Id=ctl00_btnNext”);

    The test code is readable and maintainable. Handling a Search Results table is more logical as you can see below.

    pgSATHome.lnkTermsOfUse .Click();
    TermsOfUse pgTermsOfUse = CUITe_BrowserWindow.GetBrowserWindow();
    pgTermsOfUse.txtApplicationName.SetText(“Test App by Automation”);

    CUITe also have a number of features like ability to inject JavaScript code (even large js functions) dynamically into DOM and eval it.

    Comment by Suresh Balasubramanian — March 15, 2011 @ 10:43

  4. Now we just need to NuGet WatiN package for Visual Studio to use it. There is no need for IDE and it is not that difficult to work with.

    Comment by Blaise Liu (@blaise_liu) — February 26, 2014 @ 16:24

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

Blog at WordPress.com.

%d bloggers like this: