Koen about .Net

February 16, 2011

Custom C# formatter for Selenium

Filed under: Development, dotnetmag, Testing — Tags: , , — koenwillemse @ 22:20

In my previous post I mentioned that I had been working on a C# formatter for selenium which uses MSTest as test framework and also FluentAssertions to make the tests more readable. After a few hours last Saturday night I had something working which looks pretty ok for now. I guess I’m not the only one wanting to generate this code, so I wanted to share what I’ve got.

I started off by taking the source of the C# formatter which is installed together with the selenium IDE. Note that there is a bug (maybe more, but I ran into this one Winking smile) in the source, which results in a lack of options to set in the IDE. The problem is regarding the configForm setting, which is done in the options. This is pretty strange if you look at it, because that configForm is used to generate the UI for the options (which options are available can be found here). The configForm should be set outside the options. I tried to modify the source, but that’s not possible, it’s read-only. If you still want to use the NUnit formatting as done in the formatter, you can add another formatter and add the modified source. You can download the modified source here.

Now to the formatter I wrote. I wanted a few things:

  • MSTest as test framework
  • use FluentAssertions for readability
  • Start the selenium server when starting test run (and close it when tests finished)

There is a description here on how to start creating a custom formatter. However, I was lazy, so I copied the source from the existing C# formatter and modified it to accommodate my needs. First of all I fixed the problem with the options, which I described above.
Next I changed the assertion methods to use the FluentAssertions syntax.
I updated the generated header and footer, since I don’t want all the initialization of a Selenium instance etc. to be done on every TestInitialize since it takes several seconds. I changed it to be done on a central location.

The biggest change I made was add formatting for a test suite, which is not available in the default C# formatter. In the code for the test suite, I added methods for AssemblyInitialize and AssemblyCleanup. In these method, I will start and tear down the selenium server process and the selenium instance. This uses a static selenium instance, so this will cause problems if you want to use it in a multi-threaded environment, but for now, it matches my needs. The complete source of the formatter can be downloaded here.

I hope some of you find this helpful also. If you have some good suggestions for the formatter, then please let me know.

Edit 22-02-2011: The links to the sources were incorrect. I updated the links now, so you can download the files from my skydrive.

February 15, 2011

Comparison of Web UI Testing toolkits

Filed under: Development, dotnetmag, Testing — Tags: , , — koenwillemse @ 16:00

I wanted to started using web UI tests because the current project I’m working on I lacking this and for my personal web shop application I also need it. So I had to make a choice for a testing framework. I already knew of Coded UI Tests in Visual Studio 2010 and Selenium. Another colleague at my current project also mentioned Watin, so I decided to do a quick test of the three and see if it matches my expectations.

Coded UI Tests in Visual Studio

This was my first choice, since it’s integrated in Visual Studio which makes it easier when testing my deployed applications from a build server. So I started by creating the test in de Visual Studio IDE and I clicked some pages. Then added a few assertions and done. Generated the coded test and I looked at the coded that was generated. And that what a bit of a shock Sad smile. It was very unreadable code, which actually shouldn’t matter since it’s generated code, but when you would like to tweak or edit it a bit (like I want) it’s not very nice to do that. I wanted to edit the tests to be able to use some parameters and stuff to make the tests more robust.
Now there is also the option of created tests using the Test Manager in Visual Studio 2010, but I haven’t tried this. That’s something I still want to do, to see if this makes it easier or better. For now, I’m a bit disappointed in using the coded UI tests. When I’ve got some time left, I’m going to check how the creation of coded UI tests is when using the Visual Studio 2010 Test Manager

Selenium

Selenium is a tool which I’ve heard of several times, so I wanted to give it a try now myself. First you’ll have to install some stuff. I installed the following:

After installing all stuff (which is more work than I wanted Winking smile) I started creating my first test. Clicking the test and verifications is also pretty easy with the integration in Firefox. Then creating the code. There are a few formatters available, one of them being for c#, so I generated the code, but unfortunately it was based on NUnit and there was some stuff in it I didn’t like. But before I gave up, I looked a bit better in Selenium IDE and I saw that there is also the possibility to create your own formatter, so I decided to give that a try. It took some time with some mistakes that I made, but eventually I had a working formatter which created the C# code the way I wanted it. So I started the tests in visual studio (after starting the java application for the selenium server) and it worked Smile.

Watin

Because my colleague mentioned this framework, I wanted to give it a try. Unfortunately there was no IDE or something to create the tests. Since I was already more convinced of both other frameworks, I didn’t want to spend more time on this one, since it was also more difficult to work with it in code only mode.

Comparison

So a small comparison

  Coded UI Tests Selenium Watin
Positive
  • Integrated in Visual Studio, so no installation required
  • Control over the generated code
  • Code easy to modify / extend
 
Negative
  • Ugly generated code
  • IDE is a bit buggy
  • Custom installation required
  • No IDE
  • Difficult to work with

So eventually I chose selenium for now, since some other people on my current project are also trying it out and were positive about it. I’ll write another blog post on the custom formatter I’ve been working on. It generates code for the MSTest framework and makes use of the FluentAssertions to make the tests more readable.

Blog at WordPress.com.

%d bloggers like this: