Selenium not a success story

Since the beginning of this project our Test Manager has been developing automated acceptance tests for our User Stories using Selenium. While it is great to have automated acceptance tests, it is not so great when you can't get them to run on your build server. The whole point of automated acceptance tests is to get feedback as soon as possible so you can correct either the tests or the code that broke them.

Our goal is to run the Selenium server as a Windows service. First I read this post http://clearspace.openqa.org/thread/17226 where I got the impression it was not possible to get Selenium to run as a service. Then I read this post http://clearspace.openqa.org/message/39530 telling that it was possible. Anyhow we have not got the Selenium to run successfully as a service, or rather, it does run but not reliable.

What are our options?
We decided to timebox the life of the Selenium server experiments to a maximum of a week. During that time we will set another developer to look at the problem to get a new perspective. There are three options to try here:
1. Get the Selenium server to run as a service. As we already know it might not be possible.
2. Always login to the server and start the Selenium server manually and not log out. The Selenium server works fine when you are interactively logged in.
3. Try to start the Selenium server using a scheduled task. I don't think we have tried this.

Another option than getting the acceptance tests to run on the build server is to run the tests on the developers machine, just like with unit tests.

I will give you our final decision in a future post and would of course appreciate your feedback on how you are working with automated acceptance tests and what tooling you use.