Acceptance Tests: Prove Your Worth

Our User Story states “As a user of the blogging web service, I want to be able to save my blog details to a database” and we have a REST service contract which shows us what a request to create a new blog should look like.

We now have enough information to create our Acceptance Tests.

A single developer writing tests and code is prone to creating false positives or missing out tests altogether

Even though we have yet to write a line of service code, we can implement the acceptance tests as we have enough information to determine how the service should behave once it is completed. (This is the middle ‘D’ in TDD!). As an Agile team, it is always a good idea to have one developer (or pair) write the tests and another one the actual service code. This avoids the creation over-optimistic tests which are written with the knowledge of how the implementation works. A single developer writing tests and code is prone to creating false positives or missing out tests altogether.

The code below shows a very simple acceptance test for a CreateBlog method:

 [Test] [TestCase("Little Shop of Coding Horrors", HttpStatusCode.Created, TestName = "CanSaveBlogDetails201")] [TestCase("", HttpStatusCode.Conflict, TestName = "CannotSaveBlogDetailsEmptyTitle409")] public void CanSaveBlogDetails(string title, HttpStatusCode expectedStatusCode) { //// Arrange var userId = Guid.NewGuid(); var requestDto = new CreateBlogRequestDto { Title = title }; //// Act RestServiceResponse response = Client.CreateBlog(userId, requestDto); //// Assert Assert.IsNotNull(response, "Response"); Assert.AreEqual(expectedStatusCode, response.StatusCode, "Status code"); } 

Some things to notice here are:

1) The test is completely independent of the service. In fact the test project in Visual Studio doesn’t need a reference to the service. The only “coupling” between test and service here is in the http calls made from the client.

2) The test is adhering to the classic AAA test pattern. This helps provide familiarity to the overall structure of a test, and importantly reminds developers to think about each discrete part in it’s own right. (I have seen many tests where dev’s have actually forgotten the Assert’s at the end!). The AAA pattern doesn’t preclude having assertions prior to the end block and if your code become complex I would recommend earlier assertions, but this pattern does create a nice template to work with.

3) We are using NUnit here which provides the TestCase attribute. This little nugget means with a bit of planning we can define simple tests which cover a much wider range of code paths all with the same piece of code. Here for example we can pass in the expected response for both successful and failing calls to the service. (Admittedly you might want to keep your “Happy” path tests separate from your “Sad” path tests but you get the idea…)

4) There is no “Client” setup code in the test. The actual test fixture is derrived from a base class which has all the grunt work burried in it. This helps keep the tests clean from a dev perspective allowing them to focus on importance details of the test itself.

There is one final point to note about this test, and this is IMPORTANT! We are not testing that the service has actually “created the blog”!! Whoooa! Isn’t that the whole point? Yes, but acceptance tests are black box testing and no-where in the service contract do we have a “Read Blog Details” method. We simply have a success or fail code returned from the CreateBlog call.

We are not testing that the service has actually created the blog!

The argument for a development team to create a mechanism outside of the system under test to verify the blog is created is often a compelling one but I would strongly argue against it. Sure, we could write some code to call a store proc which reads a line in a database to verify the blog entry is there but we’d then need Acceptance Tests for our Acceptance Tests…

In our case, it is pretty likely that a future User Story will be along the lines of “Read Blog Details from service”. The smart thing to do here is add a “Tech Debt” story to the product backlog to enhance the assertions in our test so that we can call the ReadBlog method on the service once it is complete such as below:

 //// Act RestServiceResponse writeResponse = Client.CreateBlog(userId, writeRequestDto); RestServiceResponse readResponse = Client.ReadBlog(userId, readRequestDto); //// Assert Assert.IsNotNull(writeResponse, "Response"); Assert.AreEqual(expectedStatusCode, writeResponse.StatusCode, "Status code"); readResponse.Result.AssertEquals(expectedReadResponse.Result); 

It may be that in order to sign a story off you may need to manually show the client or product owner that the call has worked until a later story can automate a test for it. Even then, if there is no such “get out clause” (for example firing audit logs asynchronously to a reporting service) Acceptance Tests should test the black box, AND ONLY the black box.

Ultimately, Acceptance Tests need to be able to automatically verify that the system is providing what the business wants. Over time, as complexity is added your Acceptance Tests will likely become vary large in number and will start to take longer and longer to complete. In a continuous integration environment this may become prohibitively long and it may be prudent to start to run a selective section of the test suite which covers say 80% of the functionality. However, I would still recommend running the entire suite on a daily basis to make sure you get an early warning should anything in the remaining 20% go wrong. For interest, the chart below shows the number of lines of code for various sections of a REST service I was once working on. There were several thousand Accecptance Tests which took around 45 mins to complete. Remember, good test coverage let’s you sleep at night.


code breakdown

One final tip: Use the xml comment to provide details of the user story to which the test pertains. Great for searching though code afterwards…