Disclaimer: “Visual Studio test functionality varies between 2010, 2013, 2015, and now 2017 versions. Also different functionalities are available in Enterprise, Pro and Community editions. But you can build custom test solutions or use on of many free, open source test suites. “
Test Driven Development (TDD), black box test, white box test, UI test, Unit Test, grey box tests, etc. These are some of the elements that you may encounter while looking for your best test strategy. Some say that if you stay in line with all good coding practices you don’t have to conduct any tests, or that testing can be limited to basic elements. I personally doubt it, because the bottleneck in programming is… the programmer.
Whatever strategy you choose keep in mind: you should never, ever let customer test your software first.
Of course there are UATs (User Acceptance Tests) that your customer should perform with your assistance, but they come later, when your application is ready.
Before your software reaches that stage, it’s you who must test it as thoroughly as possible. There are different strategies that may apply. I will not concentrate too much on how to choose an appropriate strategy, but rather help a little with Visual Studio based testing.
You may have heard of some tools and software suites that help to test software. Visual studio in the Ultimate version offers a lot of diagnostic tools that improve testing and debugging. Unfortunately most of these tools are very expensive and complex. In this article I will show you how to test software using basic Visual studio versions. I used Visual Studio 2013 Professional to be exact.
Please bear in mind that Microsoft changed some mechanisms of access to private methods in Visual Studio 2013 as compared to the 2010 version. In VS2010 you could test private methods by using Private Accessors, in the 2013 version they do not exist – you can use Private Objects and Types. It does not matter, it’s only some technical detail. My examples are based on 2013 therefore I use Private Objects and Types.
Let’s write a class that performs an operation on a String parameter. It has some static functions that add suffix to input text and return this concatenated text. Let’s say we have four methods that add numbers from 1 to 4, and let’s assume that thirdOperation method has an error/mistype and returns suffix ‘4’ instead of ‘3’.
You may of course test these methods by running your application when you think it’s ready to be sent to the customer or create a special application just for testing.
Both the ready application and the test one will run and show some results.
You may read and check results. Here we can see that there is something wrong because third line should read ‘test3’. OK, but what if we have seventy-seven classes and seven hundred methods ? We can’t do it manually any more. There is a number of problems that we face when we have bigger applications:
- there are too many methods to test them manually by looking at the result with your own eyes
- software changes continuously, so maintenance of hundreds of methods and their individual tests may kill your budget and make you miss your deadlines
- unit tests are not enough, because typically UAT involves scenarios not individual tests.
- write as many tests as your budget and time allows, at least test main positive scenarios and unit test public methods (including static of course)
- automatize as many tests as you can
You can’t test everything, because the number of possible negative scenarios is INFINITE
Visual Studio offers some basic tools for testing which involve TestClass and TestMethod. These are attributes that you can use to mark class and method as part of test. After building a solution, VS detects these attributes and shows them in Test Lists (or Playlists in different VS versions).
In VS2010 you could automatically generate unit tests for each method, in VS2013 however, Microsoft removed this possibility. It still is very easy to add test classes, I will not discuss the technical details here though. Let’s create another project ‘TestMyApp’ and add UnitTest1 class which will test our StringOperations class.
Let’s look at our first test method ‘TestMethod1’. It tests FirstOperation static method and checks whether expected output meets the result of FirstOperation. In cases where you have hundreds of methods it saves you hours of testing, and saves you the stress you would experience if the customer called with an information that your software restarts every 5 minutes or it is impossible to run it at all because of some small change that was done at 3am the day before. Software production is very difficult, sometimes one space or dot typed in a wrong place renders an entire system used by your customer useless.
We can test the methods SecondOperation and ThirdOperation in the same way. VS2013 has a window called TestExplorer which you can open to see all tests and arrange them in so called playlists. In VS2010 they are called differently but they do the same thing.
As expected TestMethod3 failed because ThirdOperation has a bug. It adds ‘4’ to input string instead of ‘3’.
Let’s change visibility of FourthOperation to private and add a test method for it.
This approach has two problems – neither is FourthOperation strongly typed, nor does Intellisense work in it. (Microsoft says that it will be improved in VS2015…, they may reintroduce private accessors in VS2015). If you change the name of FourthOperation using Refactor function, be sure to look for all its instances in the code, text and comments.
Testing private methods is ok, if they are static or as simple as those used in our examples, if they don ‘t require creating an object and running several other methods first. In more complicated scenarios it is better to only test public instance methods and public static methods.
Organize tests in groups: ordered tests or scenarios
By scenarios I mean a series of method calls that are typically used in UAT tests. For example user enters a new customer, then enters address and phone number and saves it. Or credit card is tested for its validity and then a product is invoiced. You may do it using OrderedTest project item (VS2013) or simply by writing a new test class with a test method that involves using many methods.
After adding OrderedTest item and building your solution you can see your ordered test and add it to an existing playlist or create new one. Remember you can always add ordered tests inside other ordered tests which makes it possible to create complex scenarios.
If your scenario is not that atomic (running one test after another without any references) and result of one method is used by another method, you may write your scenario as a TestMethod like this.
Our scenario fails, of course, because there is a bug in ThirdOperation.
Use Initialization and Cleanup methods to set up environment for scenarios
In Visual Studio 2010 you could use Initialization and Cleanup methods that were automatically added for you to use with every test method and class. They can be also used in VS2013, but they have to be written manually. Initialization and Cleanup methods can be utilized to set up an environment for your tests, e.g.: set up a new database copy to use, clear a database, clear/copy some test files or do whatever you need to do before the test begins and after it has ended.
That’s it for now. In a future article I will discuss automatization of UI tests.
Thanks for reading and regards
CEO, .Net developer, software architect
If you need help with your software project, or need customized software for your company, contact me at: dominik.steinhauf ( at) cys.biz.pl