I used to test an application that ran on four different client platforms, against four different server platforms, in two different types of environments (new installs and upgrades/migrations). Unfortunately I did not have the luxury of months available to me to test all 32 possible combinations, when a regression test of just one client/server/environment combination took at least three days. Usually I would, in cooperation with the project manager, determine what combinations were most prevalent in the field and therefore had the highest priority. I’d start with those, and do what I could.
Sometimes, though, all combinations are important. What do you do then? What if you have to test four clients against four servers in TEN different environments? James Bach has a small, free tool that can be used to determine what combinations to test, called Allpairs. From his description:
But you can’t try all combinations of all variables together, because that would be too many tests. So instead, you could create tests that pair each value of each of the variables with each value of each other variable at least once. Allpairs is a tool I wrote that will find a reasonably small set of test cases to satisfy that coverage standard.
Allpairs runs on the command line and outputs results into a text file that can be pasted into Excel for easy viewing and analysis. You can use Allpairs to condense your list of test cases, leaving you more time for fun things! Like writing bugs and begging for more test hardware.