What are the advantages of Hexawise?

  • 1
  • Question
  • Updated 9 years ago
  • Answered
We are using two applied statistcis-based tool right now, one is Hexawise and another one from a different vendor which i won't mention for the sake of this question and making the argument here in this forum.

I wanted to explore and compare the output of these two tools using their pair-wise combinations.

I entered two 2-levels, and one 3-levels parameters. This generated a total of 12 for all possible combinations. Hexawise generated six, while the other generated eight. When I compared the outputs between these two tools, only two combinations equally matched.

I'm wondering what are your thoughts around this if we're saying the tool can provide high coverage with 6 tests (out of 12 possible tests), whereas another tool has 8 tests (out of 12 possible tests) but only two are identical between these two tools.
Photo of Ed

Ed

  • 25 Posts
  • 0 Reply Likes
  • happy

Posted 9 years ago

  • 1
Photo of Justin Hunter, Hexawise Founder

Justin Hunter, Hexawise Founder, Founder and CEO

  • 246 Posts
  • 15 Reply Likes
Ed,

Great question. While I'm pleased Hexawise was able to achieve your coverage objectives in fewer tests than the other tool you compared it to, the most important differences between Hexawise and the other tool you're using (in my opinion), is that (a) Hexawise is much easier to use, (b) Hexawise allows yo to create not only 2-way test plans but also 3-way, 4-way, and even mixed strength solutions to meet the different testing thoroughness objectives you might have, Hexawise was designed to be used by software testers, (c) Hexawise makes it easy to collaborate between different team members with the "notes" feature and ability to grant access to different team members at the project level with different degrees of authority, (d) Hexawise gives you coverage analysis charts that will help you see how significantly marginal returns decrease over the course of your test plan (which can help you make better decisions about when you might want to stop testing), etc.

Now, back to the reason that Hexawise generates 25% fewer tests to meet the coverage objectives in the example. I'm sure both tools achieve 100% coverage of all two way combinations. This means that both plans will have at least one test that contains every valid combination of two specific values. Size = big and color = blue? Check. Covered. Color = green and weight = heavy? Check. Covered. Size = small and weight = heavy. Check. Covered.

Your goal, in software testing, is to achieve as much coverage as possible in as few tests as possible. Hexawise's 2-way solution will help you achieve this as quckly and efficiently as possible. 6 Hexawise-generated tests are required v. 8 for the other tool. What do you get "in return" for the time you spend executing the 2 extra tests from the other tool? Not much. Certainly not 25% more coverage. You get some additional 3-way combinations that Hexawise intentionally leaves out of it's 2-way solutions. For example, color = red and size = medium and weight = light would be a 3-way combination. It is far, far less likely for a defect to be triggered as a result of 3 specific values than it is for a bug to be caused by a combination of 2 values. See www.Hexawise.com/CaseStudies to see empirical evidence of this in the IEEE computer article. I would be happy to have a phone conversation to provide more details and a more complete explanation.

Thanks again for your questions. They're great.