How do I save test documentation time by automatically generating Expected Results in test scripts?
Problem to solve: We'd like to automatically include Expected Results into our test scripts
In the tests shown above for example, we might want to include this Expected Result every time the necessary Values appear together in a test case:
1) When a customer attempts to use frequent flier miles to pay for a flight and that customer has enough miles to do so, confirm that the transaction is successfully processed and that the customer's frequent flier balance is debited.
In the Auto-Script screen, find the specific test step you want to add your Expected Result to and highlight it
1) Navigate to the Auto-Scripts screen.
2) Hover over the Step that you want the Expected Result to appear with.
3) Click on "Add Expected Results."
Ask yourself these 3 questions and formulate precise answers:
1) What Expected Result(s) will be triggered?
- In this case, the Expected Result is "the transaction is successfully processed and the customer's frequent flier balance is debited."
- Note: Some new users of Hexawise accidentally enter Expected Results into the "Inputs" screen; we recommend against doing that. Instead, keep Values (necessary to trigger Expected Results) separate from the Expected Results themselves.
2) What is the smallest number of Values that will result in the Expected Result triggering?
- Exactly two Values are necessary to trigger the Expected Result here. One is insufficient, and we don't need three.
3) Which specific Values will need to appear in the same test case for the Expected Result to occur?
- First, the Parameter "Type of Payment" = "frequent flier points"
- Second, the awkwardly named Parameter "Customer Has Enough Frequent Flier Miles to Pay for Flight" = "has enough frequent flier miles to purchase this flight with them."
Next, with the answers to those 3 questions fresh in your mind, enter ONLY the specific Values that are needed to trigger the Expected Result
You're setting up a simple "when / then" rule here.
Note that you're not restricted to rules like when "Type of Payment" IS frequent flier points:
- In this example, you could also create a rule that reads "IS NOT" "frequent flier points."
- Alternatively, you could create a rule that says when "Type of Payment" is "ANY," then...
Finish creating your simple "When / Then" rule (1) and save it (2).
Rinse and repeat for adding your other Expected Results rules.
That's it! All your tests will now include the Expected Results you defined.
You'll notice that the Values responsible for triggering each Expected Result is bolded.
Important Usage Tips and things to know about the Expected Results feature:
1) This feature is a partial solution for straightforward Expected Results. This feature primarily exists so that you won't have to manually type many, simple expected results. It is not designed to handle especially complex Requirements that you might have.
2) Be sure you understand the similarities and differences between Hexawise's Expected Results in the Auto-scripts screen, and Expected Outcomes feature described here and the "Requirements" feature. There is a big, yet subtle, difference:
- If you want to define an Expected Result that requires 3 or more specific Values to appear in a single test script (and you're creating pairwise sets of tests), use the "Requirements" feature instead of the Expected Results feature (because you won't know whether those more-complex Expected Results would appear in your optimized tests, but you need them to appear).
- Using the Expected Results feature is like telling the test generation engine "if you coincidentally see a test that should include this Expected Result, make sure it appears."
- Using the Requirements feature, in contrast, is like telling the test generation engine "make sure that the specific combination of Values I'm specifying here definitely appears together in the set of tests at least one time (and, if I tell you that there's an Expected Result associated with that scenario, please be sure to include it)."
- Accordingly, if you're generating 2-way tests with Hexawise, you should primarily use the Expected Results feature to define simple Expected Results rules that are decided by using only one or two Values.