Usability Testing Report: Challenges

Users have difficulty figuring out how to purchase a copy of the enterprise software product. Their only option is "Contact Us", but there's no clear match between that and their mental model for buying a product.

Logged Observation(s):
User Carrie, Scenario 01: She's looking for "purchase, buy it", something to let her know she can purchase it online (Video Highlight)

User Dave, Scenario 01: It doesn't say "purchase". He tries "contact us". "That doesn't look like what I want to do." He presses the Back button on his browser. He is looking for a button that says "purchase", "buy", or something like that. He looks at the button for the order form at top. When he clicks on it, he gets the order/feedback form. He can't order it because he don't know how much it costs. He would fill out the feedback form at this point if he was seriously interested. (Order download seems hung up) (Video Highlight)

Recommendation 1, High Priority:
Change the wording to "Contact us for pricing!" so that users know how to find out the charge and how to proceed with the acquisition of the product.

The user wants information about the shipping costs, wants to confirm the tax rate, and wants to be able to get information about the product.

Logged Observation(s):
User Carrie, Scenario 05: "What's the shipping cost? What's the tax rate? Who does it apply to? Where can I get information about it? I want to make sure I'm getting charged the right tax rate." (Video Highlight)

User Dave, Scenario 03: He is unsure whether to pick 'learn more' or to click on the box. He clicks on the box. "What if I'd clicked on learn more?" He clicks purchase. He goes to his cart. He wants 2. He changes it to 2. He reads the charge. ("what's it for?") (Video Highlight)

User Veevi, Scenario Not Set: "It should say 15-dollar SHIPPING charge, not just charge. It should explain how products will be shipped.

Recommendation 1, High Priority:
Add a link to the details about the shipping policy, carriers, and costs to the billing information screen. See the attached visual design pattern for e-commerce applications for an example of how to link to and format this content.

Logged Observation(s):
User Carrie, Scenario 05: She checks the return policy (broken) and refund policy (broken).

User Veevi, Scenario Not Set: She notes the return policy link is broken.

User Veevi, Scenario Not Set: She notes the refund policy link is broken.

Recommendation 1, High Priority:
Add these pages and fix the links.

Logged Observation(s):
User Carrie, Scenario 06: She had no difficulties. Biggest fear was that somebody will get a hold of her credit card. "I don't see a way to verify the card; it doesn't seem secure." (Video Highlight)

User Carrie, Scenario 08: She wants to know it's secure. She doesn't look at the address at the top. She wants to see the billing information, shipping page, and order confirmation separate. She thinks it is checking the billing and address.

Recommendation 1, High Priority:
Include a short explanation of why the cart is secure at the beginning of the cart work flow. See the attachment on “User Trust and E-commerce” for content.

The user wants to see a confirmation of the transaction that includes the items purchased and the credit card information, which can be printed out and kept as a receipt.

Logged Observation(s):
User Carrie, Scenario 09: She expects it to re-post the shipping information if "same as billing" is checked. She is used to seeing it on a separate page.

User Carrie, Scenario 09: She would probably print the page with the reference number from here. She might expect information about the shipment, i.e., when it will be received. "Possibly a recap of what I've ordered, so it's all on one page for a reference and so I could check it when I got my credit card bill." (Video Highlight)

Recommendation 1, High Priority:
The final page in the work flow should be in a printable format and should list:

Items purchased and price
Credit card number charged
Person charged
Shipping address
Approximate arrival date of shipment
Carrier
Tracking number for shipment if possible.

Alternatively, include a “print” button that outputs a formatted page including the above items.

Use a format where most credit card numbers are blocked out with asterisks. See attached visual design pattern for electronic receipt format.

Logged Observation(s): User Carrie, Scenario 09: She can't find the shipping information. "How long will it take me to receive my product? How will it be shipped to me?" (Video Highlight) User Dave, Scenario 05: The second item shouldn't be the same to ship; it should be reduced price. (Video Highlight)	Recommendation 1, High Priority: A clearer statement of shipping fees and carriers is needed. Is a flat fee per item being charged, or a reduced fee for additional items? Who is the carrier? When can people expect to receive their shipments? All this should be clarified.

The logo on the page at end of the transaction is not active. The user expects the logo to return her to the CSS home page. The Back to Shopping button at the end of transaction is confusing.

Logged Observation(s):
User Carrie, Scenario 09: She tries the logo. It doesn't take her back to the home page as she expects. She would close it because she's done with the shopping she wanted to do. She expects "return" to take her back to the products menu. She is surprised that it takes her back to the home page (she tried it with prompting). She expects "return to shopping" and "continue shopping" to be the same. The Back button should be "continue shopping" instead of "back".

User Dave, Scenario 09: He expects the Back to Shopping button to take him back to where he could buy stuff, but it takes him someplace else. He expects to go to the purchase or product list page.

Recommendation 1, High Priority:
The logo should be interactive and should always return users to the home page when clicked.

The Back to Shopping button should return users to the products page, not to the home page.

Shipping information is below the fold on a full screen and may be missed by users.

The error results for the transaction (on Verisign's page) are difficult to read and do not address the specific problem, so users are unsure how to resolve it.

Logged Observation(s):
User Dave, Scenario 06: He enters the wrong date, goes back, and fixes it. Hs still wants to know specifically what was wrong with transaction; he isn't sure. He can only validate the credit card, not the address information, even though it says the address might not match. (Video Highlight)

User Dave, Scenario 07: He noted that the potential errors were hard to read; they were all in one block. He states that the error should give specific the problem and how to fix it, rather than a whole bunch of prose; it was tough to read.

User Dave, Scenario 09: The transaction was declined. It was hard to read. He would prefer bullet points or more specifics. He goes back and realizes the credit card number is wrong. The information on the approval screen seems light. He wants to see what he bought, a product description, some contact information. He would print this screen as a receipt. He wants more contact information (e-mail, phone, address) if there is trouble; he wants some recourse.

Recommendation 1, High Priority:
If possible, verify the specific error and redesign the error page so that it dynamically addresses the specific problem with the user’s transaction. See challenge #11.

At minimum, list the possible sources of error on an error message page in a bulleted format so that it is easier to read.

Logged Observation(s):
User Dave, Scenario 08: There should be a "shop more" button. He just backed up with a browser button. He has three in his cart and changes the quantity back to two.

User Veevi, Scenario Not Set: At the items ordered, it should allow a return to the cart to adjust the quantity.

User Veevi, Scenario Not Set: She notes that an Update Quantities button inside the cart to change quantities may be needed.

Numeric and required fields are not validated in the forms, and errors may result.

Logged Observation(s): User Veevi, Scenario Not Set: The phone number and zip code should be validated as numeric. User Veevi, Scenario Not Set: The missing required fields aren't caught in the billing form.	Recommendation 1, High Priority: Use server-side validation. When errors are detected, provide clear error messages that give the user instructions on how to resolve the problem.

The user is concerned that the shipping information may be lost because clicking the "same as billing address" check box does not result in the billing address being posted to the shipping address form.

Logged Observation(s):
User Carrie, Scenario 04: She is concerned that shipping information will get lost because it doesn't post to the form when checked. She wants to be able to type in the country field because going to u and scrolling for USA is hard. (Video Highlight)

User Carrie, Scenario 04: "Doesn't the info from billing info doesn't get transferred to the shipping screen?" This confuses her. (Video Highlight)

Recommendation 1, Low Priority:
If “saved as billing”is checked, users should be directed to an anchor at the end of the page so they don't need to scroll down through the blank form.

Recommendation 2, High Priority:
Re-post information if possible. If not, bring users past the shipping information form when the "same as billing info" check box is checked.

Logged Observation(s):
User Carrie, Scenario 02: She goes to training first for CBT and looks for it on the training page. She doesn't see computer-based training. She finds the dates. She goes back to products. (Video Highlight)

User Dave, Scenario 03: He goes to training (misunderstood CBT prompt). "But I still might have gone down the training path." He didn't see CBT under training. He then goes to products and finds it. "It seems like I could have found it from the training button." (Video Highlight)

Recommendation 1, High Priority:
Add a link plus marketing copy to the CBT information from the training page.

The View Cart link on the cart doesn't work (because one is already in the cart).

Logged Observation(s): User Dave, Scenario 08: He tries the View Cart link. It doesn't work. User Veevi, Scenario Not Set: "The View cart link doesn't seem to work." It should be grayed out when the user is actually inside the cart.	Recommendation 1, High Priority: Either gray out unavailable buttons or use the View Cart as a title rather than a button. Standard violation.

There is a lack of consistency with the way the required fields are handled in the e-commerce part of the site and on other parts of the site.

Logged Observation(s): User Veevi, Scenario Not Set: She noted "Be consistent on the site. If required fields use a red asterisk, it should be that way consistently (it's black on the order form)."	Recommendation 1, High Priority: Pick one approach as the standard, and audit the site for consistency.

Logged Observation(s): User Dave, Scenario 04: He doesn't use the mouse in the form, except for the state and country selections. He expected to find US at the top of the list. (Video Highlight)	Recommendation 1, Low Priority: Help users navigate the list by mapping alpha entry to the list position.

Logged Observation(s): User Dave, Scenario 04: He noted that shipping is below the fold.	Recommendation 1, High Priority: Redesign so an affordance appears (i.e., at least some of heading appears above the fold) and test it with various screen resolutions, browsers, and widths.

Logged Observation(s): User Carrie, Scenario 04: "Why do I have to enter e-mail twice?" (Video Highlight)	Recommendation 1, Low Priority: Clarify the reason for confirming the e-mail address: is it used it as an account password? Otherwise, delete the redundant entry.