Heuristic Evaluation of Two Travel Websites
I conducted a heuristic evaluation of two popular travel planning websites: Travelocity and Expedia.
A heuristic evaluation is an inspection method in which you systematically apply heuristics (qualitative guidelines) to find and explain problems.
I applied Jakob Nielsen’s heuristics:
- Visibility of system status
- Match between system and the real world
- User control and freedom
- Consistency and standards
- Error prevention
- Recognition rather than recall
- Flexibility and efficiency of use
- Aesthetic and minimalist design
- Help users recognize, diagnose, and recover from errors
- Help and documentation
My goal was to apply Nielsen’s heuristics to critique the user interface designs of both websites by performing detailed heuristic evaluations.
I’d like to share these evaluations as a way to illustrate how to apply this UX method.
Design Brief
The task was to compare the process of buying a round-trip ticket on two different websites. I could pick any mode of transit I liked. And I had to try purchasing a round-trip ticket from my home city, the following Saturday*, to any destination that I liked.
I had to start on the homepage and work my way through the task, while keeping a list of the heuristics handy. As I discovered usability problems with the sites, I made sure to write them down and take a screenshot.
Once I had finalized the list of violations, I gave a severity rating to each. I used this severity rating scale:
0 — don’t agree that this is a usability problem
1 — cosmetic problem
2 — minor usability problem
3 — major usability problem; important to fix
4 — usability catastrophe; imperative to fix
I was to summarize the task I was trying to accomplish, show a screenshot where the violation occurred, list the heuristic violated, and describe the violation and its severity.
The Execution — Expedia
Task: Purchasing a round-trip flight from San Francisco, California (USA) to Istanbul, Turkey departing on Saturday 07/11/15 and returning on 08/11/15.
First Issue/Violation: It was difficult to find out information about “bag fees.”
Severity: 3
Heuristics violated: Recognition and Help
Description: When I click on “More info” under “Bag fees” another tab opens and shows me this information:
Here it would have been helpful to see the following info highlighted (instead of being part of the paragraph):
If your trip includes more than one airline, baggage fees are generally determined by the first carrier on the ticket.
I’d also have wanted to have some other link where I can go check for more info, since they say “generally” and I wanted to make sure how much I would end up paying in fees. In order to find that information, I clicked on “more info” inside “bag fees” and a window opened saying: “If your trip includes more than one airline, baggage fees are generally determined by the first carrier on the ticket.” It should not take this long to get this vitally important information regarding fees.
Second Issue/Violation: Insufficient information regarding their flight rating system.
Severity: 3
Heuristics violated: Help
Description: I noticed that my flight was rated as “Satisfactory Flight (6.7 out of 10).” I clicked to find out what they meant by that. There was some information offered, but it was too generic and not enough. I was left wondering which of the factors mentioned in the pop-up description lowered this flight’s score. This could be fixed by adding a “more info” link at the end of the description for users who want to find out more.
Third Issue/Violation: Suggestions and info needed to help user choose which seat they prefer.
Severity: 2
Heuristics violated: Help
Description: It would have been nice to have some suggestions being offered to help me choose from the seats available. For example, they can ask whether I have any need to walk often to stretch in which case an aisle seat might be better. Or I could be reminded of the responsibilities that passengers seating by the exit rows have.
Fourth Issue/Violation: “Important flight information” section should have been placed in a more noticeable place as a way to get user’s attention. It should also have checkmarks to ensure the user reads that information before continuing with the purchase.
Severity: 3
Heuristics violated: Recognition and Error Prevention
Description: Once I chose my two flights, I was shown a summary. And, on the bottom right corner of the same page, there was a section called “Important Flight Information.” This section was placed almost out of the way, which makes it harder to spot. They should place it somewhere else, and not next to an ad. Also they should have had checkboxes next to the info so that the person buying the tickets would have to put checkmarks as a way to acknowledge they had read each item listed. This is important because it would prevent the user from assuming that the “free 24 hour cancellation” policy was valid when in reality the airline would charge them for any changes.
Fifth Issue/Violation: A way to register on the site (for first time users) was not shown as one of the options at checkout.
Severity: 4
Heuristics violated: Recognition and Help
Description: After I clicked on “continue booking” I saw I needed to enter my personal info. I wondered if I could register with Expedia, so next time I would not have to enter all my info again. But I saw no option. It only said “Sign in to book faster” but sign in is usually for people who already have accounts, and I did not have one.
Since there was no other option, I clicked on “Sign in to book faster” to see what it said. They made it even more attractive to register with them by listing the benefits. However, I still didn’t see an option to register. They either had “checkout as guest” or “sign in” for current registered customers.
The Execution — Travelocity
Task: Purchasing a round-trip flight from San Francisco, California (USA) to Istanbul, Turkey departing on Saturday 07/11/15, and returning on 08/11/15.
First Issue/Violation: Being unsure about which option to pick when choosing airports.
Severity: 2
Heuristics violated: Help and Error Prevention
Description: When trying to choose my “flying from” and “flying to” cities, I was not sure which of the airport choices to choose. It would have been helpful to be able to hover over the choices and see if there is a typical airport that people use when flying to and from both cities.
Second Issue/Violation: Having doubts about whether all flights were non-refundable by default and whether I would be able to change this option later.
Severity: 4
Heuristics violated: Help and Error Prevention
Description: I clicked on “Advanced Options” and was surprised to see they had an option called “Refundable Flight.” I was wondering whether not clicking there would mean I would automatically get a nonrefundable ticket. It would have been nice to have extra information. For instance, they could allow to hover over “Refundable Flight” and show a pop up window with more information to clarify. They could also show whether I can change my options later. Writing something like “Don’t worry if you are unsure. You can always change choices later.” Planning international travel can be anxiety-producing. The user doesn’t need the added anxiety of not knowing whether the tickets will be refundable or not.
Third Issue/Violation: Confusion about their “Free Cancellation within 24 Hours” policy.
Severity: 4
Heuristics violated: Recognition and Error Prevention
Description: I clicked on “Show Flight Details” and I saw a message that said there was “Free Cancellation” within 24 hours of booking. This was interesting considering that I didn’t click on the “refundable” option at the beginning. It would be a good idea to show this info prominently at the beginning or even on the main page if that’s their overall policy. I could have just left to another website after seeing that their main page shows “refundable” as an option, which implies that the default is that flights are nonrefundable.
Fourth Issue/Violation: Having difficulty deciphering the information related to departure, length of the flights and how long I have to wait at a certain airport.
Severity: 3
Heuristics violated: Mapping and Recognition
Description: They could have explained the departure, arrival and time (that one has to wait in between airplanes) better. The information is there but it’s not as easy to figure out at a glance. The user has to take extra time. They could have made it better by writing something like this:
Flight from San Francisco departs at 1:50 pm
Flight duration: 10 h 25 min (notice this is an overnight flight)
Arrives in Amsterdam at 9:15 am of next day 7/12/15
Waiting time in Amsterdam (until your next flight) 2 h 10 min
Final flight departs from Amsterdam at 11:25 am
Flight duration: 3 h 20 min
Arrives in Istanbul at: 3:45 pm
Fifth Issue/Violation: Flight rating does not provide enough information for user to make a decision based on it.
Severity: 3
Heuristics violated: Help
Description: I noticed my flight was rated as “Satisfactory (6.7 out of 10). I wanted to know how they came up with it. So I hovered over the 6.7 out of 10 and I clicked, but there was no info. Then I checked out the other flights below and saw that there was a flight that has 7.8 out of 10 and was rated as “very good flight” and another one that was 9.3 rated “excellent.” So then I wondered why if there was only 1 point difference between 6.7 and 7.8 one flight is “satisfactory” and the other one is “very good.” This made me wonder about: what their standards were; what they meant by those terms; and how their rating worked.
I hope you find these two examples helpful the next time you need to conduct a heuristic evaluation.
*Evaluation conducted on July 2015.
Enjoyed this article? Press the green heart to help others discover it :)
