ComicHub Usability Testing

Fantom Comics in Washington, D.C.

Comic book retailers tend to have limited access to online systems and inventory management tools, making it difficult for comic book subscribers to manage their subscriptions. Stu Colson, owner of New Zealand’s Heroes for Sale comic book store, decided to create his own system, ComicHub. ComicHub provides both a retailer point-of-sale (POS) system and a mobile app for customers to manage their subscriptions. Colson has been using ComicHub internally at his own store, and he has begun reaching out to other stores as early adopters and testers.

View the full report.
View the full report.

My teammates and I conducted an evaluation of the ComicHub iPhone app and presented our results to Colson. We tested the app with frequent customers at Fantom Comics, a comic book store in the Dupont Circle neighborhood of Washington, D.C. We aimed to identify usability and accessibility issues and to present recommendations for the app’s design.

Teammates: Aubrey Scarborough and Michael Brescia

Methods:

  • Heuristic Evaluation
  • Cognitive Walkthrough
  • Task Analysis
  • Open-ended Interviews and guided discussions
  • Pre- and post-test surveys

Software

  • Lookback usability testing tool
  • ComicHub App (iOS)

Read the full report, or continue scrolling to read some highlights.

Usability test participants

Participants

All participants were active members in the comic book community and were regular customers at Fantom Comics. The four participants took part in the test on Sunday, April 17, 2016. The results of the participants’ pre-questionnaire are as follows:

Pretest Questionnaire

 Question Participant 1 Participant 2 Participant 3 Participant 4
What is your age? 27 28 27 29
What is your sex? Female Male Female Female
Years subscribed to comics? 1.5 years 10 years 3 years 2.5 years
How many series do you subscribe to? ~8 ~50 ~15 ~20
How often do you subscribe to new comic series? 2-3 times per year Several times per month Once a month Once a month
How have you subscribed to comics in the past? Comixology Comixology Comixology, “the only way” Fantom’s webpage
Have you used an electronic subscription app in the past? Comixology Comixology, Marvel Unlimited Comixology app Comixology

Tasks

Test participants attempted to complete each of the below tasks. A detailed description of each task scenario can be found in the full report.

  1. Subscribe to a comic drawn by Phil Noto.
  2. Favorite a comic by Annie Wu.
  3. Purchase a variant cover by Fiona Staples.
  4. Check to see if you already own Steven Universe Vol. 2.
  5. Check to see if you’re still subscribed to Dark Horse Star Wars comics.
  6. Find some Marvel Star Wars comics and subscribe to them.
  7. See if any stores in Manhattan use ComicHub.
  8. Add your birthday to your profile.

To test participants, we used an iPhone connected to a laptop. We recorded each user’s screen activity, statements and facial movements.

Testing facility

Results

Summary of Completion, Errors, Time on Task

Task Task Completion Avg Time on Task Errors
Task 1 100% 133 4
Task 2 100% 108 2
Task 3 100% 98 2
Task 4 100% 82 4
Task 5 75% 38 1
Task 6 100% 122 2
Task 7 100% 117 3
Task 8 100% 33 0

Post-Test System Usability Scale (SUS) Questionnaire

 Statement P1 P2 P3 P4
1. I think that I would like to use this system frequently. 4 4 4 5
2. I found the system unnecessarily complex. 3 2 2 3
3. I thought the system was easy to use. 4 4 4 4
4. I think that I would need the support of a technical person to be able to use this system. 2 1 1 2
5. I found the various functions in this system were well integrated. 4 4 4 3
6. I thought there was too much inconsistency in this system. 1 1 2 3
7. I would imagine that most people would learn to use this system very quickly. 4 5 4 4
8. I found the system very cumbersome to use. 2 1 1 3
9. I felt very confident using the system. 3 4 4 3
10. I needed to learn a lot of things before I could get going with this system. 1 1 1 2
SUS Score 75 87.5 82.5 65

Liked Most

The following comments capture what the participants liked most:

  • “I like the history feature and the ability to see what’s been taken off of my list. Having an app, period, is great.”
  • “I liked filtering. It made the searches clear. Comixology is a lot more finicky with titles.”
  • “It’s intuitive. It’s a good user experience.”
  • “I like the scope of the artists included in the app. It’s not just the heavy hitters or big names.”

Liked Least

The following comments capture what the participants liked the least:

    • App navigation
      • “Scrolling. There’s so much scrolling” — noted after trying to find an artist whose name started with a P.
    • Filtering and Searching
      • “I liked that you could select multiple artists, until I realized that you couldn’t.”
      • “You need to be careful to deselect filters.”
      • “I really want a search function. Filters should clear after the search.”
      • “I was expecting to see something like Variant Cover A, B or C. It tells you that you’re looking at a variant cover but doesn’t specify which one.”
    • Other parts of app
      • “I was confused about some of the terminology. I wonder if they use different terminology in New Zealand (where the app was developed) and that’s why I didn’t understand some of the sections.
      • “There’s no locator button for the map. I’d like to see nearby stores.”

Recommendations

  • Filtering: Several tasks (1, 2, 3 and 6) required participants to use the app’s filtering search system to find comic books by artist, publisher, brand and more.
    • Automatically reset filters after user completes a search.
    • Add OR/AND search options.
      Save searches as preferences.
  • Terminology: In Task 6, participants had to find Star Wars comics published by Marvel. 
    • Adjust and/or define terminology throughout the app for audiences in different geographical areas.
  • Locator Map: Task 7 asked users to search for ComicHub stores in Manhattan.
    • Add “store locations” to sidebar so that it’s not just hidden under preferences.
    • Add a search feature to the map.
    • Add a list of locations so users don’t have to rely on the map.
    • On map page open, locator point should be at user’s current location or their specified city.
  • Variant cover: Task 3 asked participants to find a variant cover by the comic book artist Fiona Staples.
    • Add description to variant covers so users can be confident that they know which cover they are purchasing.
  • Search bar: As noted earlier, four tasks (1, 2, 3 and 6) asked participants to use the filter to find comic books.
    • Add a search bar.
    • Add autocomplete to search.

ComicHub logo

Leave a note

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s