Lecture 1- Introduction

Lecture 1- Introduction

Evaluations Sampath Jayarathna Cal Poly Pomona 1 Evaluation Methods Inspection methods (no users needed) Heuristic Evaluations Walkthroughs

put yourself in the shoes of a user On-line, remote usability tests User Tests (users needed!) Observations/ Ethnography Usability tests/ Controlled experiments 2 Discount Usability Engineering Cheap no special labs or equipment needed the more careful you are, the better it gets

Fast on order of 1 day to apply standard usability testing may take a week Easy to use can be taught in 2-4 hours 3 Heuristic Evaluation Developed by Jakob Nielsen Helps find usability problems in a UI design Small set (3-5) of evaluators examine UI independently check for compliance with usability principles

(heuristics) different evaluators will find different problems evaluators only communicate afterwards findings are then aggregated Can perform on working UI or on sketches These heuristics have been revised for current technology by Nielsen and others for: mobile devices, wearables, virtual worlds, etc. 4

Revised version (2014) of Nielsens original heuristics H-1: Visibility of system status. H-2: Match between system and real world. H-3: User control and freedom. H-4: Consistency and standards. H-5: Error prevention. H-6: Recognition rather than recall. H-7: Flexibility and efficiency of use. H-8: Aesthetic and minimalist design. H-9: Help users recognize, diagnose, recover from errors. H-10: Help and documentation. 5

Heuristics H-1: Visibility of system status The system should always keep users informed about what is going on, through appropriate feedback within reasonable time H-2: Match between system & real world speak the users language follow real world conventions, e.g., Metaphors H-3: User control & freedom emergency exits for mistaken choices, undo, redo dont force down fixed paths

If an operation takes more than 10 seconds user should be able to cancel it. 6 Heuristics (cont.) H-4: Consistency & standards Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions UI should be consistent troughout application E.g. layout of UI components should not

change Especially shortcuts, like keyboard combinations, should remain the same Style guides should be produced and used 7 Heuristics (cont.) H-5: Error prevention Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action

H-6: Recognition rather than recall Minimize the user's memory load by making objects, actions, and options visible. Instructions for use of the system should be visible or easily retrievable whenever appropriate 8 Heuristics (cont.) Edit Cut Copy Paste

H-7: Flexibility and efficiency of use accelerators for experts (e.g., gestures, kb shortcuts) allow users to tailor frequent actions (e.g., macros) UIs can be of an adaptive kind Users actions are observed UI automatically adjusts itself to the most suitable form UIs could, for example, automatically progress from novice level to expert level 9 Heuristics (cont.)

H-8: Aesthetic and minimalist design Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility. 10 Heuristics (cont.) H-9: Help users recognize, diagnose, and recover from errors error messages in plain language (no codes) precisely indicate the problem constructively suggest a solution Error messages can be used as explanations

of applications conceptual model Expressions should be polite/neutral 11 Heuristics (cont.) H-10: Help and documentation easy to search focused on the users task list concrete steps to carry out not too large Documentations are used by users as a last resort Online docs may be better than printed

ones fast search functions do not require a shift in eyesights focus Writing a good set of instructions is a demanding task 12 Phases of Heuristic Evaluation 1) Pre-evaluation training give evaluators needed domain knowledge and information on the scenario 2) Evaluation

individuals evaluate and then aggregate results 3) Severity rating determine how severe each problem is (priority) can do this first individually & then as a group 4) Debriefing discuss the outcome with design team 13 4 stages for doing heuristic evaluation Briefing session to tell experts what to do. If system is walk-up-and-use or evaluators are domain experts, no assistance needed.

otherwise might supply evaluators with scenarios Evaluation period of 1-2 hours in which: Each expert works separately; Take one pass to get a feel for the product; Take a second pass to focus on specific features. Each evaluator produces list of problems explain why with reference to heuristic or other information be specific and list each problem separately Debriefing session in which experts work together to prioritize problems.

14 Examples Cant copy info from one window to another violates User Control and freedom (H-3) fix: allow copying Typography uses mix of upper/lower case formats and fonts violates Consistency and standards (H-4) slows users down probably wouldnt be found by user testing fix: pick a single format for entire interface

15 Severity Rating Used to allocate resources to fix problems Estimates of need for more usability efforts Combination of frequency impact persistence (one time or repeating) Should be calculated after all evals. are in Should be done independently by all judges 16

Severity Ratings (cont.) 0 - dont agree that this is a usability problem 1 - cosmetic problem 2 - minor usability problem 3 - major usability problem; important to fix 4 - usability catastrophe; imperative to fix 17 Severity Ratings Example 1. [H-4 Consistency] [Severity 3]

The interface used the string "Save" on the first screen for saving the user's file, but used the string "Write file" on the second screen. Users may be confused by this different terminology for the same function. 18 Debriefing Conduct with evaluators, observers, and development team members Discuss general characteristics of UI Suggest potential improvements to address major usability problems Dev. team rates how hard things are to fix

Make it a brainstorming session little criticism until end of session 19 Advantages and problems Few ethical & practical issues to consider because users not involved. Can be difficult & expensive to find experts. Best experts have knowledge of application domain & users. Biggest problems: Important problems may get missed; Many trivial problems are often identified;

Experts have biases. 20 Number of evaluators Nielsen suggests that on average 5 evaluators identify 7580% of usability problems. Single evaluator achieves poor results only finds 35% of usability problems 5 evaluators find ~ 75% of usability problems why not more evaluators???? 10? 20? adding evaluators costs more many evaluators wont find many more problems 21

Individual vs. Teams Nielsen Recommends individual evaluations inspect the interface alone Why? Evaluation is not influenced by others Independent and unbiased Greater variability in the kinds of errors found No overhead required to organize group meetings

22 Self Guided vs. Scenario Exploration Self- guided Open ended exploration Not necessarily task-directed Good for exploring diverse aspects of the interface, and to follow potential pitfalls Scenarios

Step through the interface using representative end user tasks Ensures problems identified in relevant portions of the interface Ensures that specific features of interest are evaluated But limits the scope of the evaluation problems can be missed 23 HE vs. User Testing HE is much faster 1-2 hours each evaluator vs. days-weeks HE doesnt require interpreting users actions

User testing is far more accurate (by def.) takes into account actual users and tasks HE may miss problems & find false positives Good to alternate between HE & user testing find different problems dont waste participants 24 Summary Heuristic evaluation is a discount method Have evaluators go through the UI twice Ask them to see if it complies with heuristics

note where it doesnt and say why Combine the findings from 3 to 5 evaluators Have evaluators independently rate severity Discuss problems with design team Alternate with user testing 25 Usability Testing and Laboratories The usability lab consists of two areas: the testing room and the observation room The testing room is typically smaller and accommodates a small number of people

The observation room, can see into the testing room typically via a one-way mirror. The observation room is larger and can hold the usability testing facilitators with ample room to bring in others, such as the developers of the product being tested 26 Usability Testing and Laboratories(continued) This shows a picture of glasses worn for eye-tracking This particular device tracks the participants eye movements when using a mobile device Tobii is one of several manufacturers 27

Usability Testing and Laboratories (continued) Eye-tracking software is attached to the airline checkin kiosk It allows the designer to collect data observing how the user looks at the screen This helps determine if various interface elements (e.g. buttons) are difficult (or easy) to find 28 Usability Testing and Laboratories (continued) The special mobile camera to track and record activities on a mobile device Note the camera is up and out of the way still allowing

the user to use their normal finger gestures to operate the device 29 Experiments & usability testing Experiments test hypotheses to discover new knowledge by investigating the relationship between two or more variables. Usability testing is applied experimentation. Developers check that the system is usable by the intended user population for their tasks. 30

Testing conditions Usability lab or other controlled space. Emphasis on: selecting representative users; developing representative tasks. 5-10 users typically selected. Tasks usually around 30 minutes Test conditions are the same for every participant. Informed consent form explains procedures and deals with ethical issues. 31 Example: Usability testing the iPad 7 participants with 3+ months experience with iPhones

Signed an informed consent form explaining: what the participant would be asked to do; the length of time needed for the study; the compensation that would be offered for participating; participants right to withdraw from the study at any time; a promise that the persons identity would not be disclosed; and an agreement that the data collected would be confidential and would be available to

only the evaluators Then they were asked to explore the iPad Next they were asked to perform randomly assigned specified tasks 32 Experimental designs Different participants - single group of participants is allocated randomly to the experimental conditions. Same participants - all participants appear in both conditions. Matched participants - participants are matched in pairs, e.g., based on expertise, gender, etc. 33

Field studies Field studies are done in natural settings. In the wild is a term for prototypes being used freely in natural settings. Aim to understand what users do naturally and how technology impacts them. Field studies are used in product design to: identify opportunities for new technology; determine design requirements; decide how best to introduce new technology; evaluate technology in use. 34

Recently Viewed Presentations

  • <Insert Lesson, Module, or Course Title>

    <Insert Lesson, Module, or Course Title>

    Profile APA, MSc. Statistics, MSc. Economics, MA Arabic, MA Islamic Studies Project Manager/ Sr. Financial Consultant Oracle EBS Instructor in OTS/ TE Oracle Certified professional Oracle Certified Support Specialist Salaih Muhammad (ACA) Muhammad Ali MSc Accounting & Finance Functional Consultant...
  • Soil Organisms Classification schemes for organisms Basis for

    Soil Organisms Classification schemes for organisms Basis for

    Soil Organisms Classification schemes for organisms Basis for classification How classifications relate to each other What classification says about an organism's role in environmental processes and vice versa
  • Barrington - Koç Hastanesi

    Barrington - Koç Hastanesi

    Democracy. Selection of government officials through free and fair elections. The balance of majority rule and minority protection. Limitations on government action. Variants of democracy. Majoritarian, consensus. European, Westminster, American and Latin American models
  • Help I Need a Research Project - Commack Schools

    Help I Need a Research Project - Commack Schools

    "But Teacher I do not know what to do?" Teacher Response = Brainstorm, go nuts on the INTERNET at the sites we/you like Science Direct/Google/Google Scholar/ Science News/YouTube/Ted.com >Help I need a Project on our webpage...
  • AP Lang Journey - Mr. Mac&#x27;s Class

    AP Lang Journey - Mr. Mac's Class

    Huck Finn - a reflection of our American Self . Friendship, race, intelligence versus education, being an outsider. Good intentions but carrying the flaws of one's own historical period. Vernacular. Satire. ... Werner Heisenberg. Jacob Bronowski .
  • Kvadriplegija i paraplegija

    Kvadriplegija i paraplegija

    ASIA SKALA OŠTEĆENJA. A kompletno oštećenje (ne postoji očuvanost niti motornih niti senzitivnih funkcija u sakralnim segmentima S4-S5) ... odnosi se na dermatome i miotome ispod neurološkog nivoa lezije koji su delimično inervisani. Registruje se tačan broj parcijalno denervisanih ...
  • Briefing on Notional Core Training for Services Acquisition

    Briefing on Notional Core Training for Services Acquisition

    Required for Programs with a Total Contract Value of $1 billion and above "Contract Execution" Description: Provides Program Managers with an understanding of execution of the acquisition plan through soliciting industry and awarding a contract. "Contract Management"
  • Atmospheric Humidity

    Atmospheric Humidity

    Clouds Cloud formation Large mass of air rises, cools to dew point Condensation (or deposition) occurs on condensation nuclei salt, dust, soil, pollutants elevation at which air becomes saturated is lifting condensation level (flat base of clouds) A cloud is...