Lecture 1- Introduction

Lecture 1- Introduction

Evaluations Sampath Jayarathna Cal Poly Pomona 1 Evaluation Methods Inspection methods (no users needed) Heuristic Evaluations Walkthroughs

put yourself in the shoes of a user On-line, remote usability tests User Tests (users needed!) Observations/ Ethnography Usability tests/ Controlled experiments 2 Discount Usability Engineering Cheap no special labs or equipment needed the more careful you are, the better it gets

Fast on order of 1 day to apply standard usability testing may take a week Easy to use can be taught in 2-4 hours 3 Heuristic Evaluation Developed by Jakob Nielsen Helps find usability problems in a UI design Small set (3-5) of evaluators examine UI independently check for compliance with usability principles

(heuristics) different evaluators will find different problems evaluators only communicate afterwards findings are then aggregated Can perform on working UI or on sketches These heuristics have been revised for current technology by Nielsen and others for: mobile devices, wearables, virtual worlds, etc. 4

Revised version (2014) of Nielsens original heuristics H-1: Visibility of system status. H-2: Match between system and real world. H-3: User control and freedom. H-4: Consistency and standards. H-5: Error prevention. H-6: Recognition rather than recall. H-7: Flexibility and efficiency of use. H-8: Aesthetic and minimalist design. H-9: Help users recognize, diagnose, recover from errors. H-10: Help and documentation. 5

Heuristics H-1: Visibility of system status The system should always keep users informed about what is going on, through appropriate feedback within reasonable time H-2: Match between system & real world speak the users language follow real world conventions, e.g., Metaphors H-3: User control & freedom emergency exits for mistaken choices, undo, redo dont force down fixed paths

If an operation takes more than 10 seconds user should be able to cancel it. 6 Heuristics (cont.) H-4: Consistency & standards Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions UI should be consistent troughout application E.g. layout of UI components should not

change Especially shortcuts, like keyboard combinations, should remain the same Style guides should be produced and used 7 Heuristics (cont.) H-5: Error prevention Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action

H-6: Recognition rather than recall Minimize the user's memory load by making objects, actions, and options visible. Instructions for use of the system should be visible or easily retrievable whenever appropriate 8 Heuristics (cont.) Edit Cut Copy Paste

H-7: Flexibility and efficiency of use accelerators for experts (e.g., gestures, kb shortcuts) allow users to tailor frequent actions (e.g., macros) UIs can be of an adaptive kind Users actions are observed UI automatically adjusts itself to the most suitable form UIs could, for example, automatically progress from novice level to expert level 9 Heuristics (cont.)

H-8: Aesthetic and minimalist design Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility. 10 Heuristics (cont.) H-9: Help users recognize, diagnose, and recover from errors error messages in plain language (no codes) precisely indicate the problem constructively suggest a solution Error messages can be used as explanations

of applications conceptual model Expressions should be polite/neutral 11 Heuristics (cont.) H-10: Help and documentation easy to search focused on the users task list concrete steps to carry out not too large Documentations are used by users as a last resort Online docs may be better than printed

ones fast search functions do not require a shift in eyesights focus Writing a good set of instructions is a demanding task 12 Phases of Heuristic Evaluation 1) Pre-evaluation training give evaluators needed domain knowledge and information on the scenario 2) Evaluation

individuals evaluate and then aggregate results 3) Severity rating determine how severe each problem is (priority) can do this first individually & then as a group 4) Debriefing discuss the outcome with design team 13 4 stages for doing heuristic evaluation Briefing session to tell experts what to do. If system is walk-up-and-use or evaluators are domain experts, no assistance needed.

otherwise might supply evaluators with scenarios Evaluation period of 1-2 hours in which: Each expert works separately; Take one pass to get a feel for the product; Take a second pass to focus on specific features. Each evaluator produces list of problems explain why with reference to heuristic or other information be specific and list each problem separately Debriefing session in which experts work together to prioritize problems.

14 Examples Cant copy info from one window to another violates User Control and freedom (H-3) fix: allow copying Typography uses mix of upper/lower case formats and fonts violates Consistency and standards (H-4) slows users down probably wouldnt be found by user testing fix: pick a single format for entire interface

15 Severity Rating Used to allocate resources to fix problems Estimates of need for more usability efforts Combination of frequency impact persistence (one time or repeating) Should be calculated after all evals. are in Should be done independently by all judges 16

Severity Ratings (cont.) 0 - dont agree that this is a usability problem 1 - cosmetic problem 2 - minor usability problem 3 - major usability problem; important to fix 4 - usability catastrophe; imperative to fix 17 Severity Ratings Example 1. [H-4 Consistency] [Severity 3]

The interface used the string "Save" on the first screen for saving the user's file, but used the string "Write file" on the second screen. Users may be confused by this different terminology for the same function. 18 Debriefing Conduct with evaluators, observers, and development team members Discuss general characteristics of UI Suggest potential improvements to address major usability problems Dev. team rates how hard things are to fix

Make it a brainstorming session little criticism until end of session 19 Advantages and problems Few ethical & practical issues to consider because users not involved. Can be difficult & expensive to find experts. Best experts have knowledge of application domain & users. Biggest problems: Important problems may get missed; Many trivial problems are often identified;

Experts have biases. 20 Number of evaluators Nielsen suggests that on average 5 evaluators identify 7580% of usability problems. Single evaluator achieves poor results only finds 35% of usability problems 5 evaluators find ~ 75% of usability problems why not more evaluators???? 10? 20? adding evaluators costs more many evaluators wont find many more problems 21

Individual vs. Teams Nielsen Recommends individual evaluations inspect the interface alone Why? Evaluation is not influenced by others Independent and unbiased Greater variability in the kinds of errors found No overhead required to organize group meetings

22 Self Guided vs. Scenario Exploration Self- guided Open ended exploration Not necessarily task-directed Good for exploring diverse aspects of the interface, and to follow potential pitfalls Scenarios

Step through the interface using representative end user tasks Ensures problems identified in relevant portions of the interface Ensures that specific features of interest are evaluated But limits the scope of the evaluation problems can be missed 23 HE vs. User Testing HE is much faster 1-2 hours each evaluator vs. days-weeks HE doesnt require interpreting users actions

User testing is far more accurate (by def.) takes into account actual users and tasks HE may miss problems & find false positives Good to alternate between HE & user testing find different problems dont waste participants 24 Summary Heuristic evaluation is a discount method Have evaluators go through the UI twice Ask them to see if it complies with heuristics

note where it doesnt and say why Combine the findings from 3 to 5 evaluators Have evaluators independently rate severity Discuss problems with design team Alternate with user testing 25 Usability Testing and Laboratories The usability lab consists of two areas: the testing room and the observation room The testing room is typically smaller and accommodates a small number of people

The observation room, can see into the testing room typically via a one-way mirror. The observation room is larger and can hold the usability testing facilitators with ample room to bring in others, such as the developers of the product being tested 26 Usability Testing and Laboratories(continued) This shows a picture of glasses worn for eye-tracking This particular device tracks the participants eye movements when using a mobile device Tobii is one of several manufacturers 27

Usability Testing and Laboratories (continued) Eye-tracking software is attached to the airline checkin kiosk It allows the designer to collect data observing how the user looks at the screen This helps determine if various interface elements (e.g. buttons) are difficult (or easy) to find 28 Usability Testing and Laboratories (continued) The special mobile camera to track and record activities on a mobile device Note the camera is up and out of the way still allowing

the user to use their normal finger gestures to operate the device 29 Experiments & usability testing Experiments test hypotheses to discover new knowledge by investigating the relationship between two or more variables. Usability testing is applied experimentation. Developers check that the system is usable by the intended user population for their tasks. 30

Testing conditions Usability lab or other controlled space. Emphasis on: selecting representative users; developing representative tasks. 5-10 users typically selected. Tasks usually around 30 minutes Test conditions are the same for every participant. Informed consent form explains procedures and deals with ethical issues. 31 Example: Usability testing the iPad 7 participants with 3+ months experience with iPhones

Signed an informed consent form explaining: what the participant would be asked to do; the length of time needed for the study; the compensation that would be offered for participating; participants right to withdraw from the study at any time; a promise that the persons identity would not be disclosed; and an agreement that the data collected would be confidential and would be available to

only the evaluators Then they were asked to explore the iPad Next they were asked to perform randomly assigned specified tasks 32 Experimental designs Different participants - single group of participants is allocated randomly to the experimental conditions. Same participants - all participants appear in both conditions. Matched participants - participants are matched in pairs, e.g., based on expertise, gender, etc. 33

Field studies Field studies are done in natural settings. In the wild is a term for prototypes being used freely in natural settings. Aim to understand what users do naturally and how technology impacts them. Field studies are used in product design to: identify opportunities for new technology; determine design requirements; decide how best to introduce new technology; evaluate technology in use. 34

Recently Viewed Presentations

  • CSC 594 Topics in AI Natural Language Processing

    CSC 594 Topics in AI Natural Language Processing

    Open Class Words. Nouns. Proper nouns (Boulder, Granby, Eli Manning) English capitalizes these. Common nouns (the rest). Count nouns and mass nouns
  • Parts of Speech Guided Notes - Council Rock School District

    Parts of Speech Guided Notes - Council Rock School District

    Parts of Speech Guided Notes NOUN Function: PERSON, PLACE, THING, or IDEA. Examples: Person Place Thing Idea MOTHER HOME MONEY FREEDOM TEACHER SCHOOL IPOD HOPE OFFICER MALL BUS FAIRNESS Nouns The SUBJECT (that does the action) of a sentence is...
  • Excessive Heat CERT Basic Training Hazards Heat Wave

    Excessive Heat CERT Basic Training Hazards Heat Wave

    Excessive Heat CERT Basic Training Hazards Heat Wave Prolonged period of excessive heat, often combined with excessive humidity He-* CERT Basic Training Unit 1: Excessive Heat Effects of Excessive Heat Body must work extra hard to maintain its normal temperature...
  • Language and Communicaion

    Language and Communicaion

    Visuospatial working memory. Symbol receptive vocabulary score . Symbol expressive vocabulary score . Motor receptive vocabulary score . Motor expressive vocabulary score . Verbal Working Memory- Visuospatial working memory.30- Symbol receptive vocabulary score .16.40*- Symbol expressive vocabulary score .13.27.56**-
  • Welcome to the Life Cycle Assessment (LCA) Learning

    Welcome to the Life Cycle Assessment (LCA) Learning

    Comparative Toxic Unit. Cases. Morbidity disease cases. per kg substance released. PAF*m3*day. Potentially affected fraction integrated over volume and time. Note: PAF is the % of the species exposed to concentrations above their "no observable effects concentration" per kg substance...
  • Social Learning Theory

    Social Learning Theory

    Attachment Theory There is a deep emotional tie, almost a physical connection with a loved one This is vital throughout life John Bowlby, Harry Harlow, Mary Ainsworth Powerful Contact Comfort Several Attachment Survival Impulse Monkeys Styles Existed Theories of Attachment...
  • A propos des exposés…

    A propos des exposés…

    En 1812, la Louisiane est le seul Etat où la majorité linguistique n'est pas anglaise: populations francophones, créoles et espagnoles. 1806: la Louisiane devient un des Etat des Etats-Unis et voit arriver plus de 10 000 créoles. * CODOFIL: chargé...
  • Insolvency and Bankruptcy Code-2016 Professional ...

    Insolvency and Bankruptcy Code-2016 Professional ...

    Recovery of debts - too low vis-à-vis other countries. Understanding the Code. Vaish Associates Privileged & Confidential. Differentiation ... Liquidator to scrutinise Preferential, under-valued and extortionate credit transactions. Distribution of assets and dissolution of Corporate Debtor.