Measuring Information Architecture - Dublin Core

Measuring Information Architecture - Dublin Core

Content Metadata and Search Remarks to the Dublin Core Workshop Marti Hearst SIMS, UC Berkeley September 28, 2003 Resource Finding and the Web Web search vs. collection search When a single page is all thats needed, web search is fine Although validity is an issue Unsolved problem: How to make source-focused search more intuitive on the web? One idea (untested): task-based search

M. Hearst Faceted Metadata in Search What about Content? Dublin Core takes stances on the content-neutral aspects of metadata Q: What about content? The Metadata Marsh Getting agreement on metadata terms is difficult Even worse when talking about content! A: Domain-specific solutions Dont worry about cross-domain consistency (a necessary drawback) Success: b-to-b protocols

M. Hearst Faceted Metadata in Search Hypothesis (as yet untested): Assuming weve focused on a domain, agreement on category assignment can converge much more quickly by: 1. Focusing on the applications that will use the category system. 2. Designing metadata to be used in interfaces that show items represented by many different categories in a highly flexible, but intuitive, manner. M. Hearst

Faceted Metadata in Search One Example: Flamenco Project Goal: create intuitive, inviting search interfaces that make use of hierarchical faceted metadata Challenge: How to provide flexibility and power without overwhelming? (Answer: careful interface design) M. Hearst Faceted Metadata in Search The Flamenco Project Team Brycen Chun

Ame Elliott Jennifer English Kevin Li Rashmi Sinha Kirsten Swearingen Ping Yee http://flamenco.berkeley.edu Research funded by: NSF CAREER Grant IIS-9984741 IBM Faculty Fellowship 6 Our Approach Integrate the search seamlessly into the information architecture. Use proper HCI methodologies.

Use faceted metadata: More flexible than canned hyperlinks Less complex than full search Help users see where to go next and return to what happened previously Whats new? Putting hierarchical facets into a useable interface. M. Hearst Faceted Metadata in Search Metadata: data about data Facets: orthogonal categories

GeoRegion M. Hearst Time/Date Topic Faceted Metadata in Search Hierarchical Faceted Metadata Example: Biological Subject Headings 1. Anatomy [A] 2. Organisms [B] 3. Diseases [C]

4. Chemicals and Drugs [D] 5. Analytical, Diagnostic and Therapeutic Techniques and Equipment [E] 6. Psychiatry and Psychology [F] 7. Biological Sciences [G] 8. Physical Sciences [H] 9. Anthropology, Education, Sociology and Social Phenomena [I] 10. Technology and Food and Beverages [J] 11. Humanities [K] 12. Information Science [L] 13. Persons [M] 14. Health Care [N] 15. Geographic Locations [Z] M. Hearst Faceted Metadata in Search

Hierarchical Faced Metadata 1. Anatomy [A] 2. [B] 3. [C] 4. [D] 5. [E] 6. [F] 7. [G] 8. Physical Sciences 9. [I] 10. [J] 11. [K] 12. [L] 13. [M] M. Hearst

Body Regions [A01] Musculoskeletal System [A02] Digestive System [A03] Respiratory System [A04] Urogenital System [A05] [H] Faceted Metadata in Search Hierarchical Faceted Metadata 1. Anatomy [A] 2. [B] 3. [C] 4. [D] [A01.378] 5. [E]

6. [F] 7. [G] 8. Physical Sciences 9. [I] 10. [J] 11. [K] 12. [L] 13. [M] M. Hearst Body Regions [A01] Musculoskeletal System [A02] Digestive System [A03] Respiratory System [A04] Urogenital System [A05]

Abdomen [A01.047] Back [A01.176] Breast [A01.236] Extremities Head [A01.456] Neck [A01.598] . [H] Faceted Metadata in Search Hierarchical Faceted Metadata 1. Anatomy [A] 2. [B] 3. [C] 4. [D]

[A01.378] 5. [E] 6. [F] 7. [G] 8. Physical Sciences 9. [I] 10. [J] 11. [K] 12. [L] 13. [M] M. Hearst Body Regions [A01] Musculoskeletal System [A02] Digestive System [A03] Respiratory System [A04] Urogenital System [A05]

[H] Abdomen [A01.047] Back [A01.176] Breast [A01.236] Extremities Head [A01.456] Neck [A01.598] . Electronics Astronomy Nature Time Weights and Measures .

Faceted Metadata in Search Hierarchical Faceted Metadata 1. Anatomy [A] 2. [B] 3. [C] 4. [D] [A01.378] 5. [E] 6. [F] 7. [G] 8. Physical Sciences 9. [I] 10. [J] 11. [K] 12. [L] 13. [M]

M. Hearst Body Regions [A01] Musculoskeletal System [A02] Digestive System [A03] Respiratory System [A04] Abdomen [A01.047] Back [A01.176] Breast [A01.236] Extremities Urogenital System [A05] [H] Head [A01.456]

Neck [A01.598] . Amplifiers Electronics, Medical Transducers Electronics Astronomy Nature Time Weights and Measures . Faceted Metadata in Search Hierarchical Faceted Metadata 1. Anatomy [A]

2. [B] 3. [C] 4. [D] 5. [E] 6. [F] 7. [G] 8. Physical Sciences 9. [I] 10. [J] 11. [K] 12. [L] 13. [M] M. Hearst Body Regions [A01] Musculoskeletal System [A02]

Digestive System [A03] Respiratory System [A04] Urogenital System [A05] [H] Abdomen [A01.047] Back [A01.176] Breast [A01.236] Extremities [A01.378] Head [A01.456] Neck [A01.598] . Amplifiers Electronics, Medical Transducers

Electronics Astronomy Nature Time Weights and Measures . Calibration Metric System Reference Standard Faceted Metadata in Search The Interface Design Chess metaphor Opening Middle game End game

M. Hearst Faceted Metadata in Search M. Hearst Faceted Metadata in Search M. Hearst Faceted Metadata in Search M. Hearst Faceted Metadata in Search

M. Hearst Faceted Metadata in Search M. Hearst Faceted Metadata in Search M. Hearst Faceted Metadata in Search M. Hearst Faceted Metadata in Search M. Hearst

Faceted Metadata in Search M. Hearst Faceted Metadata in Search The Interface Design Tightly Integrated Search Supports Expand as well as Refine Dynamically Generated Pages Paths can be taken in any order Links are idempotent Consistent Color Coding Consistent Backup and Bookmarking Standard HTML

No javascript M. Hearst Faceted Metadata in Search What is Tricky About This? It is easy to do it poorly Yahoo directory structure It is hard to be not overwhelming Most users prefer simplicity unless complexity really makes a difference It is hard to make it flow Can it feel like browsing the shelves? Yes, but we iterated the design 3 times

M. Hearst Faceted Metadata in Search Usability Study Participants & Collection 32 Art History Students ~35,000 images from SF Fine Arts Museum Study Design Within-subjects Each participant sees both interfaces Balanced in terms of order and tasks Participants assess each interface after use Afterwards they compare them directly Data recorded in behavior logs, server logs, papersurveys; one or two experienced testers at each trial.

Used 9 point Likert scales. Session took about 1.5 hours; pay was $15/hour M. Hearst Faceted Metadata in Search The Baseline System Floogle Take the best of the existing keyword-based image search systems M. Hearst Faceted Metadata in Search sword

M. Hearst Faceted Metadata in Search M. Hearst Faceted Metadata in Search M. Hearst Faceted Metadata in Search M. Hearst Faceted Metadata in Search

Hypotheses We attempted to design tasks to test the following hypotheses: Participants will experience greater search satisfaction, feel greater confidence in the results, produce higher recall, and encounter fewer dead ends using FC over Baseline FC will perceived to be more useful and flexible than Baseline Participants will feel more familiar with the contents of the collection after using FC Participants will use FC to create multifaceted queries M. Hearst Faceted Metadata in Search Four Types of Tasks

Unstructured (3): Search for images of interest Structured Task (11-14): Gather materials for an art history essay on a given topic, e.g. Find all woodcuts created in the US Choose the decade with the most Select one of the artists in this periods and show all of their woodcuts Choose a subject depicted in these works and find another artist who treated the same subject in a different way. Structured Task (10): compare related images Find images by artists from 2 different countries that depict conflict between groups. Unstructured (5): search for images of interest M. Hearst

Faceted Metadata in Search Other Points Participants were NOT walked through the interfaces. The wording of Task 2 reflected the metadata; not the case for Task 3 Within tasks, queries were not different in difficulty (ts<1.7, p >0.05 according to post-task questions) Flamenco is and order of magnitude slower than Floogle on average. In task 2 users were allowed 3 more minutes in FC than in Baseline. Time spent in tasks 2 and 3 were significantly longer in FC (about 2 min more). M. Hearst

Faceted Metadata in Search Post-Interface Assessments All significant at p<.05 except simple and overwhelming M. Hearst Faceted Metadata in Search Post-Test Comparison Which Interface Preferable For: Baseline FC Find images of roses Find all works from a given period 15

16 2 30 Find pictures by 2 artists in same media 1 29 4 28

8 23 6 24 28 3 1 31 2

29 M. Hearst Faceted Metadata in Search Post-Test Comparison Which Interface Preferable For: Baseline FC Find images of roses Find all works from a given period 15 16 2

30 Find pictures by 2 artists in same media 1 29 4 28 8 23

6 24 28 3 1 31 2 29 Overall Assessment:

More useful for your tasks Easiest to use Most flexible More likely to result in dead ends Helped you learn more Overall preference M. Hearst Faceted Metadata in Search Study Results Summary Strongly positive results for the faceted metadata interface. Moderate use of multiple facets. Strong preference over the current state of the art. Chair of Architecture Dept: It felt like I was

browsing the shelves! This kind of enthusiasm is not seen in similarity-based image search interfaces. Hypotheses are supported. M. Hearst Faceted Metadata in Search Study Summary Usability studies done on 3 collections: Recipes: 13,000 items Architecture Images: 40,000 items Fine Arts Images: 35,000 items Conclusions: Users like and are successful with the

dynamic faceted hierarchical metadata, especially for browsing tasks Very positive results, in contrast with studies on earlier iterations Note: it seems you have to care about the contents of the collection to like the interface M. Hearst Faceted Metadata in Search Advantages of the Approach Supports different search types Highly constrained known-item searches Open-ended, browsing tasks Can easily switch from one mode to the other midstream Can both expand and refine

Allows different people to add content without breaking things Can make use of standard technology M. Hearst Faceted Metadata in Search Metadata Availability Many collections already have rich metadata associated with them. Automated methods are improving. Have applied this to: Tobacco documents archive MEDLINE

M. Hearst Faceted Metadata in Search Back to the Hypothesis This kind of tool may be helpful for resolving metadata creation wars. Multiple paths to get to the same item Different views on different subsets of items No need to force everything into one hierarchy What do you think? M. Hearst

Faceted Metadata in Search

Recently Viewed Presentations

  • Kin selection, inclusive fitness Hamiltons rule: r n

    Kin selection, inclusive fitness Hamiltons rule: r n

    Friendship, gratitude, sympathy, loyalty, betrayal, guilt, dislike, revenge, trust, suspicion, dishonesty, hypocrisy Game Theoretic Approaches Costs versus benefits of behaviors "tit for tat" strategy + forgiveness can lead to cooperation ("the future casts a long shadow back on the present")...
  • Role of Focus Groups and Surveys Roleinof Focus

    Role of Focus Groups and Surveys Roleinof Focus

    Users: Nursing Home/Assistive Living Aids- Care for people with transfer needs in an institutional setting. Mix of experience with manual lifting and mechanical lifts to transfer patients. In the business for at least 2 years. Users/Buyers: People with Mobility Limitations...
  • Chapter 17 Section 2: The Northern Renaissance

    Chapter 17 Section 2: The Northern Renaissance

    Chapter 17 Section 2: The Northern Renaissance Setting the Stage Classic ideas impressed academics and students who visited Italy. Merchants spread ideas from Italy to other European urban centers. 1400's Renaissance spread to Northern Europe England France Germany The Northern...
  • ehsspanish225.weebly.com

    ehsspanish225.weebly.com

    The Command Voice. El Imperativo. We use the command voice to tell people what to do. In Spanish there are . two. command voices: The familiar commands-to tell people you would normally address in the tú form what to do.
  • The NHS in West Lancashire Tuesday 25 April

    The NHS in West Lancashire Tuesday 25 April

    The December 2016 prescribing figures have shown a £189K reduction in cost compared to December 2015. We also saw a reduction in the number of items being dispensed, increased use of our online portal Patient Access which allows patients to...
  • Microstructure-Properties: II Lecture 6: Crystallization of Glass

    Microstructure-Properties: II Lecture 6: Crystallization of Glass

    Microstructure-Properties: II Crystallization of Glass 27-302 Lecture 4 Fall, 2002 Prof. A. D. Rollett Materials Tetrahedron Objective The objective of this lecture is to provide some background for the experiment involving crystallization of glass-ceramic.
  • Water Quality Standards Human Health Criteria Technical Workgroup

    Water Quality Standards Human Health Criteria Technical Workgroup

    RfD: An estimate, with uncertainty spanning perhaps an order of magnitude, of a daily oral exposure to the human population (including sensitive subgroups) that is likely to be without an appreciable risk of deleterious effects during a lifetime. EPA's policy-...
  • Theories of Reality - Lane Community College

    Theories of Reality - Lane Community College

    Theories of Reality PHL 203 - Winter 2013 The Problem of Evil Prof. Borrowdale * * * * * * * * The problem of evil Inconsistent triad - one of these must be false: God is omnipotent. God is...