Software/Information Systems Center

Software/Information Systems Center

KNOWLEDGE ON DEMAND: Knowledge and Expert Discovery Dr. Mark T. Maybury Executive Director Information Technology Division Knowledge Management Conference Baden Baden, Germany 15 March 2001 Organization: G060 Project: 05AAV061-C1 http://www.mitre.org/resources/centers/it MITRE 2 02/21/20 18:06 Knowledge on Demand Knowledge Management Strategy Knowledge Extraction and Discovery - TIDES, GeoNODE, QANDA, SIAM Expert and Expert Community Discovery - ExpertFinder, XperNET Facilitating Group Knowledge Creation - KEAN, OWL, SCOUT Facilitating Knowledge Communication/Exchange - CVW, TrIM Conclusion - Knowledge Management Lessons Learned - Grand Challenges MITRE 3

02/21/20 18:06 Why KM? Change is Accelerating 18 16 14 12 10 8 6 4 2 0 0 6 12 18 24 30 36 Optical Network Speed doubles every 8 months Storage capacity doubles every 12 months Computing

power doubles every 18 months dot COM storage requirements double every 90 days MITRE 4 02/21/20 18:06 What is KM? An Enterprise Perspective The strategies, processes, and technologies employed to enable an enterprise to acquire, create, share, and make actionable the knowledge needed to achieve mission objectives Enabling Technologies and Processes Influences Leadership CoPs Best Practice DBs Strategy Reward & Recognition KM Process MITRE 5 02/21/20 18:06

Framework MITRE y ess ent Tec hn olo gy Cu lt u re Pro c Co nte nt Po lic ure m t eg y Me as

Str a 02/21/20 18:06 6 KM Enablers MITRE Knowledge-Enabled Outcome States State 0 Where We Were State 1 Fostering Knowledge Development State 2 Harvesting the Benefits Ad Hoc Processes Common KM Understanding Local Initiatives Center Pilots Enterprise

Processes Disparate Views of Resources Consolidated Resource View Knowledge Discovery Low Tool Standardization Greater Tool Standardization Collaboration Valued KnowledgeSharing KM Targets Tool/Process integration Knowledge Creation and Re-use Impact State V Ultimate Vision Embedded KM Known Knowledge Value Pervasive Infrastructure

Innovative Outcomes 8 02/21/20 18:06 APQC Model of Stages of KM Implementation Scale Develop interest and enthusiasm No formal business case; belief in the value Define KM in terms people understand Capitalize on intranet Understand organizational readiness Pilot Path Select pilots or identify grass root efforts Business objectives are specific to pilots Form a crossfunctional KM task force Support

pilots Business case is potential gain from pilots Share pilot lessons learned Develop methodologies that can be replicated Strategic Pilots Opportunistic Pilots up; build capability Business case and measures become more formal KM coordination team Identify roles and resources for the KM function Establish awards and recognition Improve Decision Expand Disengage

KM embedded in business model Organizational alignment Project work with activity and knowledge base support Standards Way of doing business MITRE KM Portfolios of KM Best Practice Companies (APQC, 2000) Elements Central to KM Approach: Intranet, CoP/Networks, Best Practice Publication Resource Communications Collaboration Work Application X X X X World Bank X X

X X X X X X Chevron X X X X Seimens X X X X HP Consulting X X X

MITRE Xerox Resource Tool (Pull) - Yellow Pages, Best Collaboration - Access to knowledgeable human resources Practice DBs, Search Engines Communications - e-mail, Web pages Work Application - Project Management, Problem Solutions, Customer Service 10 02/21/20 18:06 Process Expertise & Knowledge Discovery Knowledge Requirement Customer(s) Knowledge Team Formation KNOWLEDGE INFORMATION INFRASTRUCTURE (KII) Knowledge Delivery Knowledge Creation MITRE

11 02/21/20 18:06 Knowledge on Demand Knowledge Management Strategy Knowledge Extraction and Discovery - TIDES, GeoNODE, QANDA, SIAM Expert and Expert Community Discovery - ExpertFinder, XperNET Facilitating Group Knowledge Creation - KEAN, OWL, SCOUT Facilitating Knowledge Communication/Exchange - CVW, TrIM Conclusion - Knowledge Management Lessons Learned - Grand Challenges MITRE 12 02/21/20 18:06 Vision: Ask Questions, Get Answers Question: Question:Where Whereare arethe the leaders leadersof ofthe theELN? ELN? Multilingual,

Multilingual, Multimedia, Multimedia, Multiparty Multiparty Resources Resources Answer: Answer:Francisco FranciscoGalan Galanand and Felipe FelipeTorres Torresare arein inthe the penitentiary penitentiaryat atItagui, Itagui,Columbia Columbia Today Documents, Documents, Not Notanswers answers Tomorrow Answers Answers& & Drill

Drilldown down MITRE Knowledge Discovery Tools Sources Disseminate/ Retrieve (TIDES, QANDA) Collect Summarize (WebSumm) Extract (Alembic) Finance Energy Trans. Telecomm Z-Ave Translate (CyberTrans) Monitor (SIAM) Collaborate (KEAN, Scout, ExpertFinder, XperNET) Browse/Visualize

(GeoNODE) Cluster/Mine (QueryFlocks) Detect, Translate, Extract, Summarize slat n a Tr ion ction Topic Dete Extraction Sum mar izat io Tamil document Source: Ron Larson (DARPA TIDES) n Today is a significant day in the history of our national liberation struggle, it marks the end of a year during which we have resisted and fought against the biggest ever offensive operation launched by the Sri Lankan armed forces code named "Jayasikuru... Liberation Tigers of Tamil Eelam (LTTE) Sri Lanka Velupillai Pirapaharan Rebellion

Org Sinhala LTTE Leader HQ Kumaratunga Pirapaharan Wanni Losses 3000 1300 The Theobjective objectiveof ofthe theSinhala Sinhalachauvinists chauvinistswas wasto to utilize maximum man power and fire power to utilize maximum man power and fire power to destroy destroythe themilitary militarycapability capabilityof ofthe theLTTE

LTTEand and to tobring bringan anend endto tothe theTamil Tamilfreedom freedommovement. movement. Before the launching of the operation Before the launching of the operation "Jayasikuru" "Jayasikuru"the theSri SriLankan Lankanpolitical politicaland andmilitary military high command miscalculated the military high command miscalculated the military strength strengthand anddetermination determinationof ofthe

theLTTE. LTTE. DARPA TIDES News Repository Translingual Information Detection Extraction Summarization Interaction CDC WHO ~ 2500 stories/day Email: ProMed Internl News Sources CATALYST CATALYST Capture Medical literature Event Extraction Time Tagging TDT Translation Summarization Alerting

Change detection Cross-language IR What is the status of the current Ebola outbreak? The epidemic is contained; as of 12/22/00, there were 421 cases with 162 deaths Ebola hemorr hagic fever - Uganda Disease Location Unfiltered 10 99 Outbreak 0 105 Cholera 1 57 Dengue Fever 0 10 Ebola 2 34 Unidentif ied hemorrhagic f UNK Unidentif ied hemorrhagic f UNK Ebola hemorrhagic fever - Ebola Ebola hemorrhagic fever in Ebola Ebola hemorrhagic fever in Ebola Re: Ebola hemorrhagi... Ebola Re: Ebola hemorrhagi...Rabies

Re: Ebola hemorrhagi...Rabies Uganda Uganda Uganda Kenya Uganda Uganda Source Date ProMED IHT ProMED WHO ProMED Annotator Jane Analyst Joe Analyst 10/14/00 23:06 10/15/00 10:50 10/16/00 21:45 10/17/00 19:12 10/17/00 19:37 10/17/00 20:42 10/18/00 7:42 10/18/00 12:34 Date Priority Status High Normal

High High High Normal Normal High read read read read read replied unread unread Infrastructure 0 50 Natural Disas... 1 1 Spi lls 0 25 Accidents 5 200 WMD Trackin... 0 45 Sus picious Illn... 0 0 Sus picious De... 0 0 Pos sible Biolo... 0 0 Pathogen threa 0 0 ---------------------------Workspace 0 6 Ebola 0 32

Drafts 0 3 Reports 0 1 Date: Disease: Descriptor: Location: Disease Date: Hospital: New cases: Total cases: Total dead: 10/16/00 Ebola hemorrhagic fever Uganda 10/14/00 missionary hospital in Gulu at least 7 51 31 Http://ti des2000.mitre.org/ ProMED/10162000/34n390h.html Ugandan Ministry identifies Ebola virus as the caus e of the outbreak. KAMPALA: The dreaded Ebola virus that struck over 300 peopl e i n Kikwit, in the Democratic Republic of Congo in 1995, has killed 31 peopl e in northern Uganda. A Ugandan Ministry of Health statem ent said laboratory tests had revealed that the Ebola virus was the cause of the epidemic hemorrhagi c fever which has been raging in the Gulu district since September. T hree of the dead were student nurses , who treated the first

Ebola patients admitt ed to a Lacor mis sionary hospital in Gulu town. A task force headed by Gul u district administrator, Walter Ochora, has been set up to co-ordinate efforts to control the epidemic. Field officials in Gulu told the Kampala-based New 02/21/20 18:06 TIDES Portal Translingual Information Detection Extraction Summarization Metaqueries supported for multiple sources Translingual system supports foreign-language sources Multiple media exploited Government & private sources utilized 16 MITRE 16 17 02/21/20 18:06 Geospatial News on Demand (GeoNode) Topic Timeline News Sources:

Broadcast World Wide Web Intel. Msg Traffic Specialist Archives News histogram GeoNODE Database Data Acquisition/ Pre-process Map overview Navigate Filter Indexed access Animate reporting trends Create reports/ web BNN Story skim Information Extraction Data Mining Indexing And And News Clustering Modeling topic t t

MITRE 18 02/21/20 18:06 Data Mining: Find Significant Warlords in a Region 0 Two-way associations: people and locations Type Person Person Person Person Person Person Person Person Person Person Value Natalie Allen Leon Harris Ron Goldman Dole Forbes Forbes Bill Cosby Pat Buchanan Steve Forbes Mobutu Sese Seko Type Person Person Person

Person Person Organization Location Person Organization Location Value Linden Soles Joie Chen Nicole Brown Simpson Bob Dole Dole New Hampshire Cosby Forbes New Hampshire Kinshasa Support 117 53 19 18 16 15 14 12 12 10 0 Three-way associations, only one association with support of at least 5 (confidence is 50%: 1/2 of the stories mentioning any item

also mention the other two) Person1 Person2 Location Support Mobutu Sese Seko Laurent Kabila Kinshasa 7 6612 stories, with 13,737 distinct concepts mentioned. The associations between pairs of concepts are ranked by support: the number of documents containing the pair). All correlations have at least 50% confidence: At least 50% of the stories mentioning one item in a pair also mention the other MITRE 19 02/21/20 18:06 The Web Has Gone Multilingual Last year, the web became more than 50% non-English Only 15% of Europe's half a billion population speaks English as a first language Only 28% speaks English at all Only 32% of Web surfers on the European continent consult the Web in English. [Source: Global Reach:www.euromktg.com/eng/GR/] 45% of Internet users from non English-speaking countries

By 2002, analysts estimate that 66% of Internet use and 40% of e-commerce revenue will come from outside the U.S. [Source: IDC] 300,000 Japanese patents filed annually "If I'm selling to you, I speak your language. If I'm buying, dann mssen Sie Deutsch sprechen Willy Brandt, former German chancellor MITRE 20 02/21/20 18:06 Open Source Analysis of Latin America Event Timeline 10Mar98 Pinochet resigns 17Mar98 Cuban defector, pitcher Orlando Hernandez 15Apr98 Execution of Paraguayan Angel Breard, convicted killer in US 21Apr98 Plane crash: Bogota, Columbia to France 17Oct98 - 27Oct98 Pinochets arrest by Scotland yard while getting medical treatment 26Oct98 House of Lords deny Pinochet diplomatic immunity 28Nov98 Columbian man surrenders near Bogota, accused of shooting DEA Agent Moreno 04Dec98 Iranian Woman has been charged by Argentina's Supreme Court for the 1992 bombing of Israeli embassy in Buenos Aires, Argentina MITRE 07Jan99 - 18Jan99 Brazilian Financial crisis

21 02/21/20 18:06 Question and Answering - QANDA Doc collection or the Web Semi-structured database Relational Database Knowledge sources Q and A Engine MII IRS hotline GeoNODE operator Multiple Knowledge Sources and Multiple Applications User BNN applications MITRE Social Indicators Analysis Method (SIAM) Issue Concept + Information Volume + Time + Location + Source => Social Interest

Concept mapped to query sets (large numbers of queries) to address topic specificity and coverage. Query Class, Retrieval Rank and/or Relevance can be used to weight items Windowing collection to specific time periods e.g., 4 quarters per year Search agent collects from multiple search engines, link traversal, and sampling strategies used to collect relevant items, and scale up results to population levels. Normalized Volume-->Importance Source weighting (Optional) for weighting importance based on source type and website structure, e.g., logical location within a Gov. site Spatial tags for tracking by country or region Social Interest of Topic X Country Y = F(relevance,scaled- volume,location, time, source) Todays Manual Approach Gartner Group: Year 2000 World Status, 2Q99: The Final Countdown

Other Gartner Reports (periodic assessments, Gartner Interactive Reports,) Lou Marcoccio Interview Y2K Assessments are Generally SurveyBased Example: Gartner Group Survey: 330 Questions Performed Quarterly 600 People Involved U.S. Senate : Investigating The Year 2000 SIAM Processing Costs 1500+ Companies Problem: The 100 Day Report New Domain: 1 Day 3 Calls/Company Department of Quarterly State: Biannual Consular2 Days Analysis: 11 Universities Provide Analysis Advisory Download Time: 1-2 Days for Current Data COMPARE Statistic Computed from Survey Results Analysis and Fine tuning: 2 Days MITRE Y2K Assessment US Agency for International Development (USAID) International Y2K Cooperation Center (IY2KCC) International Monitoring (IM) a Londonbased consultancy Example: Department of State has an ongoing Survey among its 260 Posts.

Reporting Bias Observed by Gartner...others. SIAM Status Board High Risk Inferred Moderate Risk Inferred Low Risk Inferred CNTRY Finance argentina 1.476278 bermuda 0.888375 bolivia -0.65204 brazil 1.869995 chile 1.242407 colombia -0.5338 costa_rica -0.39694 ecuador -0.6348 el_salvador -0.6304 guatemala -0.65019 guyana -0.65496 honduras -0.65361 mexico 1.656809 nicaragua -0.60775 panama

-0.64749 paraguay -0.64199 peru -0.42739 puerto_rico -0.65496 uruguay 0.987168 venezuela -0.33471 Energy Trans. Telecomm Z-Ave 2.434932 0.8366 1.9553 1.67579 -0.06594 1.9661 0.1754 0.740966 -0.55287 -0.6155 -0.6380 -0.61458 0.27763 0.7220 1.5664 1.109025 0.065475 1.9946 0.8680 1.042629 -0.38638 -0.4981 -0.5096 -0.48196 -0.29373 -0.4429 -0.2979 -0.35789 -0.54902 -0.6117 -0.6289 -0.60608

-0.52669 -0.5945 -0.6204 -0.59299 -0.54851 -0.6130 -0.6366 -0.61208 -0.55555 -0.6146 -0.6410 -0.61652 -0.55464 -0.6161 -0.6409 -0.61632 2.306324 1.1496 2.0101 1.780716 -0.52549 -0.5961 -0.6028 -0.58303 -0.54845 -0.6090 -0.6352 -0.61004 -0.54268 -0.6052 -0.6190 -0.60222 -0.31945 -0.4733 -0.3972 -0.40431 -0.55555 -0.6162 -0.6401 -0.6167 1.667379 1.2754 1.2915 1.305344 -0.2268 -0.4382 -0.3592 -0.33974

MEXICO BELIZE GUATEMALA HONDURAS EL SALVADOR NICARAGUA COSTA RICA PANAMA VENEZUELA COLOMBIA ECUADOR BRAZIL PERU BOLIVIA PARAGUAY CHILE URUGUA ARGENTINA 25 02/21/20 18:06 Variance of Opinion Senate Report: Peru is, across many sectors, either not Y2k ready or that public information is inadequate. The Gartner Group and the World Bank offer contradictory information, ranking Peru as one of the better prepared in South America. mexico nicaragua panama paraguay peru

SIAM Indicator Board 1.656809 2.306324 1.1496 -0.60775 -0.52549 -0.5961 -0.64749 -0.54845 -0.6090 -0.64199 -0.54268 -0.6052 -0.42739 -0.31945 -0.4733 2.0101 -0.6028 -0.6352 -0.6190 -0.3972 SIAM Scores Peru as Moderate Readiness Ranking 10th out of 20 Countries MITRE 26 02/21/20 18:06 Knowledge on Demand Knowledge Management Strategy Knowledge Extraction and Discovery - TIDES, GeoNODE, QANDA, SIAM Expert and Expert Community Discovery - ExpertFinder, XperNET Facilitating Group Knowledge Creation - KEAN, OWL, SCOUT Facilitating Knowledge Communication/Exchange - CVW, TrIM Conclusion

- Knowledge Management Lessons Learned - Grand Challenges MITRE 27 02/21/20 18:06 Expert Discovery Find global Experts - quick - accurate - comprehensive Challenge: Overcome limitations of manually managed skills/expertise databases (e.g. Dataware - experts self nominate) - incomplete - expensive - out of date MITRE 28 02/21/20 18:06 Related Work Autonomy - document based (docs, Notes discussions, email) - dynamic expert profiling - Problem: reading/writing not always correlated w/expertise Abuzz - Beehive email routes questions to experts based on expert profile (must seed this)

- Expertise validated by community (+/- satisfaction with answers) updates profiles - Problem: Seeding/Learning curve MITs ExpertFinder (Vivacqua) - expertise models from software library use Tacit - email based keyword profiling MITRE 29 02/21/20 18:06 Uniqueness This work: - Implicitly determines expertise from multiple sources of evidence including intellectual products (e.g., briefings, papers, web pages) and information seeking actions (e.g., web logs) - Leverages intranet publishing (staff, corporate news letters), corporate directory services, project leadership information - Exploits recent advances in information extraction (language processing) technology MITRE Expertise Management Architecture Resources Services E-dB Finder Service Broker Selection

Qualification Registration Q&A MII WWW Finder Agencies Consulting Groups Expert Goal: Place a user within one phone call of an expert Finder User Issues Simple Query Integrated Employee Database Employees Ranked by Mentions Mentions of Employee in Corporate Communications Relevant Employee Publications

Enterprise Employee Project Database 32 02/21/20 18:06 ExpertFinder Algorithm Initial Form Gather All URLs Weigh Evidence yes Cached Results? no Find Mentioned People Add Phone Book Info Find Published People Call Search Engines Combine Info Display Results Parse Results MITRE 33 02/21/20 18:06 Evaluation

Compare performance of ExpertFinder with (20) expert human resource managers Task: Find top 5 corporate experts in a given domain Measures - Agreement among humans - Agreement of machine with human(s) Precision Recall Chance: # experts/4500 employees = often less than .1% MITRE 34 02/21/20 18:06 The Questionnaire I am performing an experiment. Your participation will remain anonymous if you so desire and should only take a few short minutes. Please answer the following questions (preferably without any assistance, but if you use assistance indicate what kind you used): 1. Who are the top 5 "data mining" experts at MITRE (List them in rank order, most expert first. List as many as you can but no more than 5)? 2. the top 5 "collaboration" experts? 3. the top 5 "chemical" experts? 4. the top 5 "human computer interaction" experts? 5. the top 5 "network security" experts? 6. What is your top area of expertise (in a few words) and who do you consider to be the top 5 people in the company in your area of expertise? MITRE 35

02/21/20 18:06 Human vs ExpertFinder Comparison to 20 human resource managers Agreement = in top 5 IDed experts Precision = # correctly IDed experts / # IDed Recall = # correctly IDed experts/ # actual experts Expert Area Human Agreement (1st, 2nd, 3rd) ExpFinder Precision ExpFinder Recall Data mining 70%, 49%, 24.5% 60% 40% Chemical 40%, 8%, 0.8% 60% 40% HCI 90%, 36%, 11%

60% 40% Network Security 50%, 10%, 0.4% 20% 20% Collaboration 70%, 35%, 17.5% 5% 5% AVERAGE 63%, 28%, 11% 41% 29% MITRE 36 02/21/20 18:06 Community of Interest Modeling Organizational Theory Pattern Analysis Work/Activity Sampling Feature Extraction

Topic Detection Social Network Generation Community of Practice Registration Project Information Clustering techniques Web Pages Social network analysis methods Meetings/Conferences/... Summarization Share Folders Published documents... Emergence Monitoring Communication Sharing Expert Finding Other Applications MITRE 37 02/21/20 18:06 Expert Communities: XperNet Network 31 29 27 25 23

21 19 17 15 13 11 9 7 5 3 100.000 90.000 80.000 70.000 60.000 50.000 40.000 30.000 20.000 10.000 0.000 1 Membership Score Network Membership Ratings Member Rank

Automatic Network Expansion Core Group Expanded Group MITRE 38 02/21/20 18:06 Extracting Communities of Interest MITRE 39 02/21/20 18:06 Knowledge on Demand Knowledge Management Strategy Knowledge Extraction and Discovery - TIDES, GeoNODE, QANDA, SIAM Expert and Expert Community Discovery - ExpertFinder, XperNET Facilitating Group Knowledge Creation - KEAN, OWL, SCOUT Facilitating Knowledge Communication/Exchange - CVW, TrIM Conclusion - Knowledge Management Lessons Learned - Grand Challenges MITRE

40 02/21/20 18:06 Words of Wisdom Bentovs Law: Ones level of ignorance increases exponentially with accumulated knowledge. When one acquires a bit of new information, there are many new questions that are generated by it, and each new piece of information breeds five-ten new questions. These questions pile up at a much faster rate than does accumulated knowledge. Therefore, the more one knows, the greater his level of ignorance. Allens Tenet - The strengths of ones opinion on any matter or controversy is inversely proportional to the amount of knowledge that person has on that subject. BBs Dictum - In a group, the unknowing will try to teach the lesser-skilled or knowing MITRE 41 02/21/20 18:06 Collaboration Taxonomy Joint Efforts Coordination g sin d ea a

cr he In ver O In In cre ter as a c ing tio n Alignment Leadership Intent Information Sharing Awareness Multiple Multiplelevels levelsof ofvirtual virtualteaming teaming MITRE 42 02/21/20 18:06 Knowledge Exchange and Annotation eNgine (KEAN): Search SEARCH by - Subject - Keyword

- Employee - Rating Level - Time boykin MITRE 43 02/21/20 18:06 KEAN: Source Assessment MITRE 44 02/21/20 18:06 Questions answered (with justification) by KEAN (e.g., data mining) What information does Chris (expert in data mining) think is useful for data mining? What information do people in the data mining community of practice find useful on data mining? What information does everyone think is useful on data mining in the past few weeks? What information on data mining have I found to be useful in the past? MITRE 45 02/21/20 18:06

KEAN Evaluation 26 individuals on 295 URLs - length of time on page (reading time) - explicit utility rating Focused task - directory services questions - Which standards organization defines the X.500 specification? - How does LDAP differ from X.500? - Name some of the data types that can be stored in an LDAP attribute. After the experiment, rate utility 1-10 (10 highest) Regression test yielded positive correlation - explicit utility = .0113*time read - 66% of all URLs read for greater than 78 seconds were classified as high utility (6-10) Time --> utility MITRE 46 02/21/20 18:06 Organizational Wide Learning (OWL): Word Command Usage by Type Command Usage by Type Edit File Format View Window Insert Tools Table Help 0%

10% 20% 30% 40% 50% 60% MITRE 47 02/21/20 18:06 OWL Data: Words Top 10 Commands Sequence 1 2 3 4 5 6 7 8 9 10 Command Edit Delete File Save File Open Edit Paste File DocClose Edit Copy Format Bold

File Print Edit Cut File SaveAs Percent 34.2% 10.5% 8.7% 7.9% 5.1% 4.2% 3.7% 2.8% 2.4% 1.7% Cumulative Percent 34.2% 44.8% 53.5% 61.4% 66.5% 70.7% 74.4% 77.2% 79.7% 81.3% MITRE 48 Organizational Knowledge & Ignorance Some individuals never use a number of the more frequently-used commands Count of Users & Usage Users

Usage 100000 20 10000 15 1000 10 100 5 10 49 47 45 43 41 39 37 35 33 31 29 27

25 23 21 19 17 15 13 11 9 7 5 1 3 0 1 Users 02/21/20 18:06 Command Sequence MITRE 49

02/21/20 18:06 OWL recommends what to learn next (unique to each individual at each point in time) USER #314 ExpectedObservedInstruction Edit Paste 170 274 OK Edit Delete 129 0 New Edit Copy 107 97 OK Edit Cut 48 100 OK Edit Undo 16 14 OK Edit Find 12 1 More Edit SelectAll 9 12 OK Edit DeleteWord 4 0 New Edit Replace 3

0 New Edit PasteSpecial 2 0 New MITRE 50 02/21/20 18:06 Do OWL Users Use Help? Use of MS Word Help & Office Assistant Office Assistant More than half (12/20) use help at least monthly. Help 16 14 Only a few (6/20) use Office Assistant Number of Users 12 10 8 6 4 2 0 Daily

Weekly Monthly Annually Never Frequency of Use MITRE 51 02/21/20 18:06 Cooperative Searching (Monitoring Group Information Seeking Activities) -- a multi-user collaborative retrieval tool. The next generation in IR systems addresses multi-user, coordinated searching, shared analysis, and has a built-in recommender system. Tracks topics, users, and provides a persistent knowledge store. MITRE 52 02/21/20 18:06 Cooperative Searching Hypothesis Group (coordinated) searching can be more effective than multiple (independent) searchers working autonomously

MITRE 53 02/21/20 18:06 Cooperative Searching-- Initial Prototype Local Shared Workspace Collect UsersGenerate GenerateTask AdHoc Hoc Queries Users Folders Ad Queries Users Generate Task Folders Off-lineQueries Queriesor orWeb-page Web-pageMonitoring MonitoringSupported Supported Off-line MITRE 54 02/21/20 18:06

Cooperative Searching Local Shared Workspace Collect RetrievedInformation InformationisisOrganized Organizedby byDomain Domain Retrieved SearchEngine EngineStatistics StatisticsProvided Provided Search OfflineCluster ClusterAnalysis Analysisand andCategorization CategorizationProvided Provided Offline MITRE 55 02/21/20 18:06 Cooperative Searching Access Count and Evaluation Status Provided Local Shared Workspace Collect UserActions ActionsInfer InferRelevance Relevance

User Ratings/Annotations/ActionsAre AreStored Stored Ratings/Annotations/Actions MITRE 56 02/21/20 18:06 Cooperative Searching Local Shared Workspace Collect RatedItems Itemswith withAnnotations Annotationsare are Rated integratedinto intoaaShared SharedContext Context integrated MITRE 57 02/21/20 18:06 Knowledge on Demand Knowledge Management Strategy Knowledge Extraction and Discovery

- TIDES, GeoNODE, QANDA, SIAM Expert and Expert Community Discovery - ExpertFinder, XperNET Facilitating Group Knowledge Creation - KEAN, OWL, SCOUT Facilitating Knowledge Communication/Exchange - CVW, TrIM Conclusion - Knowledge Management Lessons Learned - Grand Challenges MITRE 58 02/21/20 18:06 Collaborative Virtual Workplace (cvw.sourceforge.net) MITRE 59 02/21/20 18:06 Translingual Instant Messaging (TrIM) Integration of - Simple Instant Messaging Protocol (simp.mitre.org) - CyberTrans machine translation framework Supports multilingual chat Research Issues: - Quality of conversational translation using document translation engines - Presentation (monolingual, multilingual) - Data collection to learn language models

MITRE 60 02/21/20 18:06 Knowledge on Demand Knowledge Management Strategy Knowledge Extraction and Discovery - TIDES, GeoNODE, QANDA, SIAM Expert and Expert Community Discovery - ExpertFinder, XperNET Facilitating Group Knowledge Creation - KEAN, OWL, SCOUT Facilitating Knowledge Communication/Exchange - CVW, TrIM Conclusion - Knowledge Management Lessons Learned - Grand Challenges MITRE Knowledge Management Capability Maturity Model (KM CMM) Level Level5:5:Optimizing Optimizing Business Businessprocess processalignment alignment Process change management Process change management Level Level4:4:Managed

Managed Integrated Integratedknowledge knowledgeprocesses processes Quantitative Quantitativeprocess processmanagement management Where we are going 01 Level 3: Defined Organizational processes Knowledge mapping Intergroup coordination Training program Level 2: Repeatable Program planning Requirements process Level 1: Initial Adhoc processes Partial technical infrastructure Level 0:Not Practiced Failure to perform KM Culture counter to learning, sharing Content QA process KFP identification Where we are 00 Where we want to be

62 02/21/20 18:06 Strategy Peter Senge Learning Organizations Process Takeuchi and Nonaka Organizational Knowledge Creation Benchmarking Norton and Kaplan Balanced Scorecard CIIS CENTERFORINTEGRA TED INTELLIGENCE SYSTEMS MITRE MITRE 63 02/21/20 18:06 Lessons Learned People, and the cultures that influence their behaviors, are the single most critical resource for successful knowledge creation, dissemination, and application. Understand and influence them. Cognitive, social, and organizational learning processes are essential to the success of a knowledge management

strategy. Focus your strategy on enhancing these processes. Measurement, benchmarking, and incentives are essential to accelerate the learning process and to drive cultural change. Create a tailored balanced scorecard to target what you want to improve. Knowledge management programs can yield impressive benefits to individuals and organizations if they are purposeful, concrete, and action-oriented. Make yours so. MITRE In times of profound change, learners inherit the Earth, while the learned find themselves beautifully equipped to deal with a world that no longer exists. - Al Rogers - 65 02/21/20 18:06 Some Grand Challenges User, Group and Organization Modeling, including knowledge, beliefs, goals and plans (U, G, O) Tailored presentation of knowledge Ontological integration of distributed DB & KB Universal knowledge access independent of user physical, perceptual, cognitive, cultural characteristics Organizational strategies for knowledge sharing Knowledge strategies in global, multicultural enterprises Privacy and Security MITRE

66 02/21/20 18:06 Events of Interest 8th Internat. Conference on User Modeling Sonthofen, Germany July 13-17, 2001 www.dfki.uni-sb.de/cgi-bin/um2001 Joint EACL - ACL Meeting Workshop on Human Language Technology and Knowledge Management Toulouse, France July 6-7, 2001 www.elsnet.org/acl2001-hlt+km.html Our work: www.mitre.org/resources/centers/it MITRE 67 02/21/20 18:06 Acknowledgements Knowledge Management - Cynthia Small, Jean Tatlias TIDES - Lynette Hirschman, Jay Ponte et al. GeoNODE - Rod Holland, John Griffith et al. QANDA - Marc Light OWL - Frank Linton CVW - Jay Carlson, Deb Ercolini et al. TrIM - Rod Holland, John Ramsdell, Flo Reeder, Jay Carlson, Justin Richer, Galen Williamson, Michael Krutsch,

Keith Crouch, Keith Miller SIAM, XperNET, SCOUT - Ray DAmore, Manu Konchady KEAN - Daryl Morey, Tim Frangioso ExpertFinder - Dave Mattox, Inderjeet Mani, David House MITRE KNOWLEDGE ON DEMAND: Knowledge and Expert Discovery Dr. Mark T. Maybury Executive Director Information Technology Division Knowledge Management Conference Baden Baden, Germany 15 March 2001 Organization: G060 Project: 05AAV061-C1 http://www.mitre.org/resources/centers/it MITRE

Recently Viewed Presentations

  • Christianity and World Religions

    Christianity and World Religions

    Summary The theologies of world religions are fundamentally and diametrically opposed to one another. Many paths to the same God is a ludicrous philosophy. The world view of Christianity is radically different from any other religion.
  • Do I do What is Needed To Live On My Own?

    Do I do What is Needed To Live On My Own?

    TOOLS FOR IMPROVING THE QUALITY OF YOUR LIFE Pinellas Hope Life Skills Training Program #6 Plate 4: The New American Plate A one-dish dinner like stir-fry is just the kind of meal that belongs on the New American Plate It's...
  • Receptor-drug interaction - Philadelphia University

    Receptor-drug interaction - Philadelphia University

    Receptor-drug interaction. The driving force for drug-receptor interaction is the low energy state of the drug-receptor complex. The biological activity is related to the drug affinity for the receptor, i.e the stability of the complex. Dissociation constant of the drug-receptor...
  • CURRENT ASSETS EMBA - 2005 Economic Consequences of

    CURRENT ASSETS EMBA - 2005 Economic Consequences of

    Factoring-without recourse-Factor company entries Factoring-with recourse-Factor entries Factoring with recourse-payment date Assume none of the customers take sales discount and by 15 December 2005 Firm Factoring collects all accounts receivable and pays Fashion Giyim Sanayi the remaining amount.
  • Sonia Lee Tera Doak Director, Affiliate Financial Services

    Sonia Lee Tera Doak Director, Affiliate Financial Services

    Policies regarding Sale of Housing Unit (Pricing) and Mortgage Origination. What is a "loan application" under RESPA . What happens when an affiliate has a completed RESPA "loan application" ... both in their origination and servicing, are of the highest...
  • TigerSHARC and Blackfin.ppt

    TigerSHARC and Blackfin.ppt

    TigerSHARC and Blackfin Different Applications
  • Välkommen till Unix Grundkurs

    Välkommen till Unix Grundkurs

    Olika X-motorer finns: Twm tabbed windows manager (modern) Mwm motif windows manager (snyggast av alla) Gnome Fri variant (standard) Kde Fri variant CDE Common Desktop Envrioment (standard) OpenWin Solaris, snygg men överkörd av Gnome. Det finns lika många terminaltyper som...
  • Ray Tracing Outline - inst.eecs.berkeley.edu

    Ray Tracing Outline - inst.eecs.berkeley.edu

    Octrees Figure out which cells it intersects, and check for intersections only with objects inside those cells More features Here are some suggestions: Transparency with refraction Anti-aliasing Lens effects / depth of field Super quadrics Programmable shading Texture, bump, and/or...