Design and Implementation of Online Braille Math Support in the Accessible Equation Editor Sam Dooley Pearson Assessment Dan Brown Pearson Assessment Susan Osterhaus Texas School for the Blind and Visually Impaired Braille math is hard! Blind students need high-quality braille math Advanced preparation is costly and takes time Math teachers are often unfamiliar with braille
2 TVIs are often unfamiliar with math notation Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille math software is harder! 3 Software translation can be problematic Forward translation takes many steps Real-time back translation is non-existent
No support for online or classroom use Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille math needs to be online! Blind students need: 4 A level playing field for STEM instruction To read and write online braille math To interact with sighted instructors and peers
To participate in online activities Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille math should be math! Math concepts are independent of notation Braille math codes capture all math notation Math software can be independent of notation Blind students only have full access to math if their math is treated the same as printed math.
5 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Classroom challenges for blind math students 6 They cannot read math the teacher can write They cannot write math the teacher can read Speech solutions incur high cognitive load
They need to read and write for themselves Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille math translation Visual math software is not truly accessible MS Word, MathType, Scientific Notebook Braille math translation software is not instantaneous is not totally accurate
LaTeX, Duxbury, HIMS, HumanWare 7 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Online Braille Math with the Accessible Equation Editor 8 Sighted user can create math for a blind user Blind user can create math for a sighted user Real-time, two-way braille math translation
Instantaneous interactions with math content Discoverable braille math encoding rules Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Nemeth Braille (1952) 9 Abraham Nemeth (AFB, APH, BANA) Tactile encoding for print math
Math for technical publications Presentational math structures Consistent with content markup Design and Implementation of the Accessible Equation Editor 10 Apr 2016 TeX/LaTeX (1978/1985) Donald Knuth, Leslie Lamport Typesetting language for print math
Math for technical publications Presentational math structures Fully programmable macro language 10 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 W3C MathML (1998) World Wide Web Consortium (W3C) XML element/attribute vocabulary
Typical K-12 and higher ed. math Presentation and content markup 11 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Presentation v. Content Presentation encodes signs/symbols Content encodes functional structure "x^2+1" v. "1346, 45, 23, 5, 346, 2"
"x^2+1" v. (plus (power x 2) 1) 12 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 QWERTY keyboard input Math symbols on the keyboard Other symbols on the buttons Keyboard cursor navigation
Implicit and explicit selection Backspace, delete, clear 13 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille terminal output Math output on the screen Braille output on the screen Braille output on the terminal
Screen reader support 14 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille terminal input Each braille key is treated like a keyboard key Sequences of braille keys create the math Button tooltips with text and braille Input cursor position and routing
15 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 QWERTY terminal output Braille input creates math content Math content becomes math output Math output becomes braille output The math markup is exactly the same 16 Design and Implementation of the Accessible Equation Editor 10 Apr 2016
Braille math is math! Blind students can read the same math Blind students can create the same math The math can be shared the same way The math can be scored the same way Blind students can now have full access to math since their math is the same as printed math. 17 Design and Implementation of the Accessible Equation Editor 10 Apr 2016
Equation Editor Architecture Syntax expression trees Expression tree transformations Configuration and customization Universal design decision points 18 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Expression trees
An operator with its child arguments The operator as the initial child argument Parse trees, syntax expressions, XML elements Often annotated with semantic attributes Functional programming languages (Lisp) 19 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Content expressions
The functional structure of a math expression Does not itself define any semantic behavior The operators carry the semantic behaviors Operator, child elements, child attributes Direct translations to and from XML markup Boxed reference to presentation expression
20 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Presentation expressions The visual notation for a math expression Does not itself define functional structure The operators carry the notational schema Operator, child elements, child attributes Direct translations to and from XML markup
Boxed reference to content expression 21 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Expression references Math document user interface elements Content node matched with a presentation Pointers to child expression references
References to the browser display nodes Resolves from DOM to presentation to content 22 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Browser display nodes Browser document user interface elements Direct references into the browser DOM tree Attached pointers into content, presentation
Listeners for math content change events Wrap nodes not owned by the equation editor 23 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Expression trees are universal! Generic encodings for math formulas Separating notation from function Separating interface from function
Enabling automated regression testing Universal design for math behaviors 24 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Transformation architecture Input transformations create expressions Output transformations create XML markup Layouts transform content to presentation
Templates create new content from old Generic rewrite rules on expression trees Enabling data-driven math behaviors 25 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Configuration architecture External data defines conventional behaviors
Allows highly customizable math interactions Fine control for solving math usability issues Allows reconfiguration for new populations Layouts, templates, buttons, palettes, keys 26 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Layout transformations Transform structure into notation
Structure encoded as Content MathML Notation encoded as ... pick your target! Layout rules encoded as custom XML Source pattern as Content MathML Output pattern as layout schemata 27 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Content to Presentation
Math operators determine layout rules Minimal contextual dependencies Presentation links back to content (plus x 1) v. "x + 1" 28 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Content to Braille
Math operators determine braille symbols Additional contextual dependencies Flattening to linear output format (plus x 1) v. "1346, 346, 2" ⠬ 29 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Template transformations
Create new structure from old structure Template rules encoded as custom XML Template names and key event names Source pattern as Content MathML Output pattern as Content MathML Template variables control rewriting
30 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Keyboard to Content Keys and button names select template rules Minimal sequential dependencies Presentation updated on each key "x + 1" v. (plus x 1)
31 Content MathML Markup from Nemeth Braille Input 23 Mar 2016 Braille to Content Braille cells indirectly select template rules Non-trivial sequential dependencies Presentation updated on each key "1346, 346, 2" v. (plus x 1)
32 Content MathML Markup from Nemeth Braille Input 23 Mar 2016 Braille key transformations Transform braille cells to template names Braille cells are received as ASCII braille Each braille cell assigned a symbolic name Symbolic names for braille cell sequences
Braille input mode transforms to key names Previous template rules operate unchanged 33 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille cells to keys Single cells that appear as keys Single cells that cause actions Sequences that cause actions
Sequences that extend others 34 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille finite state machine Symbolic names for braille prefixes Symbolic names identify input states
Braille cells identify state transitions Accepting input states invoke templates Subsequent states may replace inputs 35 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Universal design for online math Expression trees provide universal data Data-driven universal tree transformations
Flexible configuration customization rules Universal presentation output notation Universal input event transformations 36 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille Input Examples Numeric indicator Baseline indicator
Type form indicators Shape indicators Negated operators Composed relations 37 Content MathML Markup from Nemeth Braille Input 23 Mar 2016 Braille Input Testing Web browser input test page
JavaScript equation editor Screen reader device drivers Web accessibility APIs Braille terminal input 38 Content MathML Markup from Nemeth Braille Input 23 Mar 2016 Research Studies Two research studies in Fall 2015
(KY/AZ - Sep, TX - Oct) Goal - to collect feedback from multiple populations on EE functionality Populations - blind, low vision, regular print readers, learning disabled Criteria - high school students who had completed Algebra I 39 Content MathML Markup from Nemeth Braille Input 23 Mar 2016 Research Outcomes
Students had limited knowledge of Nemeth Erasing math content was unpredictable Working with grouping symbols was difficult Entering and closing fractions was unexpected Ending trigonometric expressions was unusual 40 Content MathML Markup from Nemeth Braille Input 23 Mar 2016 Advanced preparation
Create math content in advance Share with students by email Include math content with slides Students use the equation editor 41 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Extemporaneous content Teacher uses the equation editor
Projects the screen to the class Blind student uses braille terminal Everyone can read the same content 42 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Shared presentations WebEx, Google Hangouts, etc.
Shared screens for visual users Individual work shown to the class Blind students can show their work Blind students viewing others work 43 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Online assessments Common Core, PARCC, TestNav
Spring 2015 admin (15m responses) Machine scoring for constructed math Blind students can use online forms Blind students create online responses 44 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Nemeth Braille curriculum Interactive Nemeth braille exercises
Immediate practice for lessons learned Experimentation with braille concepts Discoverability for unfamiliar encodings 45 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Homework assignments Blind student uses the equation editor Email homework files to sighted teacher
Teacher reads using the equation editor Teacher marks up the math content Returns marks to the blind student 46 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 TVI support The role of the TVI remains essential
The technical work load can be reduced The software can help assist the process Discoverability for print math notations 47 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Math teachers The role of the math teacher is vital The technical work load can be reduced
Engaging directly with blind students Discoverability for braille math encodings 48 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Group interactions Effective group work with blind students Each student has an opportunity to contribute Each can learn from the work of others
The software can help assist the process 49 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Online Accessible Math! Blind students can have: A level playing field for STEM instruction Tools to read and write online braille math Interaction with sighted instructors and peers
Active participation in online activities 50 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Expression trees are universal! Generic encodings for math formulas Separating notation from function Separating interface from function Enabling automated regression testing
Universal design for math behaviors 51 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille math is math! Blind students can read the same math Blind students can create the same math The math can be shared the same way The math can be scored the same way
Blind students can now have full access to math since their math is the same as printed math. 52 Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Accessible Equation Editor Try it out, give us feedback http://accessibility.pearson.com/mathex-app/ Please let us know how you would use it [email protected] [email protected] [email protected] 53 Design and Implementation of the Accessible Equation Editor 10 Apr 2016
