Effects of Surface Characteristics on Alignment of Real

Effects of Surface Characteristics on Alignment of Real

Effects of Surface Characteristics on Alignment of Real and Graphic Objects in Stereoscopic Augmented Reality Environments Ming Hou Ergonomics in Teleoperation and Control (ETC) Laboratory Department of Mechanical and Industrial Engineering University of Toronto January 6th, 2003 3D Measurement in Unmodelled World Virtual Tape Measure Problem: Unknown relationship between real and virtual objects in Augmented

Reality displays Research on Real World Targets Virtual Pointer LINE Video Image Cylinder AREA VOLUME Hemisphere Pseudo-Transparency Phenomenon Real Object (in Video) Virtual pointer behind real surface Virtual pointer in front of surface

Transparency Effect Breakdown of Fusion + Fusion Conflict Conflict between Binocular Disparity and Occlusion Cues Virtual pointer behind real surface Virtual pointer in front of surface Fuse Real Surface Fuse Virtual Pointer Theory of Surface Interaction: Fusion Breakdown depends on Texture Density Virtual pointer behind real surface Virtual pointer in front of surface Low Texture Density High Texture Density

Research Motivation Is this conflict really significant? If yes, can it be used as an extra cue for detecting interactions between real and virtual objects, and thus locating the real objects in AR environment more easily and accurately? Main Hypotheses #1 More accurate to indicate position on (curved) surface with high texture density (HTD) than with low texture density (LTD) HTD better than LTD

Main Hypotheses #2 Orientation of observer relative to target surface will have influence Top View Top View Real Surface Stereo Camera Real Surface Stereo Camera Virtual Pointer Virtual Pointer Centre different from Off-centre Main Hypotheses #3 Form of Virtual Pointer (VP) will have impact on alignment

performance LINE AREA VOLUME Main Hypotheses #4 Binocular disparity (i.e. crossed vs 0 vs uncrossed) will affect alignment performance C ND UC No disparity (ND) > Crossed (C) or Uncrossed (UC) Experimental Investigation of AR Surface Effects

Stereo Camera Stereoscopic AR Display Virtual Pointer Spaceball Cylinder Indigo 2 Barrier Methodology Factorial Expt. # Design Independent Variables Dependent Variables

(Measurement) #1 Texture Density Surface Orientation 2x2x3x3 Binocular Disparity VP Orientation Placement Error between perceived target and its actual position on real surface #2 Texture Density Surface Orientation 2x2x3x3 Binocular Disparity VP Form

Placement Error + Confidence Rating + Preference Rating #3 5x5 Texture Density Surface Orientation Placement Error + Angular Error between estimated normal and real surface normal Subjective Comparisons Ease of Use Transparency Ease of Fusion 6 Images

(15 paired comparisons) Texture Density High Low Virtual Pointer Form LINE AREA VOLUME Methodology Factorial Expt. # Design Independent

Variables Dependent Variables (Measurement) Expt. #1 Texture Density Surface Orientation 2x2x3x3 Binocular Disparity VP Orientation Placement Error between perceived target and its actual position on real surface Expt. #2 Texture Density

Surface Orientation 2x2x3x3 Binocular Disparity VP Form Placement Error + Confidence Rating + Preference Rating Expt. #3 5x5 Texture Density Surface Orientation Placement Error + Angular Error between estimated normal and real surface normal Placement Error vs Texture Density - 1

M ean Placem en t Erro r (cm ) VP Placem ent Error vs Texture Density (Error Bar = 95% CI, F(1,11) = 11.14, P = 0.007) Farther 1.6 1.2 0.8 0.4 0 -0.4 \ Closer \ Cylinder Surface High

\ \ Low Texture Density Main Experimental Results - 1 M e a n P la c e m e n t E r r o r ( c m ) Placement Error vs Texture Density - 2 Placement Error along Surface Normal vs Texture Density (Error Bar = 95% CI, F(4,40) = 41.34, P < 0.001) 1.6 1.4 1.2 1 0.8 0.6

0.4 0.2 0 0 10 20 30 40 Texture Density (%) Main Experimental Results - 2 Placement Error vs Surface Orientation - 1 M e a n P la c e m en t E rro r (c m ) VP Placem e nt Error vs Targe t Pos ition (Error Bar = 95% CI, F(1,11) = 98.19, P < 0.001)

Farther 1.6 1.2 0.8 0.4 0 -0.4 Closer Ce ntre Right Targe t Pos ition (relative to obs e rve r) Main Experimental Results - 3 M e a n P la c e m e n t E rro r ( c m ) Placement Error vs Surface Orientation - 2

12 Place m e nt Error along Surface Norm al vs Targe t Pos ition (Error Bar = 95% CI, F(4,40) = 11.90, P < 0.001) 2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 3 4 5

1 2 3 4 5 Targe t Pos ition on the Sphe re Main Experimental Results - 4 Angular Error vs Surface Orientation M e a n A lt it u d e E rro r ( d e g re e ) Altitude Error vs Targe t Pos ition on He m is phe re (Error Bar = 95% CI, F(4, 40) = 36.72, P < 0.001)

20 15 10 5 0 1 1 2 3 4 5 2 3 4 5 Targe t Pos ition

Main Experimental Results - 5 Subjective Comparison Results w.r.t. Ease of Use and Ease of Fusion Volume VP better than other VPs, regardless of texture density w.r.t. Transparency Volume VP + Highly textured least transparent combination Main Experimental Results - 6 Summary of Results - 1 Texture Density High better than Low Surface Orientation Centre

different from Off-Centre Summary of Results - 2 VP Form VOLUME > AREA > LINE (subjective comparisons) Binocular Disparity C ND UC Conclusions - 1 Perceptual conflict does exist when real

and virtual objects interact in 3D AR environments (a model proposed) Perceptual conflict can be used as extra depth cue to indicate interaction between real and virtual objects Optimal density value for Random Dot texture pattern was found as Engineering Solution for accurate 3D measurement R-V Interaction Interaction Process Process Model Model R-V Perception Cognition / Decision Making (Stereo Matching) (Stereo Matching + Cue Conflict Resolution)

behind Fusion difficulty VP behind, or at, real surface? at Fused Image? No fusion difficulty at (Transparency) No Stereo Matching :

VP in front of real surface? VP behind, or at, real surface? behind Yes User Localisation Achieved Adjustment (Breakdown/Conflict) External World Display of virtual pointer (VP)

superimposed upon real object surface AR Display VP Controller Conclusions - 1 Perceptual conflict does exist when real and virtual objects interact in 3D AR environments (a model proposed) Perceptual conflict can be used as extra depth cue to indicate interaction between real and virtual objects Optimal density value for Random Dot texture pattern found as Engineering Solution for accurate 3D measurement Conclusions - 2 Target position does affect alignment task: centrally located targets benefit performance, but have disadvantage when along the line of sight

Volumetric stereo graphic cursor (more fusable features along three dimensions) is subjectively the most favoured VP Pseudo-transparency contributes literature of depth perception cues (shapefrom-texture and stereo) Implications for for AR AR Interface Interface Design Design Implications Random Dot Stereogram enhances 3D alignment performance Perceptual conflicts can be used as extra depth cue to detect real object position 3D VP better than other VPs Perceptual errors always exist Limitations Implementation

Display Mode : stationary display without motion parallax and motion perspective Binocular Disparity : confounded with size cue and resolution Scope VP Design: line thickness of wire-frame VP Texture Pattern: square Random Dot pattern Future Work/Impact Work/Impact -- 11 Future Near Term Research Projected lighting : more practical Motion parallax with Video-HMD Computational vision may alleviate some error (being investigated) Projected lighting with random dot

texture pattern Simulated Projector Stereo Cameras Future Work/Impact - 2 Long Term Interests Integration of Computer Assisted Object Detection in AR Displays See-through HMD AR for Dismounted Soldiers in the Battlefield (e.g., Perceptual Conflicts, Navigational Aids, etc.) Other Human Computer Interaction (HCI) Topics (e.g., Integration of Electronic Information with

Human-Machine Systems, etc.) Acknowledgement Dr. Julius Grodski at Defence Research & Development Canada (DRDC) Toronto, Prof. Allison B. Sekuler and Prof. Paul Milgram at University of Toronto Dr. Stephen Ellis at NASA and Prof. Stanley Hamstra at University of Toronto Natural Sciences and Engineering Research Council (NSERC) Doctoral Scholarship Institute of Robotics and Intelligent Systems (IRIS), Canada Ontario Graduate Scholarship (OGS) Experimental #1 Main Result P la c e m e n t E r r o r (c m ) Farther 1 0

VP Placem e nt Error vs Surface Texture (Error Bar = 95% CI, F(1,9) = 619.71, P< 0.001) \ \ Cylinder Surface \ -1 -2 -3 -4 -5 -6 -7 -8 -9 Closer

High Low Texture Density \ VP Form and Orientation in 1st Experiment : Vertical Diagonal Observer Horizontal Interaction in Experiment # 1 VP Placement vs Texture and Target Position Placement Error (cm)

(Error Bar = +/- 1SD, F(1,9) = 246.33, P< 0.001) 1 0 -1 -2 -3 -4 -5 -6 -7 -8 -9 -10 -11 High Low Centre Right Angular Displacement of Target Normal

Interaction between Surface Texture (High vs Low) and Target Position (Center vs Right) Paired Comparison Result Mean Z score 0 1.15 1.71 2.26 3.16 1 2 34

5 6 0 0.25 0.89 2.97 3.82 4.37 6 5 4 Ease of Use Image # Transparency Image #

Ease of Fusion Image # 2 0 1 1 3 0.69 0.73 2 3 2.08 2.14 5

4 2.26 6 Measurement of Placement Error along Surface Normal Estimated Target Surface Normal Real Surface Target Positive Error Positive error shows the estimated target is inside the sphere along surface normal M e a n P la c e m e n t E r r o r ( c m ) Placement Error vs Texture Density for Experiments 2 and 3

Placement Error along Surface Normal vs Texture Density (Error Bar = 95% CI, F(4,40) = 41.34, P < 0.001) 1.6 Experiment 2 Experiment 3 1.4 1.2 1 0.8 0.6 0.4 0.2 0 0 10 20

30 Texture Density (%) 40 Definition of Angular Error Y Altitude Error : Angular distance between estimated normal TP and real surface normal TN N P ' Q

Azimuth Error : Angular Error between horizontal projection OR and OS ' T O S R ' '

Z X Angular Bias in Spherical Coordinate Y N E T Bias Area S Z X Angular bias tilted upwards from real surface normal (TN) Example of Distribution of Altitude and Azimuth

Altitude Error vs Azimuth Error at Target 5 60 Altitude Error (degree) 50 40 30 20 10 0 -180 -135 -90 -45 -10 0 45 90

135 180 -20 -30 Azimuth Error (degree) Angular bias between estimated and real surface normal Ground Truth Measurements in Real Scene Cylinder Stimulus Side View Stereo Cameras Top

View Iron plate Cylinder Calibration object Real World Origin (0,0,0) Real Distance in Depth (Z) Registration Verification: Measurement of a Pin and a Cube Stereo Cameras Calibration Target Calibration Cube

P e r c e n t a g e o f D is t a n c e ( D ) f o r 1 P ix e l C h a n g e in H o r iz o n t a l D is p a r it y VP Resolution for Experiment 2 VPVP Resolution and Tests Re s olution (Pe rceAccuracy ntage of Re al Dis tance vs One Pixe l Error of V P in Dis play) 8.00% 7.00% 6.00% 5.00% 4.00% 3.00% 2.00%

1.00% 0.00% 0 1 2 3 4 Dis tance of Obje ct from Cam e ras (m ) 5 Retinal Disparity in Stereoscopic Display for One Pixel Separation (exaggerated) ZD Stereoscopic Display Monitor

C b Apparent position of point C, due to 1 pixel horizontal disparity e 2e P ) a b 2arctg ( ) 2arctg ( ) d 2d P = 1 pixel separation XD Viewers Eye a 2e

b /2 d Psychophysical Standard for Texture Density Control Spatial scale (size) Homogeneity (spatial regularity, density is approximately constant over the surface) Isotropy (no orientation bias, equally to be oriented in all directions) compression Practical Augmented Reality Example Virtual Tape Measure for Minimally Invasive Surgery Distance between point 1 and point 2 is 7mm

Coordinate of Point 1 (2.3, 14.7, 96.2) Coordinate of Point 2 (1.8, 14.4, 95.8)

Recently Viewed Presentations

  • Improved Average-Case Lower Bounds for De Morgan Formula Size

    Improved Average-Case Lower Bounds for De Morgan Formula Size

    Main Theorem:Learninglog(?)-sparse parities requires either Ω(?⋅log0.99?)memory bits or ?Ωloglog? number of samples. Implies same bounds for learning. ... Improved Average-Case Lower Bounds for De Morgan Formula Size
  • Activity 3.1.1 VEX Robotics Platform and ROBOTC Software

    Activity 3.1.1 VEX Robotics Platform and ROBOTC Software

    VEX® Structure Subsystem. VEX®Structure Subsystem forms a robot base. Contains square holes on a standardized ½ in. grid. Allows VEX® components to be assembled in various configurations. This is a very brief overview of the VEX robotics platform. Refer to...
  • 5.CBR-Retrieval - University of Calgary

    5.CBR-Retrieval - University of Calgary

    The most similar object Fi OR 2. the m most similar objects {F1,...,Fm} (ordered or unordered) OR 3. All objects {F1,...,Fm} which have to Q at least a similarity simmin Problem: How to organize a case base for efficient retrieval?
  • Haiku Poetry - Mrs. Miranda&#x27;s Classroom Website

    Haiku Poetry - Mrs. Miranda's Classroom Website

    Haiku Poetry Pink cherry blossoms Cast shimmering reflections On seas of Japan Haiku Poetry Warm soup in a bowl letters of the alphabet hang on the teaspoon HAIKU POETRY On the following slides are pictures of seasons. Now it is...
  • Personal Monitoring Dosimetry IRAD2731 R A D I

    Personal Monitoring Dosimetry IRAD2731 R A D I

    Colorimetric Method - color change. Small metal box containing several clear tubes of liquid. The solution consists of chloroform and the dye brome-creosol purple. When the chloroform absorbs radiation energy, hydrochloric acid is produced which changes the dye color from...
  • Development of Auditory-Visual Speech Perception in Toddlers ...

    Development of Auditory-Visual Speech Perception in Toddlers ...

    Auditory-visual speech perception in young children: language and age-specific factors ... (Native minus Non-native speech perception N-NN) 18 N [voiced vs. voiceless] + 18 NN [voiced vs. prevoiced] Reading Test WRAT-3 reading subtest: 15 letters + 42 words = 57...
  • STEP 1: Competing Pathway - Robbie

    STEP 1: Competing Pathway - Robbie

    Format adapted from Diana Browning Wright, Behavior/Discipline Trainings, 2005. For Robbie. Benchmark 1: Given any test situation or writing task, Robbie will use self-control strategies taught (i.e. calm down checklist, anxiety rater),
  • Assessment for learning - Resourceful Indonesian

    Assessment for learning - Resourceful Indonesian

    LOTE HOD/Coordinator Cluster Meeting 13 August 2004 Literacy through Languages