Cover Page Using a Light Image - IEEE Standards Association

Cover Page Using a Light Image - IEEE Standards Association

[Deep Learning-based VR Sickness Assessment Considering Content Quality Factor] [Sangmin Lee, Kihyun Kim, Hak Gu Kim, Minho Park, Yong Man Ro / KAIST] Compliance with IEEE Standards Policies and Procedures Subclause 5.2.1 of the IEEE-SA Standards Board Bylaws states, "While participating in IEEE standards development activities, all participants...shall act in accordance with all applicable laws (nation-based and international), the IEEE Code of Ethics, and with IEEE Standards policies and procedures." The contributor acknowledges and accepts that this contribution is subject to

The IEEE Standards copyright policy as stated in the IEEE-SA Standards Board Bylaws, section 7,, and the IEEE-SA Standards Board Operations Manual, section 6.1, http:// The IEEE Standards patent policy as stated in the IEEE-SA Standards Board Bylaws, section 6,, and the IEEE-SA Standards Board Operations Manual, section 6.3, policies/opman/sect6.html 2 IEEE 3079 HMD Based VR Sickness Reducing Technology Dongil Dillon Seo, [email protected]

Deep Learning-based VR Sickness Assessment Considering Content Quality Factor Date: 2019-7-8 Author(s): Name Affiliation Phone [optional] Email [optional] Sangmin Lee

Kihyun Kim Hak Gu Kim Minho Park Yong Man Ro KAIST KAIST KAIST KAIST KAIST +82 10 5611 6617 +82 10 4408 8264 +82 10 5137 3551

+82 10 8539 5033 +82 10 3896 1429 [email protected] [email protected] [email protected] [email protected] [email protected] 3079-19-0017-00-0002-Database-Construction-for-Quantitative-Analysis-of-VR-Sickness 3 Virtual Reality

VR contents such as 360-degree video can provide realistic and immersive viewing experience for viewers. The development of the 360-degree cameras and VR displays has increased the interest and popularity of the VR contents for various applications. 360-degree cameras VR displays Training Entertainment Education

Healthcare 4 VR Sickness VR sickness, which is one of motion sickness, could cause three major physical symptoms [1] Nausea symptom : Salivation, sweating, burping, Oculomotor symptom : Visual fatigue, headache, Disorientation symptom : Dizziness, fullness of head, 1) Nausea symptoms 2) Oculomotor symptoms

3) Disorientation symptoms [1] R. S. Kennedy, N. E. Lane, K. S. Berbaum, and M. G. Lilienthal, Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness, Int. J. Aviat. Psychol., vol. 3, no. 3, pp. 203220, 1993. 5 Triggers of VR Sickness Exceptional motion Exceptional motion patterns of VR contents such as acceleration and rapid rotation could lead to excessive motion mismatch between the simulation motion of the content and the physical motion of the user [2].

Acceleration and rapid rotation [2] H.G. Kim, H. Lim, S. Lee, and Y.M. Ro, Vrsa net: Vr sickness assessment considering exceptional motion for 360 vr video, IEEE Transactions on Image Processing, vol. 28, no. 4, pp. 16461660, 2019. 6 Triggers of VR Sickness Resolution Low resolution has higher simulator sickness than the contents of high resolution with respect to spatial and temporal inconsistency [3]. Spatial inconsistency : low quality video has a loss of spatial information. Temporal inconsistency : low quality video frames also lose temporal information from the spatial inconsistency.

High resolution low resolution [3] R. Barrette, K. Dunkley, R. Kruk, D. Kurts, S. Marshall, T. Williams, P. Weissman, and S. Antos, Flight simulator: Advanced wide field-of-view, helmet-mounted, infinity display system, Technical Report AFHRLTR- 8936, Air Force Human Resources Laboratory, Williams Air Force Base, 1990. 7 Contribution of the Proposed Work Propose a novel deep learning-based objective assessment framework for predicting VR sickness caused by quality degradation considering spatiotemporal perceptual characteristics of VR contents. It is the first work that quantifies VR sickness caused by quality-degradation.

To evaluate the effectiveness of the proposed method, we built a new benchmark dataset that consists of 360-degree videos with 4 resolution types, physiological signals, and the corresponding SSQ scores. 8 Proposed Method Overall framework Spatial perception-guider helps spatial encoder reliably extract the spatial perception considering spatial inconsistency. Temporal perception-guider helps temporal encoder reliably extract the temporal perception considering temporal inconsistency 9

Proposed Method Spatial inconsistency The Structural Similarity (SSIM) is a perceptual metric that quantifies image quality degradation. SSIM is used for measuring structural similarity (0 ~ 1) between two images. SSIM , Spatial inconsistency . Input frame Reference frame SSIM = 0.72 10

Proposed Method Temporal inconsistency Flicker amount is defined as an average of difference at pixel level between adjacent two frames as follows, . Flicker score is also computed as . Flicker score , Temporal inconsistency . Input frame Reference frame . =0.3

. =0.5 . =|0.2 0.4|=0.2 11 Proposed Method Spatial encoder and spatial perception-guider Spatial encoder and spatial perception-guider are trained with minimizing SSIM prediction loss, 12

Proposed Method Temporal encoder and temporal perception-guider Temporal encoder and Flicker predictor are trained with minimizing Flicker prediction loss, 13 Proposed Method Sickness score predictor Takes spatio-temporal feature of input frames and output predicted SSQ score passing through three FC layers. Sickness score predictor is trained with minimizing SSQ prediction loss, where denotes gaussian noise with standard deviation of

content SSQ scores. Weight parameter is set as 0.2. 14 Proposed Method Training scheme First, Spatial encoder and Temporal encoder are jointly trained with SSIM and Flicker prediction loss, After converging SSIM and Flicker prediction network, SSQ predictor is trained with SSQ loss, 15 Benchmark Database

Subjective assessment experiments 17 subjects (ranging between 20 to 31 years old / normal or corrected-tonormal vision) 20 types x 4 resolutions (SD, HD, FHD, UHD) = 80 360-degree contents For every contents, watching (1min) / SSQ marking and resting (2min) Subjective score: SSQ score Physiological signal: EKG, GSR Equipment Oculus Rift CV1 for displaying 360-degree videos. NeuLog NUL-208 and NUL-207 for measuring EKG and GSR. 16 Benchmark Database

Subjective experiment results VR sickness tends to increase as spatial resolution decreases. Especially, VR sickness in SD is severe. Average SSQ score: UHD : 25.855 FHD : 26.822 HD : 28.191 SD : 38.434 17 Experiments

Sickness score prediction performance Measures : evaluated performance with 5-fold cross validation Pearson linear correlation coefficient (PLCC) Spearman rank order correlation coefficient (SROCC) Root mean square error (RMSE) 18 Thank You

Recently Viewed Presentations

  • Penilaian prestasi kerja (performance appraisal)

    Penilaian prestasi kerja (performance appraisal)

    Arial Trebuchet MS Wingdings 2 Wingdings Calibri Arial Narrow Tahoma Opulent 1_Opulent 2_Opulent 3_Opulent 4_Opulent 5_Opulent Penilaian prestasi kerja (performance appraisal) Prolog Definisi Manfaat penilaian prestasi kerja Manfaat penilaian prestasi kerja Manfaat penilaian prestasi kerja Manfaat penilaian prestasi kerja PERSIAPAN...
  • Los mandatos -

    Los mandatos -

    Garamond Arial Wingdings Stream 1_Stream Los mandatos PowerPoint Presentation PowerPoint Presentation Mandatos afirmativos informales (Tú Commands) Irregulares Pronoun Placement Por ejemplo: Practicamos PowerPoint Presentation Mandatos negativos informales (Negative tú commands) Por ejemplo Irregulares - Negativos Irregulares - Negativos ...
  • Caracteristici radiografice Hidrocefalie acuta

    Caracteristici radiografice Hidrocefalie acuta

    Caracteristiciradiologice RMNHidrocefalieobstructivaacuta. La adultidatoritainchideriifontanelelordilatareaventriculilor nu esteatat de pronuntata.
  • Concur Travel and Expense Management - Jefferson

    Concur Travel and Expense Management - Jefferson

    Concur Travel and Expense Management Creating An Expense Report for Reimbursement 2017 Access Concur at Enter your campus key and password. If you are redirected to a page that looks like the image below, please retype the address...
  • Knee Evaluation - University of Michigan

    Knee Evaluation - University of Michigan

    Evaluation of Knee Injuries. Dr. Alan A. Zakaria, D.O., M.S. 1080 Kirts Blvd., Suite 400 ... Medial Collateral Ligament (MCL) ... Maximum tension at full extension. Restraint to valgus stress. Knee Anatomy: Lateral Collateral Ligament (LCL) Posterosuperior lateral femoral condyle...
  • Chapter 32-34: Animal Diversity Objectives 1. 2. 3.

    Chapter 32-34: Animal Diversity Objectives 1. 2. 3.

    Define common characteristics amongst all animals Animals can be characterized by "body plans" ... (Deuterostome development and includes chordates and vertebrates) Sponges p. 670 -Thought to have been plants -Suspension Feeders -Water is drawn through pores and a current provided...
  • Overview -

    Overview -

    LiveConnectClient. Requires a valid session. Required scope: wl.skydrive - Read access. wl.skydrive_update - read and write access. LiveOperationResult. contains a dynamic dictionary containing resultant key/value pairs. Working with SkyDrive API
  • Abnormal Psychology and Therapy Project -

    Abnormal Psychology and Therapy Project -

    -Heinz Kohut, contemporary psychodynamic therapist. Humanistic Therapies-humanistic therapies: treatments, unique in their emphasis on people's self-healing capacities, that encourage clients to understand themselves and grow personally ... Abnormal Psychology and Therapy Project Last modified by: