Lecture 1: Images and image filtering

Lecture 1: Images and image filtering

CS5670: Computer Vision Noah Snavely Lecture 15: Structure from motion Readings Szeliski, Chapter 7.1 7.4

Structure from motion Given many images, how can we a) figure out where they were all taken from? b) build a 3D model of the scene?

This is (roughly) the structure from motion problem Structure from motion Reconstruction (side)

Input: images with points in correspondence (ui,j,vi,j) Output structure: 3D location xi for each point pi motion: camera parameters Rj , tj possibly Kj Objective function: minimize reprojection error

(top) pi,j = Input

Camera calibration & triangulation Suppose we know 3D points And have matches between these points and an image How can we compute the camera parameters?

Suppose we have know camera parameters, each of which observes a point How can we compute the 3D location of that point? Structure from motion

SfM solves both of these problems at once A kind of chicken-and-egg problem (but solvable) Photo Tourism

First step: how to get correspondence? Feature detection and matching Feature detection Detect features using SIFT [Lowe, IJCV 2004]

Feature detection Detect features using SIFT [Lowe, IJCV 2004] Feature matching Match features between each pair of images

Feature matching Refine matching using RANSAC to estimate fundamental matrix between each pair Correspondence estimation

Link up pairwise matches to form connected components of matches across several images Image 1 Image 2

Image 3 Image 4 Image connectivity graph

(graph layout produced using the Graphviz toolkit: http://www.graphviz.org/) Structure from motion X4 X1

X3 g(R, T, X) X2

X5 minimize X7

non-linear least squares X6 p1,1 p1,3

p1,2 Camera 1 R1,t1

Camera 3 Camera 2 R2,t2 R3,t3

Problem size What are the variables? How many variables per camera? How many variables per point? Trevi Fountain collection

466 input photos + > 100,000 3D points = very large optimization problem Structure from motion Minimize sum of squared reprojection errors:

predicted image location observed image location

indicator variable: is point i visible in image j ? Minimizing this function is called bundle adjustment

Optimized using non-linear least squares, Levenberg-Marquardt e.g. Is SfM always uniquely solvable?

Is SfM always uniquely solvable? No Incremental structure from motion

Incremental structure from motion Incremental structure from motion Incremental structure from motion

Photo Explorer Demo Questions?

SfM Failure cases Necker reversal Structure from Motion Failure cases Repetitive structures

SfM applications

3D modeling Surveying Robot navigation and mapmaking Visual effects (Match moving) https://www.youtube.com/watch?v=RdYWp70P_kY

Applications Photosynth Applications Hyperlapse https://www.youtube.com/watch?v=SOpwHaQnRSY

Recently Viewed Presentations

  • PowerPoint Presentation

    PowerPoint Presentation

    (Ask for feedback) When you go through a C.O.P.E. experience, you will not succeed without some team building occurring. Sometimes you can build your group helping lead change, such as Al did in choosing this staff, but usually the people...
  • VIRUSOLOIJA (VRUSI UN RPUSHROMOSOMU ENTISKIE ELEMENTI) IEVADS, VIRUSOLOIJAS

    VIRUSOLOIJA (VRUSI UN RPUSHROMOSOMU ENTISKIE ELEMENTI) IEVADS, VIRUSOLOIJAS

    Līdzīgas idejas tika izvirzītas par hloroplastu izcelšanos. VĪRUSU IESPĒJAMĀ LOMA EVOLŪCIJĀ Filips Bells (Philip Bell, postdoctoral research fellow at Macquarie University, New South Wales, Australia; founding Director of Microbiogen Pty Ltd.) vīrusu eikarioģenēzes teorijas (2001. - 2004.) autors.
  • Rules Questions - SAE A

    Rules Questions - SAE A

    A separate battery to power the supercharger would not be acceptable, as it would contravene the rules by providing another source of unmeasured stored energy. FSAE-A Rules Committee" Question Answer. I have a question about rule IC1.1.1 from article 1...
  • JACKSON SCHOOL DISTRICT Middle School Math Informational Night

    JACKSON SCHOOL DISTRICT Middle School Math Informational Night

    Rubric: Students must earn a minimum of 19 out of a possible 23 points to qualify for 7th Grade Algebra. Students who do not exceed 19 points but have maintained an 80 or better average in 6th Grade Advanced Math...
  • Friendship With GOD Name Lay Person Kairos -

    Friendship With GOD Name Lay Person Kairos -

    God sent these men and used their friendship with Him to spread His word. He finally sent His son, Jesus, the Christ Friendship With GOD YOU ARE WORTHY TO BE GOD'S FRIEND! Jesus is God. He and the Father are...
  • Chapter 2 Drugs in the Body  Paradigm Publishing,

    Chapter 2 Drugs in the Body Paradigm Publishing,

    A drug's activity is based on its molecular shape and functional groups. ... elimination (excretion) Pharmacokinetics. Absorption. Is how drugs enter the bloodstream. Is the upward-sloping part of the time-response curve. Affects the onset and extent of drug action.
  • Title in Arial Narrow Bold All Caps

    Title in Arial Narrow Bold All Caps

    John Holland's theory of vocational choice, 1959. Focuses on the person, the environment, and how they fit together. Says most people and work environments lead with two or three of six themes . Themes next to each other are most...
  • The Human Body - local-brookings.k12.sd.us

    The Human Body - local-brookings.k12.sd.us

    Mammals metabolize ammonia into urea, a less toxic form of nitrogenous waste. Requires energy to build urea, but mammals don't need as much water to dilute urea. Allows mammals to conserve water, mammals often don't have constant access to water....