Multiple View Geometry in Computer Vision

Multiple View Geometry in Computer Vision

Multiple View Reconstruction Class 25 Multiple View Geometry Comp 290-089 Marc Pollefeys Content Background: Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation. Single View: Camera model, Calibration, Single View Geometry.

Two Views: Epipolar Geometry, 3D reconstruction, Computing F, Computing structure, Plane and homographies. Three Views: Trifocal Tensor, Computing T. More Views: N-Linearities, Self-Calibration, Multi View Reconstruction, Bundle adjustment, Cheirality, Duality, Dynamic SfM Multi-view computation practical structure and motion recovery from images

Obtain reliable matches using matching or tracking and 2/3-view relations Compute initial structure and motion sequential structure and motion recovery hierarchical structure and motion recovery Refine structure and motion bundle adjustment Auto-calibrate Refine metric structure and motion

Sequential SaM recovery Select two initial views Extract features Compute two-view geometry and matches Initialize projective pose for two-views Initialize new structure For every additional view i Extract features Compute two view geometry i-1/i and matches Compute pose using robust algorithm Refine existing structure

Initialize new structure Refine proj. SaM estimation Self-calibrate Refine metr.SaM estimation X F x Hierarchical structure and

motion recovery Compute 2-view Compute 3-view Stitch 3-view reconstructions Merge and refine reconstruction F T H PM Refining structure and motion

Minimize reprojection error m n min D mki , PkMi Pk ,M i 2

k 1 i 1 Maximum Likelyhood Estimation (if error zero-mean Gaussian noise) Huge problem but can be solved efficiently (Bundle adjustment)

Sparse bundle adjustment LM iteration: m -1 T J J J e T n Jacobian of

D m ki , Pk M i 2 has sparse block k 1 i 1 structure

P1 P2 P3 M U1 im.pts. view 1

U2 J W N JT J U3 WT 12xm

3xn (in general much larger) V Needed for non-linear minimization Sparse bundle adjustment Eliminate dependence of camera/motion parameters on structure parameters

Note in general 3n >> 11m 1 I WV N 0 I Allows much more efficient computations e.g. 100 views,10000 points, solve 1000x1000, not 30000x30000

Often still band diagonal use sparse linear algebra algorithms U-WV-1WT WT V 11xm

3xn Degenerate configurations (H&Z Ch.21) Camera resectioning Two views More views Camera resectioning Cameras as points

2D case Chasles theorem Ambiguity for 3D cameras Twisted cubic (or less) meeting lin. subspace(s) (degree+dimension<3) Ambiguous two-view reconstructions Ruled quadric containing both scene points and camera centers alternative reconstructions exist for which the reconstruction of points located

off the quadric are not projectively equivalent hyperboloid 1s cone pair of planes single plane + 2 points single line + 2 points Multiple view reconstructions Single plane is still a problem Hartley and others looked at 3

and more view critical configurations, but those are rather exotic and are not a problem in practice. Carlsson-Weinshall duality (H&Z Ch.19) Exchange role of points and cameras Dualize algorithm for n views and m+4 points to algorithm for

m views and n+4 points e.g. (2im,7+pts)(3+im,6pts) Reduced camera duality Reduced camera: Carlsson-Weinshall duality Reduced camera reconstruction

N M M N Obtain reduced cameras Pick 4 reference points to form projective basis in P 2 e1 , e2 , e3 , e4

E1 , E2 , E3 , E4 Dual algorithm outline: input transpose transform Solve dual problem Dualize

Transform to Reverse transform reduced cameras extend Applications 6 points in 3 views minimal, useful for 3-view RANSAC reduced F-matrix (eiTFei=0,i=14)

in N views useful for reconstruction from tracks 7 points in 4 or more views reduced trifocal tensor 6 points in N views (Hartley and Dano CVPR00) use Sampson error in stead of algebraic (important because of projective warping!)

Oriented projective geometry (~H&Z Ch.20) two geometric entities are equivalent if they are equal up to a strictly positive scale factor projective geometry oriented projective geometry (from PhD Stephane Laveau)

Oriented projective geometry front back Oriented line oriented line pxq goes from p to q over shortest distance Oriented plane

Front LT X 0 Camera focal plane In front of camera P1 P P2 P3 or

P3 X 0 w0 x x y PX w oriented plane through camera center and 3D point

M C Oriented epipolar plane 1 point correspondence allows orientation Eliminates zone and But not ( not known) Multi camera orientation constraint

Hartleys Cheirality Nisters QUARC Application to view synthesis Laveau96 which point is in front? Epipole orientation Next class:

Dynamic structure from motion

Recently Viewed Presentations

  • Introduction to Archetypes

    Introduction to Archetypes

    Assignment. Review the literary and villain archetypes on the class page under the "Evaluation Unit" Choose one of the books read this year: Jane Eyre ,Things Fall Apart, In Cold Blood, The Importance of Being Earnest, Othello or Frankenstein.
  • The WHITE Cross - Rubik&#x27;s: You Can Do The Cube

    The WHITE Cross - Rubik's: You Can Do The Cube

    Reviewing FACES. The flat (two-dimensional) square, on each surface of the cube. Up Face. Down Face. Left Face. Right Face. Front Face. Back Face. How many faces does a cube have?
  • Cold War Conflicts

    Cold War Conflicts

    The Nine Parts of Speech. Nouns. Verbs. Pronouns. Adjectives. Adverbs. Conjunctions. Articles. Prepositions. Interjections. Every word in a sentence has a particular ...
  • Carlos and the Skunk Vocabulary

    Carlos and the Skunk Vocabulary

    A brief look. Synonyms - glance, peek. As a child I would hide when my mother wrapped presents so that I could try and get a quick . glimpse. of the special surprises. Stunned. Verb. To be overcome with astonishment...
  • Karen ORourke Centre for Excellence in Teaching and

    Karen ORourke Centre for Excellence in Teaching and

    I will be working on this project again this year and hope to put a draft together of a paper to go into Pharmacy Education over the next few weeks.There may also be the possibilty of presenting a poster on...
  • Introduction I. S3 Infotech Private Limited S3 Infotech

    Introduction I. S3 Infotech Private Limited S3 Infotech

    Having 100+ projects experience on Implementation / Support / Development / Audit / Project Management. Already delivered projects on . Latest version of AX/NAV/CRM. Our Growth. Company starts operation with the first project from Europe . 2006. 2007.
  • Bellringer 09/29/2014 How may human activities affect marine

    Bellringer 09/29/2014 How may human activities affect marine

    -Trace the effects of basically eliminating certain parts of the food web.-For example: lets say humans take out most of the tuna in the web - the marlins now have lost one of their main food sources and the squid...
  • mTrack: High-Precision Passive Tracking Using Millimeter Wave Radios

    mTrack: High-Precision Passive Tracking Using Millimeter Wave Radios

    Especially from small objects, like pen . Costly to deploy substantial nodes. Overview the Basic Idea. 60GHz . ... 2 cm and 6 cm. Event Detection. 7 users. Each provides a 10-sampletraining set. 20~50-sample testing set. Application: Trackpad. Experiment Setup.