# CSE 240 (291) week3

CV: Perceiving 3D from 2D Many cues from 2D images enable interpretation of the structure of the 3D world producing them MSU CSE 803 Stockman 1 Topic roadmap

Labeling 3D structure in a 2D image Labeling constraints on edge graphs Huffman-Clowes-Waltz labeling Other cues motion parallax shape from texture or shading stereo from two 2d images MSU CSE 803 Stockman 2 Topic roadmap:

mathematical models Shape from shading Depth from stereo Depth from focus Perspective transformation (review) MSU CSE 803 Stockman

3 Many 3D cues How can humans and other machines reconstruct the 3D nature of a scene from 2D images? What other world knowledge needs to be added in the process? MSU CSE 803 Stockman

4 Vocabulary for image labeling Interpret the local structure of the scene in the image space MSU CSE 803 Stockman 5 Some terms for local 3D structure (left) intensity image

of 3 blocks (right) result of 5x5 Prewitt operator Blade: ( > ) as in the blade of a knife, where the normal to the occluding surface element. Occluding and occluded surfaces unrelated. Crease: (convex + or concave -) formed by an abrupt change to a surface or the joining of two surfaces. Surface on both sides of the crease can be MSU CSE 803 Stockman 6 sensed

Limb: smooth object contour egg Soup can Limb: (>>) formed by viewing a smooth 3D object, such as an arm or a soup can: when approaching the contour, the surface normal becomes perpendicular to the line of sight. (Right side of arrow is the occluding surface.) MSU CSE 803 Stockman 7

Albedo and lighting mark: (M) surface mark or change of albedo (reflectance) and not the 3d surface, creating an intensity contour in the image shading: (S) illumination change due to a change in lighting or shadow on the surface, creating an intensity contour in the image

MSU CSE 803 Stockman 8 Labeling image contours interprets the 3D scene structure shadow relates to illumination, not material Logo on cup is a mark on the material

+ MSU CSE 803 Stockman An egg and a thin cup on a table top lighted from the top right 9 Intrinsic Image stores 3D info in pixels and not intensity.

For each point of the image, we want depth to the 3D surface point, surface normal at that point, albedo of the surface material, and illumination of that surface MSU CSE 803 Stockman

10 Practice labeling the contours (left) an unopened can of Brand X soda is a solid blue can with a bright orange block X. (right) an empty box with all four flaps open so even the bottom of the box is visible. MSU CSE 803 Stockman 11

3D scene versus 2D image Creases Corners Faces Occlusions (for some viewpoint)

Edges Junctions Regions Blades, limbs, Ts MSU CSE 803 Stockman 12 Labeling of simple polyhedra

MSU CSE 803 Labeling of a block floating in space. BJ and KI are convex creases. Blades AB, BC, CD, etc model the occlusion of the background. Junction K is a convex trihedral corner. Junction D is a T-junction modeling the occlusion of blade

CD by blade JE. Stockman 13 Trihedral Blocks World Image Junctions: only 16 cases! Only 16 possible junctions in 2D formed by viewing 3D corners formed by 3 planes and viewed from a general viewpoint! From top to bottom: L-junctions, arrows, forks, and T-junctions. MSU CSE 803 Stockman 14

Challenge ! Create a scene shot of no more than 2 blocks that creates all 16 junctions in the image MSU CSE 803 Stockman 15 How do we obtain the catalog?

think about solid/empty assignments to the 8 octants about the X-Y-Z-origin think about non-accidental viewpoints account for all possible topologies of junctions and edges then handle T-junction occlusions

MSU CSE 803 Stockman 16 Blocks world labeling Left: block floating in space Right: block glued to a wall at the back MSU CSE 803 Stockman

17 Try labeling these: interpret the 3D structure, then label parts What does it mean if we cant label them? If we can label them?

MSU CSE 803 Stockman 18 Waltz filtering discards edge interpretations spanning junctions MSU CSE 803 Stockman 19 1975 researchers very

excited very strong constraints on interpretations several hundred in catalogue when cracks and shadows allowed (Waltz): algorithm works very well with them but, world is not made of blocks!

later on, curved blocks world work done but not as interesting MSU CSE 803 Stockman 20 Backtracking or interpretation MSU CSE 803 Stockman 21 Interpretation tree search MSU CSE 803 Stockman

22 Necker cube has multiple interpretations Label the different interpretations A human staring at one of these cubes typically experiences changing interpretations. The interpretation of the two forks (G and H) flip-flops between front corner and back corner. What is the explanation?MSU CSE 803 Stockman 23

Depth cues in 2D images MSU CSE 803 Stockman 24 Interposition cue Def: Interposition occurs when one object occludes another object, thus indicating that the occluding object is closer to the viewer than the occluded object. MSU CSE 803 Stockman

25 interposition T-junctions indicate occlusion: top is occluding edge while bar is the occluded edge Bench occludes lamp post leg occludes bench lamp post occludes fence railing occludes trees MSU CSE 803

Stockman 26 Perspective scaling: railing looks smaller at the left; bench looks smaller at the right; 2 steeples are far away Forshortening: the bench is sharply angled relative to the viewpoint; image length

is affected accordingly MSU CSE 803 Stockman 27 Texture gradient reveals surface orientation ( In East Lansing, we call it corn not maize. ) Note also that the rows appear to converge in 2D

Texture Gradient: change of image texture along some direction, often corresponding to a change in distance or orientation in the 3D world containing the MSU CSE 803 Stockman 28 objects creating the texture. 3D Cues from Perspective MSU CSE 803 Stockman 29 3D Cues from perspective

MSU CSE 803 Stockman 30 More 3D cues Virtual lines Falsely perceived interposition MSU CSE 803 Stockman 31

More 3D cues 2D alignment 2D image curves usually means 3d create perception of alignment 3D surface MSU CSE 803 Stockman 32

structured light can enhance surfaces in industrial vision Sculpted object Potatoes with light stripes MSU CSE 803 Stockman 33 Shape (normals) from

shading Clearly intensity encodes shape in this case Cylinder with white paper and pen stripes Intensities plotted as a surface MSU CSE 803 Stockman 34

Shape (normals) from shading Plot of intensity of one image row reveals the 3D shape of these diffusely reflecting objects. MSU CSE 803 Stockman 35

## Recently Viewed Presentations

• Inquiry means to ask questions and seek insightsinto problems that intrigue us. These questions can be broad, but also focused. They canarise from curiosity about natural phenomena, or be more socially-oriented. What happensto clouds when they disappear? What makes sugar...
• 非ST段抬高性ACS抗栓治疗 ——从循证到实践 北京协和医院 心内科 张抒扬 低分子肝素抗凝治疗是NSTEACS抗栓中重要部分 依诺肝素（克赛）是ACC/AHA 指南唯一被推荐使用的低分子肝素 总 结 1、非ST段抬高性ACS治疗的精髓—抗栓 2、抗栓与出血风险 3、循证到实践 Thank you for attention.
• In our SAI, the criteria we use in compliance audit is strictly based on laws and regulations. ... Answer from one region (CAROSAI) onlycont. Honduras: There is a Public Ethics and Probity Departement in charge of: Promotetheperformanceofpublic servants basedonprobity and...
• Marker Assisted Selection in Soybean Brian Diers University of Illinois Outline Overview of the soybean industry. Marker assisted selection (MAS) in the private sector. Experiences mining for new genes using MAS in my program. Soybean cyst nematode (SCN) Aphid resistance...
• Organisation and funders. The University of St Andrews offered to be the overall lead on this project so we can source funding and importantly have a safe neutral place for survey data.. Thanks to some end of financial year funding...
• Overcoming Unconscious Bias in Letters of Recommendation. Paula Warrick, Senior Director, Office of Merit Awards, American University ([email protected]) . Kate Mamiseishvili, Associate Dean for Academic and Student Affairs, Professor of Higher Education, University of Arkansas ([email protected])
• PCl 3, PCl 5, S 2 Cl 2, Cl 2 - all have simple molecular structures (covalently bonded molecules) held together by weak van der Waals' forces, resulting in low melting and boiling points. S 2 Cl. 2 Cl 2....
• Registration for childminders. What you need to know if you want to be a childminder. What you need to know if you want to be a childminder. Slide . Reassure participants this is an introductory session, and that theywill receive...