Image filtering Hybrid Images, Oliva et al., http://cvcl.mit.edu/hybridimage.htm 1 Image filtering Hybrid Images, Oliva et al., http://cvcl.mit.edu/hybridimage.htm 2 Reading Szeliski, Chapter 3.1-3.2 3 What is an image? 4 Images as functions We can think of an image as a function, f, from R2 to R: f( x, y ) gives the intensity at position ( x, y ) Realistically, we expect the image only to be defined over a rectangle, with a finite range: f: [a,b]x[c,d] [0,1] A color image is just three functions pasted together. We can write this as a vector-valued function: r ( x, y ) f ( x, y ) g ( x, y ) b( x, y )

5 Images as functions snoop 3D view 6 What is a digital image? In computer vision we usually operate on digital (discrete) images: Sample the 2D space on a regular grid Quantize each sample (round to nearest integer) If our samples are apart, we can write this as: f[i ,j] = Quantize{ f(i , j ) } The image can now be represented as a matrix of integer values 7 Image processing An image processing operation typically defines a new image g in terms of an existing image f. We can transform either the domain or the range of f. Range transformation: Whats kinds of operations can this perform? g (x,y) = f (x,y) + 20 8 Image processing

Some operations preserve the range but change the domain of f : What kinds of operations can this perform? g (x,y) = f (-x,y) 9 Image processing Still other operations operate on both the domain and the range of f . 10 Noise Image processing is useful for noise reduction... Common types of noise: Salt and pepper noise: contains random occurrences of black and white pixels Impulse noise: contains random occurrences of white pixels Gaussian noise: variations in intensity drawn from a Gaussian normal distribution 11 Ideal noise reduction Given a camera and a still scene, how can you reduce noise? 12 Ideal noise reduction Given a camera and a still scene, how can you reduce noise? 13

Practical noise reduction How can we smooth away noise in a single image? 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 100 130 110 120 110 0 0 0 0 0 110 90 100 90 100 0 0 0 0 0 130 100 90 130 110 0

0 0 0 0 120 100 130 110 120 0 0 0 0 0 90 110 80 120 100 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 14 Mean filtering 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0

0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0

0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 15 Mean filtering 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 0 0 0

0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 0 0 0 0 0

90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 Mean filtering 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90

90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0

0 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 10 20 30 30 30 20 10 0 20 40 60 60 60 40 20 0 30 60 90 90 90 60 30 0 30 50 80 80 90 60 30 0 30 50 80 80 90 60 30 0 20 30 50 50 60 40 20 10 20 30 30 30 30 20 10 10 10 10 0 17 0 0 0

0 Mean filtering When does this work? F x I x N x 1 G x F x 1 F x F x 1 3 1 G x I x 1 I x I x 1 3 N x 3 1 E G x I x 1 I x I x 1 3 When does E(G(x)) = I(x)? 3I x I x 1 I x I x 1 I x 1 2 I x I x 1 0 18 Mean filtering When does this work? 19 Effect of mean filters 20 Cross-correlation filtering Lets write this down as an equation. Assume the averaging window is (2k+1)x(2k+1): We can generalize this idea by allowing different weights for different neighboring pixels:

This is called a cross-correlation operation and written: H is called the filter, kernel, or mask. The above allows negative filter indices. When you implement need to use: 21 Mean kernel Whats the kernel for a 3x3 mean filter? 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0

0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0

0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 22 Gaussian Filtering A Gaussian kernel gives less weight to pixels further from the center of the window 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 0 0 0

0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 1

2 4 2 1 2 1 This kernel is an approximation of a Gaussian function: What happens if you increase ? 23 = 1 pixel = 5 pixels = 10 pixels = 30 pixels 24 Mean vs. Gaussian filtering 25 Filtering an impulse a b

c d e f g h i 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 26 Convolution A convolution operation is a crosscorrelation where the filter is flipped both horizontally and vertically before being

applied to the image: It is written: Suppose H is a Gaussian or mean kernel. How does convolution differ from crosscorrelation? Suppose F is an impulse function (previous slide) What will G look like? 27 Continuous Filters We can also apply filters to continuous images. In the case of cross correlation: In the case of convolution: Note that the image and filter are infinite. 28 Median filters A Median Filter operates over a window by selecting the median intensity in the window. What advantage does a median filter have over a mean filter? Is a median filter a kind of convolution? 29 Comparison: salt and pepper noise 30

Comparison: Gaussian noise 31 Linear filters: example * 0 0 0 0 1 0 0 0 0 Original = Identical image 32 Source: D. Lowe Linear filters: example * 0 0 0 1 0 0 0 0 0 Original = Shift by 1 pixel 33 Source: D. Lowe

Linear filters: example * 1 1 1 1 1 1 1 1 1 Original = Blur (with a mean filter) 34 Source: D. Lowe Linear filters: example * 0 0 0 0 2 0 0 0 0 - 1 1 1 1 1 1 1 1 1 Original = 35 Sharpening

filter (accentuates edges) Source: D. Lowe Sharpening 36 Source: D. Lowe Sharpening What does blurring take away? = detail smoothed (5x5) original Lets add it back: = + original detail 37 sharpened Sharpen filter

unfiltered filtered 38 Sharpen filter blurred image image scaled impulse unit impulse (identity) Gaussian 39 Laplacian of Gaussian Enhance it!