Introduction to Computer Vision

Object Detection

Tasks in Computer Vision:

Object Detection

Tasks in Computer Vision:

Object Detection

Color features

Gradient features

Tasks in Computer Vision:

Object Detection - Color Histogram Features

Histogram

Tasks in Computer Vision:

Object Detection - Color Histogram Features

Tasks in Computer Vision:

Object Detection - Color Histogram Features

Tasks in Computer Vision:

Object Detection - Color Histogram Features

Tasks in Computer Vision:

Object Detection - Color Histogram Features

C

o

n

v

o

l

u

t

i

o

n

a

l

F

i

l

t

e

r

s

Features Detection with Convolutional Filters

?

Image is represented as matrix of pixel values… and computers are literal!

We want to be able to classify an X as an X even if it’s shifted, shrunk, rotated, deformed.

Features of X

Filters to Detect X Features

The Convolution Operation

The Convolution Operation

Suppose we want to compute the convolution of a 5x5 image and a 3x3 filter:

We slide the 3x3 filter over the input image, element-wise multiply, and add the outputs…

The Convolution Operation

We slide the 3x3 filter over the input image, element-wise multiply, and add the outputs:

The Convolution Operation

We slide the 3x3 filter over the input image, element-wise multiply, and add the outputs:

The Convolution Operation

We slide the 3x3 filter over the input image, element-wise multiply, and add the outputs:

Producing Feature Maps

Feature Extraction with Convolution

1. Apply a set of weights – a filter – to extract local features

2. Use multiple filters to extract different features

3. Spatially share parameters of each filter

C

o

n

v

o

l

u

t

i

o

n

a

l

N

e

u

r

a

l

N

e

t

w

o

r

k

s

(

C

N

N

s

)

CNNs for Classification

1. Convolution: Apply filters with learned weights to generate feature maps.

2. Non-linearity: Often ReLU.

3. Pooling: Downsampling operation on each feature map.

Train model with image data.

Learn weights of filters in convolutional layers.

Convolutional Layers: Local Connectivity

For a neuron in hidden layer:

• Take inputs from patch the neuron “sees”
• Compute weighted sum
• Apply bias

Convolutional Layers: Local Connectivity

For a neuron in hidden layer:

• Take inputs from patch the neuron “sees”
• Compute weighted sum
• Apply bias

CNNs: Spatial Arrangement of Output Volume

Layer Dimensions:

ℎ x w x d

where h and w are spatial dimensions

d (depth) = number of filters

Stride:

Filter step size

Receptive Field:

Locations in input image that

a node is path connected to

Introducing Non-Linearity

• Apply after every convolution operation (i.e., after

convolutional layers)

• ReLU: pixel-by-pixel operation that replaces all negative

values by zero. Non-linear operation

Pooling

Representation Learning in Deep CNNs

CNNs for Classification: Feature Learning

Learn features in input image through convolution

Introduce non-linearity through activation function (real-world data is non-linear!)

1.

2.

3. Reduce dimensionality and preserve spatial invariance with pooling

CNNs for Classification: Class Probabilities

• CONV and POOL layers output high-level features of input
• Fully connected layer uses these features for classifying input image
• Express output as probability of image belonging to a particular class

CNNs: Training with Backpropagation

Learn weights for convolutional filters and fully connected layers

Backpropagation: cross-entropy loss

D

e

e

p

N

e

u

r

a

l

N

e

t

A

r

c

h

i

t

e

c

t

u

r

e

s

f

o

r

C

l

a

s

s

i

f

i

c

a

t

i

o

n

LeNet

Resnet

Inception

DenseNet

D

e

e

p

N

e

u

r

a

l

N

e

t

A

r

c

h

i

t

e

c

t

u

r

e

s

f

o

r

O

t

h

e

r

T

a

s

k

s

Different Tasks in Deep Learning for

computer vision

Image classification

Object Detection

The network tells if the image contains a specific object or not

The Network localizes and detects each object in the image

Car

Not a car

Different Tasks in Deep Learning for

computer vision

Semantic Segmentation

Instance Segmentation

The network assigns a class for each pixel in the image

The network detects the objects then provides the semantic

segmentation of each one of them

Object Detection:

You Only Look Once (YOLO)

Object Detection:

R-CNN

Object Detection:

Faster R-CNN

Semantic Segmentation:

U-Net

Instance Segmentation:

Mask R-CNN