ConvNets
Convolutions and architecture
Dr. Haithem Hermessi
•
•
Sr. Compu te r Vision Engine e r @ SCYLLA
AI research Scientist @ LIMTIC - University of Tunis El Manar
Motivation
We know it is good to learn a small model.
From this fully connected model, do we really need all the edges?
Can some of these be shared?
Shared weights, why?
Motivation
Consider learning an image:
• Some patterns are much smaller than the whole
image
Can represent a small region with fewer parameters
“beak” detector
Motivation
Same pattern appears in different places:
• They can be compressed!
• What about training a lot of such “small” detectors
• Each detector must “move around”.
“upper-left beak”
detector
They can be compressed
to the same parameters.
“middle beak”
detector
Convolutional layer
A CNN is a neural network with some convolutional layers (and some other layers).
A convolutional layer has a number of filters that does convolutional operation.
Beak detector
A filter
Convolution
Convolution
stride=1 (jumping step)
1
0
0
1
0
0
0
1
0
0
1
0
0
0
1
0
0
1
0
0
1
0
0
0
0
1
0
1
1
1
1
0
0
0
0
0
6 x 6 image
1
-1
-1 -1
-1
1
-1 -1
1
Filter 1
Dot
product
3
-1
…
Convolution
If stride=2
1
0
0
1
0
0
0
1
0
0
1
0
0
0
1
0
0
1
0
0
1
0
0
0
0
1
0
1
1
1
1
0
0
0
0
0
6 x 6 image
Filter 1
1
-1
-1 -1
-1
1
-1 -1
Publicité
1
*
=
3
-3
Convolution
stride=1
1
0
0
1
0
0
0
1
0
0
1
0
0
0
1
0
0
1
0
0
1
0
0
0
0
1
0
1
1
1
1
0
0
0
0
0
6 x 6 image
Filter 1
1
-1 -1
-1
1
-1 -1
-1
1
=
*
Feature map 1
3
-3
-3
3
-1
1
-3
-2
-3
0
0
-2
-1
-3
1
-1
Convolution
stride=1
1
0
0
1
0
0
0
1
0
0
1
0
0
0
1
0
0
1
0
0
1
0
0
0
0
1
0
1
1
1
1
0
0
0
0
0
6 x 6 image
Repeat this for each filter
Filter 2
-1
1
-1
-1
-1
1
1
-1
-1
*
=
3
-1
-3
-1
-3
-1
3
-1
-1
-1
-3
Publicité
-1
-1
-1
0
1
-2
-1
Feature
Maps
0
-2
-1
-3
-2
0
-2
-4
-3
1
1
1
-1
3
Two 4 x 4 images
Forming 2 x 4 x 4 matrix
Convolution
Color image
-1 -1
1
-1 -1
1
-1 -1
1
-1
1
-1
-1
1
-1
-1
1
-1
-1 -1
1
1
-1 -1
1
-1 -1
Filter 1
-1
-1
-1
-1
-1
-1
-1
-1
-1
1
1
1
1
1
1
1
1
1
-1
-1
-1
-1
-1
-1
-1
-1
-1
Filter 2
1
0
0
1
0
0
1
0
0
1
0
0
0
1
1
0
0
0
0
1
1
0
0
0
0
1
0
0
1
0
0
0
0
1
1
0
0
0
0
1
1
0
0
0
1
0
0
1
0
0
0
0
1
1
0
0
0
Publicité
0
0
1
0
0
1
0
0
0
0
0
1
0
0
1
1
0
1
0
1
0
0
1
0
1
1
1
1
0
0
1
0
0
0
1
0
1
0
1
1
0
0
0
0
0
1
0
0
0
0
0
Convolution v.s. Fully Connected
1 0 0 0 0 1
0 1 0 0 1 0
0 0 1 1 0 0
1 0 0 0 1 0
0 1 0 0 1 0
0 0 1 0 1 0
image
1 -1 -1
-1 1 -1
-1 -1 1
-1
-1
-1
1
1
1
-1
-1
-1
convolution
Fully-connected:
1 0 0 0 0 1
0 1 0 0 1 0
0 0 1 1 0 0
1 0 0 0 1 0
0 1 0 0 1 0
0 0 1 0 1 0
……
……
1x2x36xConvolution
Convolution
Convnets
The whole CNN
cat or dog ……
Fully Connected
Feedforward network
Flattened
Convolution
Max Pooling
Convolution
Max Pooling
Can repeat
many times
Pooling
Max Pooling:
1 -1 -1
-1 1 -1
-1 -1 1
Filter 1
3
-3
-3
3
-1
1
-3
-2
-3
0
0
-2
-1
-3
1
-1
-1 1 -1
-1 1 -1
-1 1 -1
Filter 2
-1
-1
-1
-1
-1
-1
-1
0
-1
-2
-2
-4
-1
Publicité
1
1
3
Pooling
Why Pooling?
Subsampling pixels will not change the object
bird
bird
Subsampling
We can subsample the pixels to make image smaller
fewer parameters to characterize the image
Max Pooling
1
0
0
1
0
0
0
1
0
0
1
0
0
1
0
0
0
0
1
0
0
0
1
0
6 x 6 image
0
1
0
1
1
1
1
0
0
0
0
0
Conv
Max
Pooling
New image
but smaller
3
-1
3
0
0
1
1
3
2 x 2 image
Each filter
is a channel
ConvNets architectures
A CNN compresses a fully connected network in two ways:
• Reducing number of connections
• Shared weights on the edges
• Max pooling further reduces the complexity
ConvNets architectures
The whole CNN
3
-1
3
0
0
1
1
3
A new image
Convolution
Max Pooling
Can repeat many times
Smaller than the original image
Convolution
The number of channels is the number of filters
Max Pooling
ConvNets architectures
Flattening
3
-
1
3
0
0
1
1
3
Flattene
d
3
0
1
3
-
1
1
0
3
Fully Connected
Feedforward
network
How we code it:
How we code it:
How we code it:
CNN in speech recognition
CNN
The filters move in the frequency direction.
y
c
n
e
u
q
e
r
F
Image
Time
Spectrogram
CNN in text classification
?