Devoir Surveillé in Fondamentaux du Deep Learning

1- How does splitting a dataset into train, dev and test sets help identify overfitting? (2pts).

2- You are designing a deep learning system to detect driver fatigue in cars. It is crucial

that that your model detects fatigue, to prevent any accidents. Which of the following is

the most appropriate evaluation metric: Accuracy, Precision, Recall, Loss Value.

Explain your choice (1pt).

3- Which of the following techniques does NOT prevent a model from overfitting? (1pts).

Data augmentation

Dropout

Early stopping

(i)

(ii)

(iii)

(iv) None of the above

4- You are given a dataset of 10×10 grayscale images. Your goal is to build a 5-class

classifier. You have to adopt one of the following two options (2pts):

 The input is flattened into a 100-dimensional vector, followed by a fully-

connected layer with 5 neurons •

 The input is directly given to a convolutional layer with five 10 × 10 filters

5- You are doing full batch gradient descent using the entire training set (not stochastic

gradient descent). Is it necessary to shuffle the training data? Explain your answer.

(2pts).

Question 2 (Architectures and training, 12 points):

A. Convolution:

Table 1 depicts two matrices. One of them (5x5 one) represents an image. The second (3x3

matrix) represents a convolution kernel the padding is 'valid'. (Consider the bias term to be

zero).

(a) How many values will be generated if we forward propagate the image over the given

convolution kernel? (01pts)

(b) Calculate these values. (03pts)

1

(a) Consider the figure below (1pts):

What is the output shape after performing the convolution step in Figure 1? Write your answer

in the following format: (nH, nW, nc).

B. Linearity (1 pts):

You are given the following piece of code for forward propagation through a single hidden

layer in a neural network. This layer uses the sigmoid activation. Identify and correct the error.

C. Convolutional neural network (04 pts):

Consider the convolutional neural network defined by the layers in the left column below. Fill

in the shape of the output volume and the number of parameters at each layer. You can write

the activation shapes in the format (H,W,C), where H,W,C are the height, width and channel

dimensions, respectively. Unless specified, assume padding 1, stride 1 where appropriate.

Notation:

• CONVx-N denotes a convolutional layer with N filters with height and width equal to x.

• POOL-n denotes a n×n max-pooling layer with stride of n and 0 padding.

• FLATTEN flattens its inputs, identical to torch.nn.flatten / tf.layers.flatten

2

• FC-N denotes a fully-connected layer with N neurons

D. Training (02pts):

You would like to train a dog/cat image classifier using mini-batch gradient descent. You

have already split your dataset into train, dev and test sets. The classes are balanced. You

realize that within the training set, the images are ordered in such a way that all the dog

images come first and all the cat images come after. A friend tells you: ”you absolutely need

to shuffle your training set before the training procedure.”

Is your friend right? Explain.

3