A good model should be able to recognize objects of different categories correctly, and meanwhile can correctly classify images taken from different points of view, under different illuminations, with object distortion or partial occlusion (we call these image disturbance). Figure 3 show some images with various disturbance. A good model should be able to classify these images correctly like humans.
A good model should be able to recognize objects of different categories correctly, and meanwhile can correctly classify images taken from different points of view, under different illuminations, with object distortion or partial occlusion (we call these image disturbance). Figure 3 show some images with various disturbance. A good model should be able to classify these images correctly like humans.
Traditional CNNs consist of convolutional and fully-connected layers, and employ softmax multi-category classifier and cross-entropy as loss function. Figure 5 shows a typical CNN. We first introduce the common parts of a CNN.
Traditional CNNs consist of convolutional and fully-connected layers, and employ softmax multi-category classifier and cross-entropy as loss function. Figure 5 shows a typical CNN. We first introduce the common parts of a CNN.
<palign="center">
<palign="center">
<imgsrc="image/lenet.png"><br/>
<imgsrc="image/lenet_en.png"><br/>
Figure 5. A CNN example [20]
Figure 5. A CNN example [20]
输入层 ==> input layer
卷积层 ==> convolutional layer
特征图 ==> feature maps
降采样（池化）层 ==> pooling layer
全连接层 ==> fully-connected layer
</p>
</p>
- convolutional layer: It uses convolution operation to extract low-level and high-level features, and to discover local correlation and spatial invariance.
- convolutional layer: It uses convolution operation to extract low-level and high-level features, and to discover local correlation and spatial invariance.
...
@@ -113,13 +101,8 @@ NIN model has two main characteristics: 1) it replaces the single-layer convolut
...
@@ -113,13 +101,8 @@ NIN model has two main characteristics: 1) it replaces the single-layer convolut
Figure 7 depicts two Inception blocks. Figure 7(a) is the simplest design, the output of which is a concat of features from three convolutional layers and one pooling layer. The disadvantage of this design is that the pooling layer does not change the number of filters and leads to an increase of outputs. After going through several of such blocks, the number of outputs and parameters will become larger and larger, leading to higher computation complexity. To overcome this drawback, the Inception block in Figure 7(b) employs three 1x1 convolutional layers to reduce dimension or the number of channels, meanwhile improves non-linearity of the network.
Figure 7 depicts two Inception blocks. Figure 7(a) is the simplest design, the output of which is a concat of features from three convolutional layers and one pooling layer. The disadvantage of this design is that the pooling layer does not change the number of filters and leads to an increase of outputs. After going through several of such blocks, the number of outputs and parameters will become larger and larger, leading to higher computation complexity. To overcome this drawback, the Inception block in Figure 7(b) employs three 1x1 convolutional layers to reduce dimension or the number of channels, meanwhile improves non-linearity of the network.
<palign="center">
<palign="center">
<imgsrc="image/inception.png"width="800"><br/>
<imgsrc="image/inception_en.png"width="800"><br/>
Figure 7. Inception block
Figure 7. Inception block
输入层 ==> input layer
卷积层 ==> convolutional layer
最大池化层 ==> max-pooling layer
Inception简单模块 ==> Inception module, naive version
Inception含降维模块 ==> Inception module with dimensionality reduction
</p>
</p>
GoogleNet consists of multiple stacking Inception blocks followed by an avg-pooling layer as in NIN in place of by traditional fully connected layers. The difference between GoogleNet and NIN is that GoogleNet adds a fully connected layer after avg-pooling layer to output a vector of category size. Besides these two characteristics, the features from middle layers of a GoogleNet are also very discriminative. Therefore, GoogeleNet inserts two auxiliary classifiers in the model for enhancing gradient and regularization when doing backpropagating. The loss function of the whole network is the weighted sum of these three classifiers.
GoogleNet consists of multiple stacking Inception blocks followed by an avg-pooling layer as in NIN in place of by traditional fully connected layers. The difference between GoogleNet and NIN is that GoogleNet adds a fully connected layer after avg-pooling layer to output a vector of category size. Besides these two characteristics, the features from middle layers of a GoogleNet are also very discriminative. Therefore, GoogeleNet inserts two auxiliary classifiers in the model for enhancing gradient and regularization when doing backpropagating. The loss function of the whole network is the weighted sum of these three classifiers.
...
@@ -476,12 +459,8 @@ Tester.cpp:115] Test samples=10000 cost=1.99246 Eval: classification_error_eval
...
@@ -476,12 +459,8 @@ Tester.cpp:115] Test samples=10000 cost=1.99246 Eval: classification_error_eval
Figure 12 shows the curve of training error rate, which indicates it converges at Pass 200 with error rate 8.54%.
Figure 12 shows the curve of training error rate, which indicates it converges at Pass 200 with error rate 8.54%.
