-[2.2 Model Structure](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/code_overview.md#2.2)
-[2.3 Loss Function](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/code_overview.md#2.3)
-[2.4 Optimizer, Learning Rate Decay, and Weight Decay](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/code_overview.md#2.4)
-[2.5 Evaluation During Training](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/code_overview.md#2.5)
-[2.6 Model Saving](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/code_overview.md#2.6)
-[2.7 Model Pruning and Quantification](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/code_overview.md#2.7)
-[Codes and Methods for Inference and Deployment](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/code_overview.md#3)
## 1 Overview of Code and Content
The main code and content structure of PaddleClas are as follows:
- benchmark: The folder stores shell scripts to test the speed metrics of different models in PaddleClas, such as single-card training speed metrics, multi-card training speed metrics, etc.
- dataset: The folder stores datasets and the scripts used to process datasets. The scripts are responsible for processing the dataset into a suitable format for Dataloader.
- deploy: Deploy the core code, the folder stores the deployment tools, which support python/cpp inference, Hub Serveing, Paddle Lite, Slim offline quantification and other deployment methods.
- ppcls: Train the core code, the folder holds the main body of the PaddleClas framework. It also has configuration files, and specific code of model training, evaluation, inference, dynamic to static export, etc.
- tools: The file contains the entry functions and scripts for training, evaluation, inference, and dynamic to static export.
- The requirements.txt file is adopted to install the dependencies for PaddleClas. Use pip for upgrading, installation, and application.
- tests: Full-link tests of PaddleClas models from training to prediction to verify that whether each function works properly.
## 2 Training Module
The training of deep learning model mainly contains data, model structure, loss function, strategies such as optimizer, learning rate decay, and weight decay strategy, etc., which are explained below.
## 2.1 Data
For supervised tasks, the training data generally contains the original data and its annotation. In a single-label-based image classification task, the raw data refers to the image data, while the annotation is the class to which the image data belongs. In PaddleClas, a label file, in the following format, is required for training, with each row containing one training sample and separated by a separator (space by default), representing the image path and the class label respectively.
```
train/n01440764/n01440764_10026.JPEG 0
train/n01440764/n01440764_10027.JPEG 0
```
The code `ppcls/data/dataloader/common_dataset.py` contains the `CommonDataset` class inherited from `paddle.io.Dataset`, which is a dataset class that can index and fetch a given sample by a key value. Dataset classes such as `ImageNetDataset`, `LogoDataset`, `CommonDataset`, etc. are all inherited from this class.
For the read-in data, the raw image needs to be transformed by data conversion. The standard data preprocessing during training contains `DecodeImage`, `RandCropImage`, `RandFlipImage`, `NormalizeImage`, and `ToCHWImage`. The data preprocessing is mainly in the `transforms` field, which is presented in a list, and then converts the data in order, as reflected in the configuration file below.
PaddleClas also contains `AutoAugment`, `RandAugment`, and other data augmentation methods, which can also be configured in the configuration file and thus added to the data preprocessing of the training. Each data conversion method is implemented as a class for easy migration and reuse. For more specific implementation of data processing, please refer to the code under `ppcls/data/preprocess/ops/`.
You can also use methods such as mixup or cutmix to augment the data that make up a batch. PaddleClas integrates `MixupOperator`, `CutmixOperator`, `FmixOperator`, and other batch-based data augmentation methods, which can be configured by deploying the mix parameter in the configuration file. For more specific implementation, please refer to `ppcls/data/preprocess /batch_ops/batch_operators.py`.
In image classification, the data post-processing is mainly `argmax` operation, which is not elaborated here.
## 2.2 Model Structure
The model in the configuration file is structured as follows:
```
Arch:
name: ResNet50
class_num: 1000
pretrained: False
use_ssld: False
```
`Arch.name` indicates the name of the model, `Arch.pretrained` whether to add a pre-trained model, and `use_ssld` whether to use a pre-trained model based on `SSLD` knowledge distillation. All model names are defined in `ppcls/arch/backbone/__init__.py`.
Correspondingly, the model object is created in `ppcls/arch/__init__.py` with the `build_model` method.
```
def build_model(config):
config = copy.deepcopy(config)
model_type = config.pop("name")
mod = importlib.import_module(__name__)
arch = getattr(mod, model_type)(**config)
return arch
```
## 2.3 Loss Function
PaddleClas contains `CELoss` , `JSDivLoss`, `TripletLoss`, `CenterLoss` and other loss functions, all defined in `ppcls/loss`.
In the `ppcls/loss/__init__.py` file, `CombinedLoss` is used to construct and combine loss functions. The loss functions and calculation methods required in different training strategies are disparate, and the following factors are considered by PaddleClas in the construction of the loss function.
1. whether to use label smooth
2. whether to use mixup or cutmix
3. whether to use distillation method for training
4. whether to train metric learning
The user can specify the type and weight of the loss function in the configuration file, such as adding TripletLossV2 to the training, the configuration file is as follows:
```
Loss:
Train:
- CELoss:
weight: 1.0
- TripletLossV2:
weight: 1.0
margin: 0.5
```
## 2.4 Optimizer, Learning Rate Decay, and Weight Decay
In image classification tasks, `Momentum` is a commonly used optimizer, and several optimizer strategies such as `Momentum`, `RMSProp`, `Adam`, and `AdamW` are provided in PaddleClas.
The weight decay strategy is a common regularization method, mainly adopted to prevent model overfitting. Two weight decay strategies, `L1Decay` and `L2Decay`, are provided in PaddleClas.
Learning rate decay is an essential training method for accuracy improvement in image classification tasks. PaddleClas currently supports `Cosine`, `Piecewise`, `Linear`, and other learning rate decay strategies.
In the configuration file, the optimizer, weight decay, and learning rate decay strategies can be configured with the following fields.
```
Optimizer:
name: Momentum
momentum: 0.9
lr:
name: Piecewise
learning_rate: 0.1
decay_epochs: [30, 60, 90]
values: [0.1, 0.01, 0.001, 0.0001]
regularizer:
name: 'L2'
coeff: 0.0001
```
Employ `build_optimizer` in `ppcls/optimizer/__init__.py` to create the optimizer and learning rate objects.
Different optimizers and weight decay strategies are implemented as classes, which can be found in the file `ppcls/optimizer/optimizer.py`; different learning rate decay strategies can be found in the file `ppcls/optimizer/learning_rate.py`.
## 2.5 Evaluation During Training
When training the model, you can set the interval of model saving, or you can evaluate the validation set every several epochs so that the model with the best accuracy can be saved. Follow the fields below to configure.
```
Global:
save_interval: 1 # epoch interval of model saving
eval_during_train: True # whether evaluate during training
eval_interval: 1 # epoch interval of evaluation
```
## 2.6 Model Saving
The model is saved through the `paddle.save()` function of the Paddle framework. The dynamic graph version of the model is saved in the form of a dictionary to facilitate further training. The specific implementation is as follows:
```
def save_model(program, model_path, epoch_id, prefix='ppcls'): model_path = os.path.join(model_path, str(epoch_id)) _mkdir_if_not_exist(model_path) model_prefix = os.path.join(model_path, prefix) paddle.static.save(program, model_prefix) logger.info( logger.coloring("Already save model in {}".format(model_path), "HEADER"))
```
When saving, there are two things to keep in mind:
1. Only save the model on node 0, otherwise, if all nodes save models to the same path, a file conflict may occur during multi-card training when multiple nodes write files, preventing the final saved model from being loaded correctly.
2. Optimizer parameters also need to be saved to facilitate subsequent loading of breakpoints for training.
- Model pruning and quantification training
If you want to conduct compression training, please configure with the following fields.
## 2.7 Model Pruning and Quantification
1. Model pruning：
```
Slim: prune: name: fpgm pruned_ratio: 0.3
```
2. Model quantification：
```
Slim: quant: name: pact
```
For details of the training method, see [Pruning and Quantification Application](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/model_prune_ quantization.md), and the algorithm is described in [Pruning and Quantification algorithms](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/algorithm_introduction/ model_prune_quantization.md).
## 3 Codes and Methods for Inference and Deployment
- If you wish to quantify the classification model offline, please refer to the [Model Pruning and Quantification Tutorial](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/model_) for offline quantification.
- If you wish to use python for server-side deployment, please refer to [Python Inference Tutorial](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/inference_ deployment/python_deploy.md).
- If you wish to use cpp for server-side deployment, please refer to [Cpp Inference Tutorial](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/inference_ deployment/cpp_deploy.md).
- If you wish to deploy the classification model as a service, please refer to the [Hub Serving Inference Deployment Tutorial](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/inference_deployment/ paddle_hub_serving_deploy.md).
- If you wish to use classification models for inference on mobile, please refer to [PaddleLite Inference Deployment Tutorial](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/inference_ deployment/paddle_lite_deploy.md)
- If you wish to use the whl package for inference of classification models, please refer to [whl Package Inference](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/inference_deployment/whl_ deploy.md) .
-[1. How to Contribute Code](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/how_to_contribute.md#1)
-[1.1 Branches of PaddleClas](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/how_to_contribute.md#1.1)
-[1.2 Commit Code to PaddleClas](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/how_to_contribute.md#1.2)
-[1.2.1 Codes of Fork and Clone](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/how_to_contribute.md#1.2.1)
-[1.2.2 Connect to the Remote Repository](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/how_to_contribute.md#1.2.2)
-[1.2.3 Create the Local Branch](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/how_to_contribute.md#1.2.3)
-[1.2.5 Modify and Commit Code](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/how_to_contribute.md#1.2.5)
-[1.2.6 Keep the Local Repository Updated](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/how_to_contribute.md#1.2.6)
-[1.2.7 Push to Remote Repository](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/advanced_tutorials/how_to_contribute.md#1.2.7)
PaddleClas will maintain the following two branches:
- release/x.x series: A stable release branch, which will be tagged with the release version of Paddle in due course. The latest branch and the default one is the release/2.3, which is compatible with Paddle v2.1.0. The branch of release/x.x series will continue to grow with future iteration, and the latest release will be maintained by default, while the former one will fix bugs with no other branches covered.
- develop : A development branch, which is adapted to the develop version of Paddle and is mainly used for developing new functions. A good choice for secondary development. To ensure that the develop branch can pull out the release/x.x when needed, only the API that is valid in Paddle's latest release branch can be adopted for its code. In other words, if a new API has been developed in this branch but not yet in the release, please do not use it in PaddleClas. Apart from that, features that do not involve the performance optimizations, parameter adjustments, and policy updates of the API can be developed normally.
The historical branches of PaddleClas will not be maintained, but will be remained for the existing users.
- release/static: This branch was used for static graph development and testing, and is currently compatible with >=1.7 versions of Paddle. It is still practicable for the special need of adapting an old version of Paddle, but the code will not be updated except for bug fixing.
- dygraph-dev: This branch will no longer be maintained and accept no new code. Please transfer to the develop branch as soon as possible.
PaddleClas welcomes code contributions to the repo, and the basic process is detailed in the next part.
### 1.2 Commit the Code to PaddleClas
#### 1.2.1 Codes of Fork and Clone
- Skip to the home page of [PaddleClas GitHub](https://github.com/PaddlePaddle/PaddleClas) and click the Fork button to generate a repository in your own directory, such as `https://github.com/USERNAME/ PaddleClas`.
The above information only contains the cloned remote repository, which is the PaddleClas under your username. Then we create a remote host of the original PaddleClas repository named upstream.
Adopt `git remote -v` to view the current information of the remote repository, and 2 remote repositories including origin and upstream can be found, as shown below.
This is mainly to keep the local repository updated when committing a pull request (PR).
#### 1.2.3 Create the Local Branch
Run the following command to create a new local branch based on the current one.
```
git checkout -b new_branch
```
Or you can create new ones based on remote or upstream branches.
```
# Create the new_branch based on the develope of origin (unser remote repository)
git checkout -b new_branch origin/develop
# Create the new_branch base on the develope of upstream
# If you need to create a new branch from upstream, please first employ git fetch upstream to fetch the upstream code
git checkout -b new_branch upstream/develop
```
Then it is shown that it has switched to the new branch with the following output:
```
Branch new_branch set up to track remote branch develop from upstream.
Switched to a new branch 'new_branch'
```
#### 1.2.4 Employ Pre-commit Hook
Paddle developers adopt the pre-commit tool to manage Git pre-commit hooks. It helps us format the source code (C++, Python) and automatically check basic issues before committing (e.g., one EOL per file, no large files added to Git, etc.).
The pre-commit test is part of the unit tests in Travis-CI, and PRs that do not satisfy the hook cannot be committed to PaddleClas. Please install it first and run it in the current directory:
```
pip install pre-commit
pre-commit install
```
-**Note**
3. Paddle uses clang-format to format C/C++ source code, please make sure `clang-format` has a version of 3.8 or higher.
4.`yapf` installed by `pip install pre-commit` and `conda install -c conda-forge pre-commit` is slightly different, and the former one is chosen by PaddleClas developers.
#### 1.2.5 Modify and Commit Code
You can check the changed files via `git status`. Follow the steps below to commit the `README.md` of PaddleClas after modification:
```
git add README.mdpre-commit
```
Repeat the above steps until the pre-commit format check does not report an error, as shown below.
Get the latest code for upstream and update the current branch. The upstream here is from the `Connecting to a remote repository` part in section 1.2.
```
git fetch upstream# If you want to commit to another branch, please pull the code from another branch of upstream, in this case it is developgit pull upstream develop
```
#### 1.2.7 Push to Remote Repository
```
git push origin new_branch
```
#### 1.2.8 Commit Pull Request
Click new pull request and select the local branch and the target branch, as shown in the following figure. In the description of the PR, fill out what the PR accomplishes. Next, wait for the review, and if any changes are required, update the corresponding branch in origin by referring to the above steps.
- When you first commit a Pull Request to PaddlePaddle, you will be required to sign a CLA (Contributor License Agreement) to ensure that your code can be merged, please follow the step below to sign CLA:
1. Please examine the Check section of your PR, find license/cla, and click the detail on the right side to enter the CLA website
2. Click `Sign in with GitHub to agree` on the CLA website, and you will be redirected back to your Pull Request page when you are done.
#### 1.2.10 Delete Branch
- Delete remote branch
When the PR is merged into the main repository, you can delete the remote branch from the PR page.
You can also delete the remote branch using `git push origin :branch name`, e.g.
```
git push origin :new_branch
```
- Delete local branch
```
# Switch to the develop branch, otherwise the current branch cannot be deletedgit checkout develop# Delete new_branchgit branch -D new_branch
```
#### 1.2.11 Conventions
To help official maintainers focus on the code itself when reviewing it, please adhere to the following conventions each time you commit code:
1）Please pass the unit test in Travis-CI first. Otherwise, the submitted code may have problems and usually receive no official review.
2）Before committing a Pull Request:
Note the number of commits.
Reason: If only one file is modified but more than a dozen commits are committed with a few changes for each, this may overwhelm the reviewer for they need to check each and every commit for specific changes, including the case that the changes between commits overwrite each other.
Recommendation: Minimize the number of commits each time, and add the last commit with `git commit --amend`. For multiple commits that have been pushed to a remote repository, please refer to [squash commits after push](https://stackoverflow.com/questions/5667884/how-to-squash-commits-in-git-after- they-have-been-pushed).
Please pay attention to the name of each commit: it should reflect the content of the current commit without being too casual.
3）If an issue is resolved, please add `fix #issue_number` to the first comment box of the Pull Request, so that the corresponding issue will be closed automatically when the Pull Request is merged. Please choose the appropriate term with keywords such as close, closes, closed, fix, fixes, fixed, resolve, resolves, resolved, please choose the appropriate term. See details in [Closing issues via commit messages](https://help.github.com/articles/closing-issues-via-commit-messages).
In addition, please stick to the following convention to respond to reviewers' comments:
1）Every review comment from the official maintainer is expected to be answered, which will better enhance the contribution of the open source community.
- If you agree with the review and finish the corresponding modification, please simply return Done;
- If you disagree with the review, please give your reasons.
2）If there are plenty of review comments,
- Please present the revision in general.
- Please reply with `start a review` instead of a direct approach, for it may be overwhelming to receive the email of every reply.
## 2. Summary
- The open source community relies on the contributions and feedback of developers and users. We highly appreciate that and look forward to your valuable comments and Pull Requests to PaddleClas in the hope that together we can build a leading practical and comprehensive code repository for image recognition!
## 3. References
1.[Guide to PaddlePaddle Local Development](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/guides/08_contribution/index_cn.html)
