Created by: ceci3

add ofa

• ✅ support some op convert SuperNet

• ✅ convert SuperNet base class

• ✅ OFA SuperNetBase class

• ✅ demo in ofa.py and convert_supernet.py

• ✅ Support OP and elastic config: - SuperConv2D (elastic kernel, elactic channel) - SuperDepthwiseConv2D(elastic kernel) - SuperGroupConv2D (elastic kernel, elactic channel) - SuperConv2DTranspose (elastic kernel, elactic channel) - SuperDepthwiseConv2DTranspose (elastic kernel) - SuperGroupConv2DTranspose (elastic kernel, elactic channel) - SuperBatchNorm (elactic channel) - SuperInstanceNorm (elactic channel) - SuperLinear (elactic channel)

• ✅ add progressive shrinking

• ✅ add user-define/ default distill

• ✅ support user-defined elastic_depth to search space

• ✅ fix pre-channel elastic in convert

TODO in next PR：

• ⭕ search process (evolution search)
• ⭕ accuracy model
• ⭕ latency model

NOT SUPPORT NOW

• ❌ add re_organize_middle_weights to elastic_width because it only re-orginize in each MB_Block accoding to the importance of middle conv
• ❌ decorator supernet