paddleslim.dist package

Submodules

paddleslim.dist.single_distiller module

paddleslim.dist.single_distiller.fsp_loss(teacher_var1_name, teacher_var2_name, student_var1_name, student_var2_name, program=None)

Combine variables from student model and teacher model by fsp-loss.

Parameters:
  • teacher_var1_name (str) – The name of teacher_var1.
  • teacher_var2_name (str) – The name of teacher_var2. Except for the second dimension, all other dimensions should be consistent with teacher_var1.
  • student_var1_name (str) – The name of student_var1.
  • student_var2_name (str) – The name of student_var2. Except for the second dimension, all other dimensions should be consistent with student_var1.
  • program (Program) – The input distiller program. If not specified, the default program will be used. Default: None
Returns:

fsp distiller loss.
Return type:

Variable
paddleslim.dist.single_distiller.l2_loss(teacher_var_name, student_var_name, program=None)

Combine variables from student model and teacher model by l2-loss.

Parameters:
  • teacher_var_name (str) – The name of teacher_var.
  • student_var_name (str) – The name of student_var.
  • program (Program) – The input distiller program. If not specified, the default program will be used. Default: None
Returns:

l2 distiller loss.
Return type:

Variable
paddleslim.dist.single_distiller.loss(loss_func, program=None, **kwargs)

Combine variables from student model and teacher model by self defined loss.

Parameters:
  • program (Program) – The input distiller program. If not specified, the default program will be used. Default: None
  • loss_func (function) – The user self defined loss function.
Returns:

self defined distiller loss.
Return type:

Variable
paddleslim.dist.single_distiller.merge(teacher_program, student_program, data_name_map, place, scope=None, name_prefix='teacher_')

Merge teacher program into student program and add a uniform prefix to the names of all vars in teacher program

Parameters:
  • teacher_program (Program) – The input teacher model paddle program
  • student_program (Program) – The input student model paddle program
  • data_map_map (dict) – Mapping of teacher input interface name and student input interface name, where key of dict is the input name of teacher_program, and value is the input name of student_program.
  • place (fluid.CPUPlace()|fluid.CUDAPlace(N)) – This parameter represents paddle run on which device.
  • scope (Scope) – This parameter indicates the variable scope used by the program. If not specified, the default global scope will be used. Default: None
  • name_prefix (str) – Name prefix added for all vars of the teacher program. Default: ‘teacher_‘
Returns:

None
paddleslim.dist.single_distiller.soft_label_loss(teacher_var_name, student_var_name, program=None, teacher_temperature=1.0, student_temperature=1.0)

Combine variables from student model and teacher model by soft-label-loss.

Parameters:
  • teacher_var_name (str) – The name of teacher_var.
  • student_var_name (str) – The name of student_var.
  • program (Program) – The input distiller program. If not specified, the default program will be used. Default: None
  • teacher_temperature (float) – Temperature used to divide teacher_feature_map before softmax. Default: 1.0
  • student_temperature (float) – Temperature used to divide student_feature_map before softmax. Default: 1.0
Returns:

l2 distiller loss.
Return type:

Variable