• R
    [MoE] Moe apis (#41092) · aac7879a
    Roc 提交于
    * add random routing op
    
    add _random_routing api in utils
    
    add random routing ut
    
    * # This is a combination of 10 commits.
    # The first commit's message is:
    add expert count op
    
    add ut for expert_count
    
    # This is the 2nd commit message:
    
    update UT only for cuda
    
    # This is the 3rd commit message:
    
    fix for rocm
    
    # This is the 4th commit message:
    
    update ut
    
    # This is the 5th commit message:
    
    add moe module
    
    # This is the 6th commit message:
    
    add expert count op
    
    add ut for expert_count
    
    # This is the 7th commit message:
    
    update UT only for cuda
    
    # This is the 8th commit message:
    
    update ut
    
    # This is the 9th commit message:
    
    add moe module
    
    # This is the 10th commit message:
    
    make expert count private
    
    * add assign pos op
    
    * fix upper num name
    
    * add api _assign pos
    
    * add ut for assign pos op
    
    * update date
    
    * add op about moe gate
    
    update utils
    
    add limit by capacity op
    
    add ut for limit_by_capacity
    
    add ut for prune_gate_by_capacity
    
    add ut for limit_by_capacity
    
    add ut for prune_gate_by_capacity
    
    * fix for win
    
    * fix bugs in test_limit_by_capacity_op
    
    * update ut
    
    * update for test (timeout)
    
    * fix ut
    
    * update
    
    * update(fix) ut for win
    
    * moe apis in incubate
    
    * # This is a combination of 10 commits.
    # The first commit's message is:
    add expert count op
    
    add ut for expert_count
    
    # This is the 2nd commit message:
    
    update UT only for cuda
    
    # This is the 3rd commit message:
    
    fix for rocm
    
    # This is the 4th commit message:
    
    update ut
    
    # This is the 5th commit message:
    
    add moe module
    
    # This is the 6th commit message:
    
    add expert count op
    
    add ut for expert_count
    
    # This is the 7th commit message:
    
    update UT only for cuda
    
    # This is the 8th commit message:
    
    update ut
    
    # This is the 9th commit message:
    
    add moe module
    
    # This is the 10th commit message:
    
    make expert count private
    
    * add assign pos op
    
    * fix upper num name
    
    * add api _assign pos
    
    * add ut for assign pos op
    
    * update date
    
    * fix for win
    
    * update for test (timeout)
    
    * fix ut
    
    * update
    
    * fix ut for number count
    
    * add apis and utils
    
    * add gate apis
    
    * add moe and grad clip apis
    
    * update moe apis
    
    * add ops for moe gate
    
    * fix
    
    * update for base moe layer api
    
    * add random routing op
    
    add _random_routing api in utils
    
    add random routing ut
    
    * fix for dygraph
    
    * update with ranodm routing
    
    * update
    
    * fix ut for limit by capacity
    
    * update
    
    * update limit by capacity for easily to switch to single thread mode
    
    * update api docs
    Co-authored-by: Nhlygit66666 <2570058140@qq.com>
    aac7879a
gshard_gate.py 2.3 KB