MOE pruning

knowledge

Identity()

model.fc2 = nn.Identity(): replace fc2 as identity, which just return what it gets, do that may be you want to disable that layer.

pruned_model.fc1.weight.copy_(model.fc1.weight): copy model.fc1.weight to pruning_model.fc1.weight