
Microsoft's DeBERTa-v3-small Beats RoBERTa Using Half the Parameters
Takeaways − DeBERTa-v3-small is a 44M-parameter English encoder with a 128K vocabulary from Microsoft Research. Uses ELECTRA-style replaced-token-detection training instead of masked language modeling for better sample efficiency.











