Research Article
Training Method and Device of Chemical Industry Chinese Language Model Based on Knowledge Distillation
Table 1
The description of previous research on language models.
| Feature-based methods | Fine-tuning methods |
| Context-independent word representation | Pretraining a language model on a large corpus with an unsupervised objective and then fine-tuning the model with in-domain labeled data | Sentence-level representation | Contextualized word representation |
|
|