Domain-Specific BERT
Learn about the domain-specific models and get detailed insight into ClinicalBERT by fine-tuning it for patient re-admission prediction tasks.
We learned how BERT is pre-trained using the general Wikipedia corpus and how we can fine-tune and use it for downstream tasks. Instead of using the BERT that is pre-trained on the general Wikipedia corpus, we can also train BERT from scratch on a domain-specific corpus. This helps the BERT model to learn embeddings specific to a domain, and it also helps in learning the domain-specific vocabulary that may not be present in the general Wikipedia corpus.
Get hands-on with 1200+ tech skills courses.