LLMs are pre-trained on a large text corpus to learn general language features and representations.
The process relies on unsupervised methods to enable the model to learn from huge amounts of unlabelled data.
Some of the largest data sources curated for this task are freely available for use on the internet.
The pre-trained model will then serve as the ‘foundation’ for a domain-specific LLM application which will involve a different class of data requirements.