Supervised learning is a fundamental concept in machine learning where labelled datasets are used to train computer models.
Models learn from pre-labelled training datasets provided by human annotators. They are then tested with labelled data to evaluate their accuracy using metrics like loss functions and accuracy.
The key steps in the supervised learning process are as follows:
- Data Collection: Gathering and labelling training data that is representative of the problem space.
- Data Splitting: Typically, data is split into a training set used for learning, a validation set used for hyperparameter tuning and model selection, and a test set for final evaluation.
- Algorithm Selection: The choice of algorithm depends on the nature of the task (e.g., classification, regression) and the characteristics of the data.
- Training Procedure: Involves adjusting the model's parameters to minimize the loss function. This process is iterative and often involves techniques such as gradient descent.
- Model Evaluation: Checking model accuracy using test datasets. Model evaluation will depend on the task’s nature and requirements.
Supervised Fine-tuning
Fine-tuning is a specific application of supervised learning in the context of LLMs.
It involves adjusting the weights of a pre-trained model so that it can perform well on tasks it wasn't specifically trained for during its initial pre-training phase.
This is achieved by continuing the training process, but now with a focus on minimizing the loss function on the new, task-specific dataset.
The model learns to apply its general understanding of language, acquired during pre-training, to the nuances of the specific task, guided by the labelled examples provided in the fine-tuning stage.
Sources
- Fine-Tuning LLMs: Overview, Methods & Best Practices (turing.com)
- https://huggingface.co/blog/rishiraj/finetune-llms
- https://www.datacamp.com/tutorial/fine-tuning-large-language-models