In the context of fine-tuning LLM applications to perform well in a particular domain, it is helpful to distinguish between ‘task’ domain and ‘industry’ domain.
Particular task domains - such as Q&A chatbots or document summarisation - require training data with embedded patterns that, with a high degree of accuracy and relevance, showcase the desired features of the task output. Such as the conversational style of the chatbot, or the typical length of the summary.
Particular industry domains - such as law or medicine - also require data that is relevant and accurate. However the success criteria here will relate to the knowledge and best practices of the given industry. Such as the correct application of the law to the fact pattern, or the correct medical diagnosis in response to a given set of symptoms.