Annotation of Named Entities in the Gaming domain

Abstract

This paper aims to analyse the effects of including gaming entities in the performance of the NER system, for the English language and in a machine translation industrial context of customer support content. To identify and classify gaming entities (by the Named Entity Recognition (NER) model), three new categories were created and added to the already used annotation typology: GAME NAME, GAME FEATURE and GAME CURRENCY. A set of reference annotations (gold standard) was also developed, allowing not only the training of the NER system but also the evaluation of its performance and accuracy in a more objective way, namely by counting the number of entities that the system identifies and categorises correctly. In the scope of this work, 6618 sentences from 7 gaming clients were manually annotated, constituting the gold standard which was then used to train and evaluate the NER system. The objective of the experiments was to assess whether the existing NER system improved its performance when trained with the gold standard created specifically for the gaming domain and if it could handle the new gaming categories added to the typology by identifying and categorizing them correctly. The results of both experiments were auspicious and positive, demonstrating the relevance of greater investment in domain-specific entity recognition, namely in the context of customer service text processing.