Ampliando modelos de Image Captioning em português através das informações linguísticas.

Abstract

The increase in the number of applications that require accessibility, information retrieval and human-computer interaction has culminated in a growing need for automated generation of the description of an image. This automated description requires an identification of the scenario, characters and objects present and how these elements relate to each other. From these elements it becomes possible to generate a sentence in natural language describing the content of the image. The development of methods capable of automatically generating a sentence that describes an image permeates a research area called Image Captioning. Most research and datasets in the Image Captioning area focus on the English language, developing models and building efficient state-of-the-art resources. Languages with few resources for development, such as Portuguese, require more research to achieve a descriptive and understandable sentence. However, only the agglomeration of several objects contained in the image does not generate a descriptive sentence of a scene. In this context, this work proposes the analysis and incorporation of linguistic resources that can guide the language model in generating a description that is more informative of the image in Portuguese. Experiments were performed with the translation of datasets for the generation of the description in Portuguese. The results give evidence that there is morphological learning in the training of an Image Captioning model and that the incorporation of grammatical classes during training can contribute to the generation of sentences with greater length and more informative.