New Frontiers in Metaphor Visualization: An Approach Through Generative Artificial Intelligence

Margaux, Cerutti; Miroslw, Lis Mateusz; Piero, Polidoro

doi:10.2478/lf-2024-0022

Abstract

In the last two years, our culture and our everyday lives have literally been invaded by the use of new technological systems based on the operation of artificial intelligence. In a very short time, we have seen the development of new platforms capable of generating images or texts autonomously, following only instructions (input) of a few words. From this thought comes our question. What would happen if we asked such a system to produce something using a metaphor as input? The metaphor was chosen because, according to important semiotic and linguistic theories (Eco, Lakoff & Johnson), it is one of the foundational elements of human thought. As we will see, generative artificial intelligence systems exhibit different behaviors when faced with metaphorical inputs. In this article, we will attempt to explore these behaviors and the variables that trigger them through various approaches.

References

Bernard, J. 2024 . What are word embeddings? IBM, available at: https://www.ibm.com/topics/word-embeddings
Search in Google Scholar Back to article
Bondielli, A., Passaro, L. C., 2021. Sfruttare CLIP per il riconoscimento delle emozioni nelle immagini. In Cabrio, E., Croce, D., Passaro, L. C., Sprugnoli, R. (Eds.), Atti del Quinto Workshop su Linguaggio Naturale per l’Intelligenza Artificiale (NL4AI 2021) in concomitanza con la 20a Conferenza Internazionale dell’Associazione Italiana per l’Intelligenza Artificiale (AI*IA 2021), Evento online, 29 novembre 2021. CEUR Workshop Proceedings, 3015. CEUR-WS.org. available at: https://ceur-ws.org/Vol-3015/paper172.pdf
Search in Google Scholar Back to article
Bustos, C., Civit, C., Du, B. et al., 2023. Sull’uso dei modelli Vision-Language per l’analisi del sentimento visuale: uno studio su CLIP. In 11a Conferenza Internazionale sull’Interazione Intelligente e l’Informatica Affettiva (ACII) (pp. 1-8). IEEE Computer Society. available at: https://doi.ieeecomputersociety.org/10.1109/ACII59096.2023.10388075
Search in Google Scholar Back to article
Chakrabarty, T., Saakyan, A., Winn, O. et al., 2023. I spy a metaphor: Large language models and diffusion models co-create visual metaphors. arXiv preprint arXiv:2305.14724.
Search in Google Scholar Back to article
Chen, M., Radford, A., Child, R. et al., 2020 . Preallenamento generativo dai pixel. In III, H. Daumé & Singh, A. (Eds.), Atti della 37a Conferenza Internazionale sull’Apprendimento Automatico (pp. 1691-1703). PMLR. available at: https://proceedings.mlr.press/v119/chen20s.html
Search in Google Scholar Back to article
Eco, U. 1984 . Semiotica e filosofia del linguaggio, Einaudi, Torino.
Search in Google Scholar Back to article
Fernández Gambín, A., Yazidi, A., Vasilakos, A. et al., 2024. Deepfakes: current and future trends. Artificial Intelligence Review, 57:64.
Search in Google Scholar Back to article
Garrido-Muñoz, I., Montejo-Ráez, A., Martínez-Santiago, F., Ureña-López, L. A. 2021 . A survey on bias in deep NLP. Applied Sciences, 11(7), 3184.
Search in Google Scholar Back to article
Groupe μ (1970). Rhétorique générale, Larousse, Paris (trad. it. Retorica generale, Bopiani, Milano 1976; II ediz. 1980)
Search in Google Scholar Back to article
Groupe μ 1992 . Traité du signe visuel. Seuil, Paris.
Search in Google Scholar Back to article
Ho, J., Jain, A., Abbeel, P. 2020 . Denoising diffusion probabilistic models. Advances in neural information processing systems, 33, 6840-6851.
Search in Google Scholar Back to article
Kaur, D., Kaur, Y. 2014 . Various image segmentation techniques: a review. International Journal of Computer Science and Mobile Computing, 3(5), 809-814.
Search in Google Scholar Back to article
Lakoff, G., Johnson, M. 1980 . Metaphors we live by, The University of Chicago Press, Chicago (trad. it. Metafora e vita quotidiana, Bompiani, Milano 1998). LeCun, Y., Bengio, Y., Hinton, G. 2015 . Deep learning. nature, 521(7553), 436-444.
Search in Google Scholar Back to article
Prystawski, B., Thibodeau, P., Potts, C. et al., 2022. Psychologically-informed chain-of-thought prompts for metaphor understanding in large language models. arXiv preprint arXiv:2209.08141.
Search in Google Scholar Back to article
Ramesh, A., Dhariwal, P., Nichol, A., et al., 2022. Generazione di immagini gerarchica condizionata al testo con latenti CLIP. ArXiv, cs.CV. available at: https://arxiv.org/abs/2204.06125
Search in Google Scholar Back to article
Rombach, R., Blattmann, A., Lorenz, D. et al., 2022 . Sintesi di immagini ad alta risoluzione con modelli di diffusione latenti. ArXiv, cs.CV. available at: https://arxiv.org/abs/2112.10752
Search in Google Scholar Back to article
Sonesson, G. 1989 . Pictorial concepts, Lund University Press, Lund.
Search in Google Scholar Back to article
Tong, X., Choenni, R., Lewis, M. et al., 2024. Dataset di sfida per la comprensione delle metafore per i modelli di linguaggio di grandi dimensioni. ArXiv, cs.CL. available at: https://arxiv.org/abs/2403.11810
Search in Google Scholar Back to article
Yakura, H. 2023 . Evaluating large language models’ ability to understand metaphor and sarcasm using a screening test for Asperger syndrome. arXiv preprint arXiv:2309.10744.
Search in Google Scholar Back to article
Zoph, B., Raffel, C., Schuurmans, D. et al., 2022 . Abilità emergenti dei grandi modelli di linguaggio. TMLR.
Search in Google Scholar Back to article

New Frontiers in Metaphor Visualization: An Approach Through Generative Artificial Intelligence

Abstract

Paradigm

My account