Cognitive biases in digital food advertising and their emotional impact: a comparative approach based on artificial intelligence and computer vision models
DOI:
https://doi.org/10.61799/2216-0388.2093Keywords:
affective modeling, computer vision, food advertising, Markov chain, multilabel classification, visual biasesAbstract
Digital food advertising systematically exploits perceptual visual biases to influence both emotions and consumer behavior; however, the existing body of research lacks computational frameworks that link the automated detection of visual biases with probabilistic emotional inference. The purpose of this article is to design and validate a hybrid architecture for inferring probable emotional profiles based on visual biases in food advertising images, that is, a combination of Deep Computer Vision techniques and probabilistic modeling. To this end, a two-module architecture was constructed: the first is based on a Deep Computer Vision model with multi-label classification, using the EfficientNet-B0 model, which was compared with the ResNet-50, ViT-B/16, and MobileNet-V3 Large models; the second was the corresponding probabilistic component based on a Markov chain that maps the detected bias samples into ten emotional categories relevant to food marketing. A dataset was also created comprising 1,375 food advertising images from social media, which were manually annotated across eleven established categories of visual bias. Regarding the results, the EfficientNet-B0 architecture achieved the following scores: mean Average Precision = 0.961, ROC-AUC = 0.968, and weighted F1 score = 0.898, outperforming the architectures implemented in this study, while the Markov-inspired affective module produced emotional profiles that are easily interpreted in empirical literature. Finally, the conclusion is that the proposed architecture decouples the perceptual level from the affective level, enabling mechanistic explainability, updates via modules, and the extensibility of the emotional space, and allowing for practical applicability in marketing research, public health, and the regulation of commercial communication.
Downloads
References
[¨1] E. Boyland et al., "Food marketing, eating and health outcomes in children and adults: A systematic review and meta-analysis," British Journal of Nutrition, vol. 133, no. 6, pp. 781–805, 2025, doi: https://doi.org/10.1017/S0007114524000102
[2] K. Kim, S. Lee, and Y. K. Choi, "Image proximity in advertising appeals: Spatial distance and product types," Journal of Business Research, vol. 99, pp. 490–497, 2019, doi: https://doi.org/10.1016/j.jbusres.2017.08.031
[3] J. R. Cummings, L. V. Hoover, and A. N. Gearhardt, "A randomized experiment of the effects of food advertisements on food-related emotional expectations in adults," Journal of Health Psychology, vol. 28, no. 10, pp. 929–942, 2023, doi: https://doi.org/10.1177/13591053231168340
[4] C. Baquedano et al., "The Power of Food Advertisements: A Brief Mindfulness Instruction Does Not Prevent Psychophysiological Responses Triggered by Food Ads," Brain Sciences, vol. 15, no. 3, p. 240, 2025, doi: https://doi.org/10.3390/brainsci15030240
[5] F. Barbosa Escobar, Q. J. Wang, A. Corredor, and C. Velasco, "The taste of visual textures," Food Quality and Preference, vol. 100, p. 104602, 2022, doi: https://doi.org/10.1016/j.foodqual.2022.104602
[6] B. De Martino, D. Kumaran, B. Seymour, and R. J. Dolan, "Frames, Biases, and Rational Decision-Making in the Human Brain," Science, vol. 313, no. 5787, pp. 684–687, 2006, doi: https://doi.org/10.1126/science.1128356
[7] S. Fabbrizzi, S. Papadopoulos, E. Ntoutsi, and I. Kompatsiaris, "A Survey on Bias in Visual Datasets," arXiv:2107.07919, 2022, doi: https://doi.org/10.48550/arXiv.2107.07919
[8] J. Dai, J. Cone, and J. Moher, "Perceptual salience influences food choices independently of health and taste preferences," Cognitive Research: Principles and Implications, vol. 5, no. 1, p. 2, 2020, doi: https://doi.org/10.1186/s41235-019-0203-2
[9] J. Liang, Y.-Q. Zou, S.-Y. Liang, Y.-W. Wu, and W.-J. Yan, "Emotional Gaze: The Effects of Gaze Direction on the Perception of Facial Emotions," Frontiers in Psychology, vol. 12, p. 684357, 2021, doi: https://doi.org/10.3389/fpsyg.2021.684357
[10] L. Miccoli, R. Delgado, P. Guerra, F. Versace, S. Rodríguez-Ruiz, and M. C. Fernández-Santaella, "Affective Pictures and the Open Library of Affective Foods (OLAF): Tools to Investigate Emotions toward Food in Adults," PLOS ONE, vol. 11, no. 8, p. e0158991, 2016, doi: https://doi.org/10.1371/journal.pone.0158991
[11] S. Adil, S. Lacoste-Badie, and O. Droulers, "Face Presence and Gaze Direction in Print Advertisements: How They Influence Consumer Responses—An Eye-Tracking Study," Journal of Advertising Research, vol. 58, no. 4, pp. 443–455, 2018, doi: https://doi.org/10.2501/JAR-2018-004
[12] M. Mehta and C. Buntain, "Emotional Images: Assessing Emotions in Images and Potential Biases in Generative Models," arXiv:2411.05985, 2024, doi: https://doi.org/10.48550/arXiv.2411.05985
[13] R. Ismail and Z. Yuan, "Food Ingredients Recognition through Multi-label Learning," 2022.
[14] Y. Shu, H. Gao, Y. Wang, and Y. Wei, "Food Emotional Perception and Eating Willingness Under Different Lighting Colors: A Preliminary Study Based on Consumer Facial Expression Analysis," Foods, vol. 14, no. 19, p. 3440, 2025, doi: https://doi.org/10.3390/foods14193440
[15] P. Cipresso, F. Borghesi, and A. Chirico, "Affects affect affects: A Markov Chain," Frontiers in Psychology, vol. 14, p. 1162655, 2023, doi: https://doi.org/10.3389/fpsyg.2023.1162655
[16] F. Bockting, S. T. Radev, and P.-C. Bürkner, "Simulation-based prior knowledge elicitation for parametric Bayesian models," Scientific Reports, vol. 14, no. 1, p. 17330, 2024, doi: https://doi.org/10.1038/s41598-024-68090-7
[17] A. M. Stefan, D. Katsimpokis, Q. F. Gronau, and E.-J. Wagenmakers, "Expert agreement in prior elicitation and its effects on Bayesian inference," Psychonomic Bulletin & Review, vol. 29, no. 5, pp. 1776–1794, 2022, doi: https://doi.org/10.3758/s13423-022-02074-4
[18] S. Peker, G. G. Menekse Dalveren, and Y. İnal, "The Effects of the Content Elements of Online Banner Ads on Visual Attention: Evidence from An Eye-Tracking Study," Future Internet, vol. 13, no. 1, p. 18, 2021, doi: https://doi.org/10.3390/fi13010018
[19] J. García-Madariaga, I. Moya, N. Recuero, and M.-F. Blasco, "Revealing Unconscious Consumer Reactions to Advertisements That Include Visual Metaphors. A Neurophysiological Experiment," Frontiers in Psychology, vol. 11, p. 760, 2020, doi: https://doi.org/10.3389/fpsyg.2020.00760
[20] G. Hu et al., "Using artificial intelligence and computer vision to detect and monitor unhealthy child-directed food and beverage marketing: A data-driven study," eClinicalMedicine, vol. 89, p. 103549, 2025, doi: https://doi.org/10.1016/j.eclinm.2025.103549
[21] I. Vermeir and G. Roose, "Visual Design Cues Impacting Food Choice: A Review and Future Research Agenda," Foods, vol. 9, no. 10, p. 1495, 2020, doi: https://doi.org/10.3390/foods9101495
[22] A. Fenko, R. De Vries, and T. Van Rompay, "How Strong Is Your Coffee? The Influence of Visual Metaphors and Textual Claims on Consumers' Flavor Perception and Product Evaluation," Frontiers in Psychology, vol. 9, p. 53, 2018, doi: https://doi.org/10.3389/fpsyg.2018.00053
[23] J. Palcu, J. Sudkamp, and A. Florack, "Judgments at Gaze Value: Gaze Cuing in Banner Advertisements, Its Effect on Attention Allocation and Product Judgments," Frontiers in Psychology, vol. 8, p. 881, 2017, doi: https://doi.org/10.3389/fpsyg.2017.00881
[24] S. Spinelli, C. Masi, G. P. Zoboli, J. Prescott, and E. Monteleone, "Emotional responses to branded and unbranded foods," Food Quality and Preference, vol. 42, pp. 1–11, 2015, doi: https://doi.org/10.1016/j.foodqual.2014.12.009
[25] A. Toet, M. G. Van Schaik, D. Kaneko, and J. B. F. Van Erp, "Do food cinemagraphs evoke stronger appetitive responses than stills?," International Journal of Food Design, vol. 4, no. 1, pp. 63–83, 2019, doi: https://doi.org/10.1386/ijfd.4.1.63_1
[26] Y. Huang, X. Xue, E. Spelke, L. Huang, W. Zheng, and K. Peng, "The aesthetic preference for symmetry dissociates from early-emerging attention to symmetry," Scientific Reports, vol. 8, no. 1, p. 6263, 2018, doi: https://doi.org/10.1038/s41598-018-24558-x
[27] A. R. Tartaglini, W. K. Vong, and B. M. Lake, "A Developmentally-Inspired Examination of Shape versus Texture Bias in Machines," arXiv:2202.08340, 2022, doi: https://doi.org/10.48550/arXiv.2202.08340
[28] Y. Lu, "An Investigation of Gestalt Principles in Advertising Design," Arts, Culture and Language, vol. 1, no. 4, 2024, doi: https://doi.org/10.61173/z3jn0p62
[29] M. Almourad, E. Bataineh, M. Hussein, and Z. Wattar, "Strategic Placement of Branding Elements in Digital Marketing: Insights from Eye-Tracking Data," in Proc. 27th Int. Conf. on Enterprise Information Systems, 2025, pp. 417–423, doi: https://doi.org/10.5220/0013281500003929
[30] T. L. Childers and J. Jass, "All Dressed Up With Something to Say: Effects of Typeface Semantic Associations on Brand Perceptions and Consumer Memory," n.d.
[31] M. J. Madsen, "A Q-Method Study of Visual Metaphors in Advertising," n.d.
[32] G. Chochlakis, G. Mahajan, S. Baruah, K. Burghardt, K. Lerman, and S. Narayanan, "Using Emotion Embeddings to Transfer Knowledge Between Emotions, Languages, and Annotation Formats," arXiv:2211.00171, 2023, doi: https://doi.org/10.48550/arXiv.2211.00171
[33] F. J. Otamendi and D. L. Sutil Martín, "The Emotional Effectiveness of Advertisement," Frontiers in Psychology, vol. 11, p. 2088, 2020, doi: https://doi.org/10.3389/fpsyg.2020.02088
[34] K. Han et al., "A Survey on Visual Transformer," IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 1, pp. 87–110, 2023, doi: https://doi.org/10.1109/TPAMI.2022.3152247
[35] M. Tan and Q. V. Le, "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks," arXiv:1905.11946, 2020, doi: https://doi.org/10.48550/arXiv.1905.11946
[36] X. Wu et al., "CTransCNN: Combining transformer and CNN in multilabel medical image classification," Knowledge-Based Systems, vol. 281, p. 111030, 2023, doi: https://doi.org/10.1016/j.knosys.2023.111030
[37] S. Kornblith, J. Shlens, and Q. V. Le, "Do Better ImageNet Models Transfer Better?," in Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 2656–2666, doi: https://doi.org/10.1109/CVPR.2019.00277
[38] K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778, doi: https://doi.org/10.1109/CVPR.2016.90
[39] A. Dosovitskiy et al., "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale," arXiv:2010.11929, 2021, doi: https://doi.org/10.48550/arXiv.2010.11929
[40] A. Howard et al., "Searching for MobileNetV3," in Proc. IEEE/CVF Int. Conf. on Computer Vision (ICCV), 2019, pp. 1314–1324, doi: https://doi.org/10.1109/ICCV.2019.00140
[41] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, "MobileNetV2: Inverted Residuals and Linear Bottlenecks," in Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 4510–4520, doi: https://doi.org/10.1109/CVPR.2018.00474
[42] O. Rainio, J. Teuho, and R. Klén, "Evaluation metrics and statistical tests for machine learning," Scientific Reports, vol. 14, no. 1, p. 6086, 2024, doi: https://doi.org/10.1038/s41598-024-56706-x
[43] G. Lee, S. Yi, and J. Lee, "A Study on Deep Learning Performances of Identifying Images' Emotion: Comparing Performances of Three Algorithms to Analyze Fashion Items," Applied Sciences, vol. 15, no. 6, p. 3318, 2025, doi: https://doi.org/10.3390/app15063318
[44] R. Reber, N. Schwarz, and P. Winkielman, "Processing Fluency and Aesthetic Pleasure: Is Beauty in the Perceiver's Processing Experience?," Personality and Social Psychology Review, vol. 8, no. 4, pp. 364–382, 2004, doi: https://doi.org/10.1207/s15327957pspr0804_3
[45] C. Janiszewski, "Preattentive Mere Exposure Effects," Journal of Consumer Research, vol. 20, no. 3, p. 376, 1993, doi: https://doi.org/10.1086/209356
[46] C. K. Friesen and A. Kingstone, "The eyes have it! Reflexive orienting is triggered by nonpredictive gaze," Psychonomic Bulletin & Review, vol. 5, no. 3, pp. 490–495, 1998, doi: https://doi.org/10.3758/BF03208827
[47] J. K. Maner, M. T. Gailliot, D. A. Rouby, and S. L. Miller, "Can't take my eyes off you: Attentional adhesion to mates and rivals," Journal of Personality and Social Psychology, vol. 93, no. 3, pp. 389–401, 2007, doi: https://doi.org/10.1037/0022-3514.93.3.389
[48] A. Tversky and D. Kahneman, "The Framing of Decisions and the Psychology of Choice," Science, vol. 211, no. 4481, pp. 453–458, 1981, doi: https://doi.org/10.1126/science.7455683
[49] V. Allom et al., "Reflective and impulsive processes underlying saving behavior and the additional roles of self-control and habit," Journal of Neuroscience, Psychology, and Economics, vol. 11, no. 3, pp. 135–146, 2018, doi: https://doi.org/10.1037/npe0000093
[50] R. F. Bornstein, "Exposure and Affect: Overview and Meta-Analysis of Research, 1968–1987," Psychological Bulletin, vol. 106, no. 2, pp. 265–289, 1989, doi: https://doi.org/10.1037/0033-2909.106.2.265
[51] L. Pileliene and V. Grigaliunaite, "The effect of female celebrity spokesperson in FMCG advertising: Neuromarketing approach," Journal of Consumer Marketing, vol. 34, no. 3, pp. 202–213, 2017, doi: https://doi.org/10.1108/JCM-02-2016-1723
[52] P. Vuilleumier, "How brains beware: Neural mechanisms of emotional attention," Trends in Cognitive Sciences, vol. 9, no. 12, pp. 585–594, 2005, doi: https://doi.org/10.1016/j.tics.2005.10.011
[53] L. Pessoa and R. Adolphs, "Emotion processing and the amygdala: From a 'low road' to 'many roads' of evaluating biological significance," Nature Reviews Neuroscience, vol. 11, no. 11, pp. 773–782, 2010, doi: https://doi.org/10.1038/nrn2920
[54] P. Locher, E. A. Krupinski, C. Mello-Thoms, and C. F. Nodine, "Visual interest in pictorial art during an aesthetic experience," n.d.
[55] D. M. Oppenheimer, "The secret life of fluency," Trends in Cognitive Sciences, vol. 12, no. 6, pp. 237–241, 2008, doi: https://doi.org/10.1016/j.tics.2008.02.014
[56] R. B. Zajonc, "Attitudinal effects of mere exposure," Journal of Personality and Social Psychology, vol. 9, no. 2, pt.2, pp. 1–27, 1968, doi: https://doi.org/10.1037/h0025848
[57] R. F. Bornstein and P. R. D'Agostino, "The Attribution and Discounting of Perceptual Fluency: Preliminary Tests of a Perceptual Fluency/Attributional Model of the Mere Exposure Effect," Social Cognition, vol. 12, no. 2, pp. 103–128, 1994, doi: https://doi.org/10.1521/soco.1994.12.2.103
[58] A. Krishna, "An integrative review of sensory marketing: Engaging the senses to affect perception, judgment and behavior," Journal of Consumer Psychology, vol. 22, no. 3, pp. 332–351, 2012, doi: https://doi.org/10.1016/j.jcps.2011.08.003
[59] A. Krishna and N. Schwarz, "Sensory marketing, embodiment, and grounded cognition: A review and introduction," Journal of Consumer Psychology, vol. 24, no. 2, pp. 159–168, 2014, doi: https://doi.org/10.1016/j.jcps.2013.12.006
[60] M. Boulanger and P. Garnham, "Typography matters: The neuroscience of type," Monotype, 2022. [Online]. Available: https://www.monotype.com/typography-matters
[61] I. Schindler et al., "Measuring aesthetic emotions: A review of the literature and a new assessment tool," PLOS ONE, vol. 12, no. 6, p. e0178899, 2017, doi: https://doi.org/10.1371/journal.pone.0178899
[62] P. P. L. Tinio and H. Leder, "Just how stable are stable aesthetic features? Symmetry, complexity, and the jaws of massive familiarization," Acta Psychologica, vol. 130, no. 3, pp. 241–250, 2009, doi: https://doi.org/10.1016/j.actpsy.2009.01.001
[63] J. Cohen, Statistical power analysis for the behavioral sciences, 2nd ed. Hillsdale, NJ, USA: L. Erlbaum Associates, 1988.
[64] J. Ruscio and T. Mullen, "Confidence Intervals for the Probability of Superiority Effect Size Measurement and the Area Under a Receiver Operating Characteristic Curve," Multivariate Behavioral Research, vol. 47, no. 2, pp. 201–223, 2012, doi: https://doi.org/10.1080/00273171.2012.658329
[65] M. Bondaschi et al., "From Markov to Laplace: How Mamba In-Context Learns Markov Chains," arXiv:2502.10178, 2025, doi: https://doi.org/10.48550/arXiv.2502.10178
[66] J. M. Arengas Acosta, M. Lopez Ramirez, and R. Guzman Cabrera, “Impacto del preprocesamiento en la clasificación automática de textos usando aprendizaje supervisado y Reuters 21578,” Revista Colombiana de Tecnologías de Avanzada (RCTA), vol. 1, no. 43, pp. 110–118, 2024, doi: 10.24054/rcta.v1i43.2506.
[67] L. Contreras-Manrique, R. de B. Contreras-Manrique, T. V. Ovalle-Lizcano, and J. F. Rincón-Leal, “Análisis descriptivo de rasgos de personalidad en relación con el género y la edad de los estudiantes de la Universidad de Pamplona en la sede Villa Rosario,” Eco Matemático, vol. 11, no. 1, pp. 51–61, 2020, doi: 10.22463/17948231.2943.
[68] A. M. Machacado-Rojas and L. E. Aparicio-Pico, “Técnicas de inteligencia artificial aplicadas al análisis de imágenes diagnóstico,” Eco Matemático, vol. 12, no. 2, pp. 100–111, 2021, doi: 10.22463/17948231.3237.
[69] C. A. Mejia Rodriguez, M. A. Rincon Pinzon, L. M. Palmera Quintero, and L. M. Arevalo Vergel, “Aplicación de machine learning y metodología CRISP-DM para la clasificación precisa de severidad en casos de dengue,” Revista Colombiana de Tecnologías de Avanzada (RCTA), vol. 1, no. 43, pp. 78–85, 2024, doi: 10.24054/rcta.v1i43.2822.
Published
Issue
Section
License
Copyright (c) 2026 Mundo FESC Journal
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
