Have a personal or library account? Click to login
A visual perception-guided data augmentation method for efficient machine learning-based detection of facial micro-expressions Cover

A visual perception-guided data augmentation method for efficient machine learning-based detection of facial micro-expressions

Open Access
|Nov 2024

References

  1. H. E. Adelson, and J. R. Bergen, Spatiotemporal energy models for the perception of motion, Journal of the Optical Society of America A vol. 2, no. 2, 1985.
  2. C.C. Aggarwal, Neural Networks and Deep Learning, A Textbook, Springer 2018.
  3. A. Asthana, S. Zafeiriou, S. Cheng, and M. Pantic, Robust discriminative response map fitting with constrained local models, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3444-3451, 2013.
  4. D. Brunet, E.R. Vrscay, and Z. Wang, On the mathematical properties of the structural similarity index, IEEE Transactions on Image Processing, vol. 21, no. 4, 2012.
  5. V. Bruni, G. Ramponi, A. Restrepo, and D. Vitulano, Context-Based Defading of Archive Photographs, Journal of Image and Video Processing, 2009.
  6. V. Bruni, E. Rossi, and D. Vitulano, On the Equivalence Between Jensen-Shannon Divergence and Michelson Contrast, IEEE Transactions on Information Theory, vol. 58, no. 7, pp. 4278-4288, 2012.
  7. V. Bruni, D. Vitulano, and Z. Wang, Special issue on human vision and information theory, Signal, Image and Video Processing, vol. 7, no.3,pp. 389-390, 2013.
  8. V. Bruni, D. De Canditiis, and D. Vitulano, Speed up of Video Enhancement based on Human Perception, Signal Image and Video Processing, vol. 8, pp. 1199-1209, 2014.
  9. V. Bruni, D. Panella, and D. Vitulano, Non local means image denoising using noise-adaptive SSIM, Proceedings of the 23rd European Signal Processing Conference, EUSIPCO, 2015.
  10. V. Bruni, and D. Vitulano, Jensen shannon divergence as reduced reference measure for image denoising, Lecture Notes in Computer Science, vol. 10016, 2016.
  11. V. Bruni, and D. Vitulano, An entropy based approach for SSIM speed up, Signal Processing, vol. 135, pp. 198-209, 2017.
  12. V. Bruni, and D. Vitulano, SSIM Based Signature of Facial Micro-Expressions, Proceedings of International Conference in Image Analysis and Recognition (ICIAR 2020), Lecture Notes in Computer Science, vol. 12131, 2020.
  13. V. Bruni, and D. Vitulano, A Fast Preprocessing Method for Micro-Expression Spotting via Perceptual Detection of Frozen Frames, Journal of Imaging, MDPI, vol. 7, no. 4, 2021.
  14. F.W. Campbell, and J.G. Robson, Application of Fourier analysis to the visibility of gratings, Journal of Physiology, vol. 197, no. 3, pp. 551-566, 1968.
  15. D. Cristinacce, T. F. Cootes, et al., Feature detection and tracking with constrained local models, Proceedings of the British Machine Vision Conference 2006, Edinburgh, UK, vol. 1, 2006.
  16. C. Duque, O. Alata, R. Emonet, A.C. Legrand, and H. Konik, Micro-expression spotting using the Riesz pyramid, Proceedings of WACV, Lake Tahoe, 2018.
  17. P. Ekman, and M.V. Friesen, Nonverbal leakage and clues to deception, Psychiatry, vol. 32, pp. 88-106, 1969.
  18. P. Ekman, Lie catching and microexpressions. The Philosophy of Deception, ed C. Martin (Oxford University Press), pp. 118-133, 2009.
  19. P. Ekman, Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage, WW Norton & Company, 2009.
  20. V. Esmaeili, M. Mohassel Feghhi, and S.O. Shahdi, A comprehensive survey on facial micro-expression: approaches and databases, Multimedia Tools and Applications, 2022.
  21. W. Gong, and N.M. Elfiky, Deep learning-based microexpression recognition: a survey, Neural Computing and Applications, 2022.
  22. E.A. Haggard, and K.S. Isaacs, Micromomentary facial expressions as indicators of ego mechanisms in psychotherapy, Methods of Research in Psychotherapy. L. A. Gottschalk and H. Auerbach (Boston, MA: Springer), pp. 154-165, 1966
  23. Y. He, S.J. Wang, J. Li, and M. H. Yap, Spotting macro-and micro-expression intervals in long video sequences, Proceedings of the 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), pp. 742-748, 2020.
  24. U. Hess, and R.E. Kleck, Differentiating emotion elicited and deliberate emotional facial expressions, European Journal of Social Psychology, vol. 20, pp. 369-385, 1990.
  25. M. Kendall, and A. Stuart, The Advanced Theory of Statistics, Chareles Griffinn & Company Limited, 1976.
  26. D. E. King, Dlib-ml, A machine learning toolkit, The Journal of Machine Learning Research, vol. 10, pp. 1755-1758, 2009.
  27. L. Jingting, S.J. Wang, M. H. Yap, J. See, X. Hong, and X. Li, Megc2020-the third facial microexpression grand challenge, Proceedings of the 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), pp. 777-780, 2020.
  28. Y. Li, X. Huang, and G. Zhao, Can micro-expression be recognized based on single apex frame?, Proceedings of the 25th IEEE International Conference on Image Processing (ICIP), pp. 3094-3098, 2018.
  29. J. Li, C. Soladie, and R. Seguier, Ltp-ml, Micro-expression detection by recognition of local temporal pattern of facial movements, Proceedings of the 13th IEEE international conference on automatic face & gesture recognition (FG 2018), pp. 634-641, 2018.
  30. J. Li, C. Soladie, and R. Seguier, Local temporal pattern and data augmentation for micro-expression spotting, IEEE Transactions on Affective Computing, 2020.
  31. S.T. Liong, J. See, K. Wong, A.C. Le Ngo, Y.H. Oh, and R. Phan, Automatic apex frame spotting in micro-expression database, Proceedings of the 3rd IAPR Asian Conference on Pattern Recognition (ACPR), pp. 665-669, 2015.
  32. S.T. Liong, J. See, K. Wong, and R. C.W. Phan, Automatic microexpression recognition from long video using a single spotted apex, Proceedings of the Asian Conference on Computer Vision, Springer, pp. 345-360, 2016.
  33. G.B. Liong, J. See, and L.K. Wong, Shallow optical flow three- stream cnn for macro-and microexpression spotting from long videos, Proceedings of the IEEE International Conference on Image Processing (ICIP), Anchorage, AK, USA, pp. 2643-2647, 2021.
  34. G.B. Liong, J. See, and C.S. Chan, Spot-then-recognize: A Micro-Expression Analysis Network for seamless evaluation of long videos, Signal Processing: Image Communication, vol. 110, 2023.
  35. H. Ma, G. An, S. Wu, and F. Yang, A region histogram of oriented optical flow (RHOOF) feature for apex frame spotting in micro-expression, Proceedings of the International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS), pp. 281–286, 2017.
  36. V. Mante, R. A. Frazor, V. Bonin, W. S. Geisler, and M. Carandini, Independence of luminance and contrast in natural scenes and in the early visual system, Nature Neuroscience, vol.8, pp. 1690-1697, 2005.
  37. I. Megvii, Face++ research toolkit, 2013.
  38. A. Mehrabian, Nonverbal Communication, Publisher, ALDINE-ATHERTON, 1972 (eBook Published, 31 October 2017).
  39. MEVIEW Homepage, url http://cmp.felk.cvut.cz/cechj/ME/.
  40. S. Milborrow, and F. Nicolls, Active shape models with sift descriptors and mars, Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP), vol.2, pp. 380-387, 2014.
  41. A. Moilanen, G. Zhao, and M. Pietikainen, Spotting rapid facial movements from videos using appearance-based feature difference analysis, Proceedings of the 22nd International Conference on Pattern Recognition (ICPR), pp. 1722–1727, 2014.
  42. Y.H. Oh, J. See, A. C. Le Ngo, R.C. Phan, and V.M. Baskaran, A Survey of Automatic Facial Micro- Expression Analysis: Databases, Methods, and Challenges, Frontiers in Psychology, 2018.
  43. S. Polikovsky, and Y. Kameda, Facial micro-expression detection in hi-speed video based on facial action coding system (facs), IEICE Transactions on Information and Systems, vol. 9, pp. 81-92, 2013.
  44. S. Porter, and L. Ten Brinke, Reading between the lies identifying concealed and falsified emotions in universal facial expressions, Psychological Science, vol. 19, pp. 508-514, 2008.
  45. F. Qu, S.J. Wang, W.J. Yan, H. Li, S. Wu, and X. Fu, Cas(me)2: a database for spontaneous macroexpression and micro-expression spotting and recognition, IEEE Transactions on Affective Computing, vol. 9, no. 4, pp. 424-436, 2017.
  46. C. Shorten, and T.M. Khoshgoftaar, A survey on Image Data Augmentation for Deep Learning, Journal of Big Data, vol. 6, no. 60, 2019.
  47. M. Shreve, S. Godavarthy, D. Goldgof, and S. Sarkar, Macro-and micro-expression spotting in long videos using spatio-temporal strain, Proceedings of the IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), pp. 51-56, 2011.
  48. M. Shreve, J. Brizzi, S. Felatyev, T. Luguev, D. Goldgof, and S. Sarkar, Automatic expression spotting in videos, Image and Vision Computing, vol. 32, no. 8, pp. 476-486, 2014.
  49. M.F. Valstar, and M. Pantic, Fully automatic recognition of the temporal phases of facial actions, IEEE Transactions on Systems Man and Cybernetics Part B, vol.42, pp. 28-43, 2012.
  50. M. Verburg, and V. Menkovski, Micro-expression detection in long videos using optical flow and recurrent neural networks, Proceedings of the 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), pp. 1-6, 2019.
  51. Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, Image Quality Assessment: From Error Visibility to Structural Similarity, IEEE Transactions on Image Processing, vol. 13, pp. 600-612, 2004.
  52. S.J. Wang, S. Wu, X. Qian, J. Li, and X. Fu, A main directional maximal difference analysis for spotting facial movements from long-term videos, Neurocomputing, vol. 230, pp. 382-389, 2016.
  53. S.J. Wang, Y. He, J. Li, and X. Fu, Mesnet: A convolutional neural network for spotting multi-scale micro-expression intervals in long videos, IEEE Transactions on Image Processing, vol. 30, pp. 3956-3969, 2021.
  54. S. Weinberger, Airport security: intent to deceive?, Nature, vol. 465, pp. 412-415, 2010.
  55. S. Winkler, Digital Video Quality - Vision Models and Metrics, John Wiley & Sons, Ltd, 2005.
  56. W.J. Yan, Q. Wu, J. Liang, Y.H. Chen, and X. Fu, How fast are the leaked facial expressions: the duration of micro-expressions, Journal of Nonverbal Behavior, vol. 37, pp. 217-230, 2013.
  57. W.J. Yan, Q. Wu, Y.J. Liu, S.J. Wang, and X. Fu, Casme database: a dataset of spontaneous microexpressions collected from neutralized faces, Proceedinds of the 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1-7, 2013.
  58. W.J. Yan, X. Li, S.J.Wang, G. Zhao, Y.J. Liu, Y.H. Chen, et al., CASME II: an improved spontaneous micro-expression database and the baseline evaluation, PLoS ONE, vol. 9, 2014.
  59. W.J.Yan, and Y.H Chen, Measuring dynamic micro-expressions via feature extraction methods, Journal of Computational Science, vol. 25, pp. 318-326, 2017.
  60. C.H. Yap, C. Kendrick, and M.H. Yap, Samm long videos: A spontaneous facial micro-and macroexpressions dataset, Proceedings of the 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), pp. 771-776, 2020.
  61. Z. Zhang, T. Chen, H. Meng, G. Liu, and X. Fu, Smeconvnet: A convolutional neural network for spotting spontaneous facial micro-expression from long videos, IEEE Access, vol. 6, pp. 71143-71151, 2018.
  62. H. Zhang, L. Yin, and H. Zhang, A review of micro-expression spotting: methods and challenges, Multimedia Systems, vol. 29, pp. 1897-1915, 2023.
Language: English
Page range: 102 - 123
Submitted on: Jun 25, 2024
Accepted on: Oct 3, 2024
Published on: Nov 21, 2024
Published by: Italian Society for Applied and Industrial Mathemathics
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year

© 2024 Vittoria Bruni, Salvatore Cuomo, Domenico Vitulano, published by Italian Society for Applied and Industrial Mathemathics
This work is licensed under the Creative Commons Attribution 4.0 License.