Feasibility Analysis of Phenotype Quantification from Unstructured Clinical Interactions

Daniel S. Barron; Stephen Heisig; Carla Agurto; Raquel Norel; Brittany Quagan; Albert Powers; Michael L. Birnbaum; Todd Constable; Guillermo Cecchi; John H. Krystal

doi:10.5334/cpsy.78

Abstract

We conducted a feasibility analysis to determine the quality of data that could be collected ambiently during routine clinical conversations. We used inexpensive, consumer-grade hardware to record unstructured dialogue and open-source software tools to quantify and model face, voice (acoustic and language) and movement features. We used an external validation set to perform proof-of-concept predictive analyses and show that clinically relevant measures can be produced without a restrictive protocol.

References

1Agurto, C., et al. (2019). Analyzing progression of motor and speech impairment in ALS. Conf Proc Annu Int Conf Ieee Eng Medicine Biology Soc Ieee Eng Medicine Biology Soc Annu Conf, 2019, 6097–6102. DOI: 10.1109/EMBC.2019.8857300
Back to article
2Baltrusaitis, T., Zadeh, A., Lim, Y. C., & Morency, L.-P. (2018). OpenFace 2.0: Facial Behavior Analysis Toolkit. 2018 13th Ieee Int Conf Automatic Face Gesture Recognit Fg 2018, 59–66. DOI: 10.1109/FG.2018.00019
Back to article
3Bedi, G., et al. (2015). Automated analysis of free speech predicts psychosis onset in high-risk youths. NPJ schizophrenia, 1, 15030. DOI: 10.1038/npjschz.2015.30
Back to article
4Bishay, M., Palasek, P., Priebe, S., & Patras, I. (2019). SchiNet: Automatic Estimation of Symptoms of Schizophrenia from Facial Behaviour Analysis. Ieee T Affect Comput, 1–1.
Back to article
5Cohn, J. F., et al. (2009). Detecting Depression from Facial Actions and Vocal Prosody. 2009 3rd Int Conf Affect Comput Intelligent Interact Work, 1–7. DOI: 10.1109/ACII.2009.5349358
Back to article
6Cohn, J. F., Ambadar, Z., & Ekman, P. (2007). Observer-Based Measurement of Facial Expression With the Facial Action Coding System. Handbook of emotion elicitation and assessment, 203–221.
Back to article
7Corcoran, C. M., et al. (2018). Prediction of psychosis across protocols and risk cohorts using automated language analysis. World Psychiatry, 17, 67–75. DOI: 10.1002/wps.20491
Back to article
8de Jong, N. H., & Wempe, T. (2009). Praat script to detect syllable nuclei and measure speech rate automatically. Behavior Research Methods, 41, 385–390. DOI: 10.3758/BRM.41.2.385
Back to article
9Ekman, P. (1964). Body Position, Facial Expression, and Verbal Behavior During Interviews. Journal of Abnormal and Social Psychology, 68, 295–301. DOI: 10.1037/h0040225
Back to article
10Ekman, P. (2013). Emotion in the Human Face. (Malor Books Reprint Edition, 2013).
Back to article
11Ekman, P., Sorenson, E. R., & Friesen, W. V. (1969). Pan-Cultural Elements in Facial Displays of Emotion. Science, 164, 86–88. DOI: 10.1126/science.164.3875.86
Back to article
12Mahl, G. F. (1964). MEASUREMENT OF VERBAL AND VOCAL BEHAVIOR. SOME OBSERVATIONS ABOUT RESEARCH ON VOCAL BEHAVIOR. Res Publ – Assoc Res Nerv Ment Dis, 42, 466–83.
Back to article
13Mahl, G. F., Dollard, J., & Redlich, F. C. (1954). Facilities for the Sound Recording and Observation of Interviews. Science, 120, 235–239. DOI: 10.1126/science.120.3111.235
Back to article
14Overall, J. E., & Gorham, D. R. (1962). The Brief Psychiatric Rating Scale. Psychological Reports, 10, 799–812. DOI: 10.2466/pr0.1962.10.3.799
Back to article
15Pennington, J., Socher, R., & Manning, C. (2014). Glove: Global Vectors for Word Representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, 1532–1543. DOI: 10.3115/v1/D14-1162
Back to article
16Scherer, S., Lucas, G. M., Gratch, J., Rizzo, A. S., & Morency, L.-P. (2016). Self-Reported Symptoms of Depression and PTSD Are Associated with Reduced Vowel Space in Screening Interviews. Ieee T Affect Comput, 7, 59–73. DOI: 10.1109/TAFFC.2015.2440264
Back to article
17Stratou, G., Scherer, S., Gratch, J., & Morency, L.-P. (2015). Automatic nonverbal behavior indicators of depression and PTSD: the effect of gender. J Multimodal User In, 9, 17–29. DOI: 10.1007/s12193-014-0161-4
Back to article
18van Steenbergen-Weijenburg, K. M., et al. (2010). Validation of the PHQ-9 as a screening instrument for depression in diabetes patients in specialized outpatient clinics. Bmc Health Serv Res, 10, 235. DOI: 10.1186/1472-6963-10-235
Back to article

Feasibility Analysis of Phenotype Quantification from Unstructured Clinical Interactions

Abstract

Paradigm

My account