A review of privacy-preserving human and human activity recognition

Im Y. Jung

doi:10.21307/ijssis-2020-008

Figures & Tables

A machine learning process (Osia et al., 2018).

Privacy issues during the machine learning process in Figure 1.

An inference attack model against collaborative learning (Melis et al., 2018).

Sensor-based human activity recognition (Hu et al., 2019).

Homomorphic encryption (El-Yahyaoui and Ech-Cherif El Kettani, 2019).

Sensitive areas are encrypted in the image (Chattopadhyay and Boult, 2007).

Anonymized images: different modified pictures of the same person (Ren et al., 2018).

A cartooning image (Winkler et al., 2014).

A user privacy protection in image-based localization (Speciale et al., 2019).

Central computing versus edge computing.

Differential privacy (Wood et al., 2018).

The private-feature extraction framework (Osia et al., 2020).

Collective protection of all sensitive information at once (Zhang et al., 2019).

Model extraction attack (Wang and Gong, 2018).

Human activity recognition_

Research	Object	Recognition methods	Data	Application domain	Privacy issue
Iwasawa et al. (2017) You et al. (2012)	Human activity	Deep learning	Sensor data from smart wearable devices	Daily activity investigation	Training data privacyUser data privacy
Chen et al. (2018) Hu et al. (2019) Zhang et al. (2019)	Physical activities such as walking and running	Deep learning	Time series sensor data from smart wearable devices	Daily activity investigation	User data privacy
Phan et al. (2016)	Human activity	Deep learning	Physical activities, biomarkers, biometric measures	Health social network	Training data privacy

Privacy concerns in vision-based machine learning_

Research	Application domain	Privacy concerns
Chattopadhyay and Boult (2007)	Intelligent surveillance system	Conflict between the purpose of intelligent surveillance systems and the privacy of individuals
Wu et al. (2018)	Smart camera application	Private information leakage during device-captured visual data upload to centralized cloud for analysis
Gomathisankaran et al. (2013)	Medical image analysis on the Cloud	Private information leakage of medical data transmitted in the network and processed in the cloud
Shokri et al. (2017)	‘Machine learning as a service’ provided by Google and Amazon	Information leakage about training datasets
Speciale et al. (2019)	Augmented/Mixed reality (AR/MR) and autonomous robotic system	Confidential information disclosure about captured 3D scene

Privacy-preserving approaches_

Research	Privacy issue	Privacy-preserving approach	Protected object
Garcia and Jacobs (2010) Fontaine and Galand (2007) Gomathisankaran et al. (2013) Chattopadhyay and Boult (2007)	Private information leakage (Public dataset privacy) (User data privacy)	Cryptography	Private information (medical image, lifestyle, financial information, face, private location, biometric information, disease information), Human activity (daily life activity, movement)
Butler et al. (2015) Dai et al. (2015) Ryoo et al. (2017) Ren et al. (2018) Winkler et al. (2014) Speciale et al. (2019)	Private information leakage (Public dataset privacy)(User data privacy)	Anonymized videos
Garcia Lopez et al. (2015)	Private information leakage from database (Public dataset privacy)(User data privacy)	Local processing
Liu (2019) Bun and Steinke (2016)	Information leakage from large-scale database (Public dataset privacy)	Differential privacy
Bian et al. (2020)	Information leakage in visual recognition(Public dataset privacy)(Training data privacy)	Secure inference by homomorphic encryptionSecret sharingHomomorphic convolution
Iwasawa et al. (2017) Ajakan et al. (2015) Edwards and Storkey (2016) Malekzadeh et al. (2018, 2019) Osia et al. (2020)	Information disclosure by unintentional discriminating of user information during deep learning (Training data privacy)	Adversarial training
Zhang et al. (2019)	Adversarial training which is effective on particular sensitive attributes (Training data privacy)	Image style transformation
Phan et al. (2016) Abadi et al. (2016) Papernot et al. (2017)	Information leakage during deep learning (Training data privacy)	Differential privacy
Tramèr et al. (2016) Wang and Gong (2018) Juuti et al. (2019) Kariyappa and Kariyappa (2019)	Information leakage during deep learning (Model privacy)	Analyze attacker’s queries, Defense against attacks

Human recognition_

Research	Object	Recognition methods	Data	Application domain	Privacy issue
Chattopadhyay and Boult (2007) Wu et al. (2018) Facial Network, Ren et al. (2016)	Human, object	Deep learning	Image, video	Video surveillance	Public dataset privacyUser data privacy
Song and Shmatikov (2020) Nelus and Martin (2019)	Human face	Deep learning	Image	Binary gender classification	Model privacy
Haris et al. (2014) Gajjar et al. (2017) Nike, Malinowski (2010)	Human location	Deep learning	Sensor data	Location-based services, mobile heath applications	Public dataset privacyUser data privacy
Gomathisankaran et al. (2013) Wang et al. (2014) Ertin et al. (2011)	Human disease, human health	Deep learning	Clinical records, image	Medical care	Public dataset privacyUser data privacy

A review of privacy-preserving human and human activity recognition

Figures & Tables

Figure 1:

Figure 2:

Figure 3:

Figure 4:

Figure 5:

Figure 6:

Figure 7:

Figure 8:

Figure 9:

Figure 10:

Figure 11:

Figure 12:

Figure 13:

Figure 14:

Figure 15:

Human activity recognition_

Privacy concerns in vision-based machine learning_

Privacy-preserving approaches_

Human recognition_

Paradigm

My account