Figure 1:
A machine learning process (Osia et al., 2018).
Figure 2:
Privacy issues during the machine learning process in Figure 1.
Table 1.
Privacy concerns in vision-based machine learning.
| Research | Application domain | Privacy concerns |
|---|---|---|
| Chattopadhyay and Boult (2007) | Intelligent surveillance system | Conflict between the purpose of intelligent surveillance systems and the privacy of individuals |
| Wu et al. (2018) | Smart camera application | Private information leakage during device-captured visual data upload to centralized cloud for analysis |
| Gomathisankaran et al. (2013) | Medical image analysis on the Cloud | Private information leakage of medical data transmitted in the network and processed in the cloud |
| Shokri et al. (2017) | ‘Machine learning as a service’ provided by Google and Amazon | Information leakage about training datasets |
| Speciale et al. (2019) | Augmented/Mixed reality (AR/MR) and autonomous robotic system | Confidential information disclosure about captured 3D scene |
Figure 3:
An inference attack model against collaborative learning (Melis et al., 2018).
Table 2.
Human recognition.
| Research | Object | Recognition methods | Data | Application domain | Privacy issue |
|---|---|---|---|---|---|
|
Chattopadhyay and Boult (2007)
Wu et al. (2018) Facial Network , Ren et al. (2016) | Human, object | Deep learning | Image, video | Video surveillance | Public dataset privacy User data privacy |
|
Song and Shmatikov (2020)
Nelus and Martin (2019) | Human face | Deep learning | Image | Binary gender classification | Model privacy |
|
Haris et al. (2014)
Gajjar et al. (2017) Nike , Malinowski (2010) | Human location | Deep learning | Sensor data | Location-based services, mobile heath applications | Public dataset privacy User data privacy |
|
Gomathisankaran et al. (2013)
Wang et al. (2014) Ertin et al. (2011) | Human disease, human health | Deep learning | Clinical records, image | Medical care | Public dataset privacy User data privacy |
Figure 4:
Object detection (Ren et al., 2016).
Table 3.
Human activity recognition.
| Research | Object | Recognition methods | Data | Application domain | Privacy issue |
|---|---|---|---|---|---|
|
Iwasawa et al. (2017)
You et al. (2012) | Human activity | Deep learning | Sensor data from smart wearable devices | Daily activity investigation | Training data privacy User data privacy |
|
Chen et al. (2018)
Hu et al. (2019) Zhang et al. (2019) | Physical activities such as walking and running | Deep learning | Time series sensor data from smart wearable devices | Daily activity investigation | User data privacy |
| Phan et al. (2016) | Human activity | Deep learning | Physical activities, biomarkers, biometric measures | Health social network | Training data privacy |
Figure 5:
Sensor-based human activity recognition (Hu et al., 2019).
Table 4.
Privacy-preserving approaches.
| Research | Privacy issue | Privacy-preserving approach | Protected object |
|---|---|---|---|
|
Garcia and Jacobs (2010)
Fontaine and Galand (2007) Gomathisankaran et al. (2013) Chattopadhyay and Boult (2007) | Private information leakage (Public dataset privacy) (User data privacy) | Cryptography | Private information (medical image, lifestyle, financial information, face, private location, biometric information, disease information), Human activity (daily life activity, movement) |
|
Butler et al. (2015)
Dai et al. (2015) Ryoo et al. (2017) Ren et al. (2018) Winkler et al. (2014) Speciale et al. (2019) | Private information leakage (Public dataset privacy) (User data privacy) | Anonymized videos | |
| Garcia Lopez et al. (2015) | Private information leakage from database (Public dataset privacy) (User data privacy) | Local processing | |
|
Liu (2019) Bun and Steinke (2016) | Information leakage from large-scale database (Public dataset privacy) | Differential privacy | |
| Bian et al. (2020) | Information leakage in visual recognition (Public dataset privacy) (Training data privacy) | Secure inference by homomorphic encryption Secret sharing Homomorphic convolution | |
|
Iwasawa et al. (2017)
Ajakan et al. (2015) Edwards and Storkey (2016) Malekzadeh et al. (2018, 2019) Osia et al. (2020) | Information disclosure by unintentional discriminating of user information during deep learning (Training data privacy) | Adversarial training | |
| Zhang et al. (2019) | Adversarial training which is effective on particular sensitive attributes (Training data privacy) | Image style transformation | |
|
Phan et al. (2016)
Abadi et al. (2016) Papernot et al. (2017) | Information leakage during deep learning (Training data privacy) | Differential privacy | |
|
Tramèr et al. (2016)
Wang and Gong (2018) Juuti et al. (2019) Kariyappa and Kariyappa (2019) | Information leakage during deep learning (Model privacy) | Analyze attacker’s queries, Defense against attacks |
Figure 6:
Homomorphic encryption (El-Yahyaoui and Ech-Cherif El Kettani, 2019).
Figure 7:
Sensitive areas are encrypted in the image (Chattopadhyay and Boult, 2007).
Figure 8:
Anonymized images: different modified pictures of the same person (Ren et al., 2018).
Figure 9:
A cartooning image (Winkler et al., 2014).
Figure 10:
A user privacy protection in image-based localization (Speciale et al., 2019).
Figure 11:
Central computing versus edge computing.
Figure 12:
Differential privacy (Wood et al., 2018).
Figure 13:
The private-feature extraction framework (Osia et al., 2020).
Figure 14:
Collective protection of all sensitive information at once (Zhang et al., 2019).
Figure 15:
Model extraction attack (Wang and Gong, 2018).