A Brief Overview of Federated Learning - A New Perspective on Data Privacy

Lipovanu, Iuliana-Alexandra; Pascal, Carlos; Căruntu, Constantin-Florin

doi:10.2478/bipie-2022-0019

Abstract

While privacy concerns remain the main challenge starting with the promulgation of the General Data Protection Regulation (GDPR), for deep learning applications, Google introduced recently the Federated Learning (FL) technique to offer support for high privacy-sensitive data, which makes FL a hot research topic nowadays. Thus, it is a distributed machine learning technique in which multiple devices (clients) collaboratively train a global model to solve issues where the first concern is data privacy. This work provides a brief study of FL: an overview of this new topic, related works, a comparison with other machine learning techniques, an overview of algorithms that are currently used, and, in the end, some simulation results and new directions of research. The simulations show the distributed behavior of the FL algorithm and the way in which the Federated Averaging method can be applied. Through the performed analysis of the obtained results, it was figured out that approach would be beneficial for several applications in domains like automotive, 5G and others.

References

Abadi M., Barham P., Chen J., Chen Z., Davis A., Dean J., Devin M., Ghemawat S., Irving G., Isard M., Kudlur M., Levenberg J., Monga R., Moore S., Murray D. G., Steiner B., Tucker P., Vasudevan V., Warden P., Wicke M., Yu Y., Zheng X., TensorFlow: a system for large-scale machine learning, In Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation (OSDI’16), USENIX Association, USA, pp. 265–283, 2016.
Search in Google Scholar Back to article
Beutel D.J., Topal T., Mathur A., Qiu X., Fernandez-Marques J., Gao Y., Sani L., Li KH., Parcollet T., de Gusmão P.P., Lane ND., Flower: A friendly federated learning framework, preprint arXiv:2007.14390, 2021.
Search in Google Scholar Back to article
Caldas S., Duddu S.M., Wu P., Li T., Konečný J., McMahan H.B., Smith V., Talwalkar A., Leaf: A benchmark for federated settings, arXiv preprint arXiv:1812.01097, 2018.
Search in Google Scholar Back to article
Hard A., Rao K., Mathews R., Ramaswamy S., Beaufays F., Augenstein S., Eichner H., Kiddon C., Ramage D., Federated learning for mobile keyboard prediction, arXiv preprint arXiv:1811.03604, 2018.
Search in Google Scholar Back to article
Hsieh K., Machine learning systems for highly-distributed and rapidly-growing data, Ph.D. Dissertation, Carnegie Mellon University, 2019.
Search in Google Scholar Back to article
Jiang J. C., Kantarci B., Oktug S., Soyata T., Federated learning in smart city sensing: Challenges and opportunities, Sensors, 20(21), 6230, 2020.
Search in Google Scholar Back to article
Konečný J., McMahan H.B., Yu F.X., Richtárik P., Suresh A.T., Bacon D., Federated learning: Strategies for improving communication efficiency, arXiv preprint arXiv:1610.05492. 2016.
Search in Google Scholar Back to article
LeCun Y., The MNIST database of handwritten digits, http://yann.lecun.com/exdb/mnist/, 1998.
Search in Google Scholar Back to article
Li L., Fan Y., Tse M., Lin KY., A review of applications in federated learning,Computers & Industrial Engineering, pp. 106854, 2020.
Search in Google Scholar Back to article
Li Q., Wen Z., Wu Z., Hu S., Wang N., Li Y., Liu X., He B., A survey on federated learning systems: vision, hype, and reality for data privacy and protection, IEEE Transactions on Knowledge and Data Engineering, 2021.
Search in Google Scholar Back to article
Li T., Sahu A.K., Talwalkar A., Smith V., Federated learning: Challenges, methods, and future directions,IEEE Signal Processing Magazine, vol 37, no. 3, pp. 50–60, 2020.
Search in Google Scholar Back to article
Liu Y., James J.Q., Kang J., Niyato D., Zhang S., Privacy-preserving traffic flow prediction: A federated learning approach, IEEE Internet of Things Journal, vol. 7, no. 8, pp. 7751–7763, 2020.
Search in Google Scholar Back to article
Liu Y., Peng J., Kang J., Iliyasu A.M., Niyato D., Abd El-Latif A.A., A secure federated learning framework for 5G networks, IEEE Wireless Communications, vol. 27, no. 4, pp. 24–31, 2020.
Search in Google Scholar Back to article
McMahan H.B., Moore E., Ramage D., Arcas B.A., Federated learning of deep networks using model averaging, arXiv, vol. abs/1602.05629, 2016.
Search in Google Scholar Back to article
McMahan H.B., Moore E., Ramage D., Hampson S., y Arcas B.A., Communication-efficient learning of deep networks from decentralized data, In Artificial intelligence and statistics, pp. 1273–1282, 2017.
Search in Google Scholar Back to article
Nilsson A., Smith S., Ulm G., Gustavsson E., Jirstrand M., A performance evaluation of federated learning algorithms, InProceedings of the Second Workshop on Distributed Infrastructures for Deep Learning, pp. 1–8, 2018.
Search in Google Scholar Back to article
Niknam S., Dhillon HS., Reed JH., Federated learning for wireless communications: Motivation, opportunities, and challenges, IEEE Communications Magazine, vol. 58, no. 6, pp. 46–51, 2020.
Search in Google Scholar Back to article
Pang G., Shen C., Cao L., Hengel A.V., Deep learning for anomaly detection: A review, ACM Computing Surveys (CSUR), vol. 54, no. 2, pp. 1–38, 2021.
Search in Google Scholar Back to article
Rieke N., Hancox J., Li W., Milletari F., Roth H.R., Albarqouni S., Bakas S., Galtier M.N., Landman B.A., Maier-Hein K., Ourselin S., The future of digital health with federated learning, NPJ digital medicine, vol. 3, no. 1, pp. 1–7, 2020.
Search in Google Scholar Back to article
Saputra Y.M., Hoang D.T., Nguyen D.N., Dutkiewicz E., Mueck M.D., Srikanteswara S., Energy demand prediction with federated learning for electric vehicle networks, in 2019 IEEE Global Communications Conference (GLOBECOM), IEEE, pp. 1–6, 2019.
Search in Google Scholar Back to article
Tijani S.A., Ma X., Zhang R., Jiang F., Doss R., Federated Learning with Extreme Label Skew: A Data Extension Approach, In 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1–8, 2021.
Search in Google Scholar Back to article
Wahab O.A., Mourad A., Otrok H., Taleb T., Federated machine learning: Survey, multi-level classification, desirable criteria and future directions in communication and networking systems, IEEE Communications Surveys & Tutorials, vol. 23, no. 2, pp. 1342–1397, 2021.
Search in Google Scholar Back to article
Wu Q., He K., Chen X., Personalized federated learning for intelligent IoT applications: A cloud-edge based framework,IEEE Open Journal of the Computer Society, vol. 1, pp. 35–44, 2020.
Search in Google Scholar Back to article
Yang T., Andrew G., Eichner H., Sun H., Li W., Kong N., Beaufays F., Applied federated learning: Improving google keyboard query suggestions, arXiv preprint arXiv:1812.02903, 2018.
Search in Google Scholar Back to article
Zhang N., Wang FY., Zhu F., Zhao D., Tang S., DynaCAS: Computational experiments and decision support for ITS, IEEE Intelligent Systems, vol. 23, no. 6, pp. 19–23, 2008.
Search in Google Scholar Back to article
Zhang W., Lu Q., Yu Q., Li Z., Liu Y., Lo S.K., Chen S., Xu X., Zhu L., Blockchain-based federated learning for device failure detection in industrial IoT, IEEE Internet of Things Journal, vol. 8, no. 7, pp. 5926–5937, 2020.
Search in Google Scholar Back to article

A Brief Overview of Federated Learning - A New Perspective on Data Privacy

Abstract

Paradigm

My account