{"id":2676,"date":"2022-11-11T12:45:11","date_gmt":"2022-11-11T12:45:11","guid":{"rendered":"https:\/\/iot-ngin.eu\/?p=2676"},"modified":"2022-11-11T12:46:50","modified_gmt":"2022-11-11T12:46:50","slug":"privacy-preserving-mechanisms-with-nvidia-flare","status":"publish","type":"post","link":"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/","title":{"rendered":"Privacy preserving mechanisms with NVIDIA FLARE"},"content":{"rendered":"\n<p>The Internet of Things (IoT) is penetrating in our daily lives with multiple intelligent services and applications empowered by Artificial Intelligence (AI).&nbsp; Traditionally, AI techniques require centralized data collection and processing that may not be feasible in realistic application scenarios due to the high scalability of modern IoT networks. The main issue with the centralized collection of data is that it can expose individuals to privacy risks and organizations to legal risks if data is not properly managed. Federated Learning (FL) has emerged as a distributed collaborative AI approach that can enable many intelligent IoT applications, by allowing AI training to be performed at distributed IoT devices without the need for data sharing. Numerous open-source frameworks have been released that implement FL e.g., Flower&nbsp;[1], NV Flare [2] (NVIDIA Flare), TensorFlow Federated [3]. In this blog we will deal with the NV Flare by analyzing the privacy preserving techniques that it provides.<\/p>\n\n\n\n<p>NV FLARE mainly consists of the main node (server) and the federated nodes (clients). There is consecutive communication between them; specifically, the server is responsible for broadcasting tasks to clients (e.g., train, validation). After the clients have executed their tasks, they return the results to the server, where they are aggregated. Definitely, FL solves many privacy problems compared to traditional AI training but is still prone to malicious attacks. To minimize the odds of malicious attacks NV Flare provides different types of privacy-preserving mechanisms, such as percentile privacy, homomorphic encryption, etc. The supported privacy-preserving mechanisms can be applied as filters when the information is sent or received between peers. These mechanisms are briefly described below.<\/p>\n\n\n\n<ul class=\"ul-custom\"><li>Exclude Vars<\/li><\/ul>\n\n\n\n<p>The first filter is the \u201cExclude Vars\u201d and the behavior of the filter depends on the input. If the input is a list of variable\/layer names, only specified variables will be excluded. In the case that the input is a string, it will be converted into a regular expression, and only matched variables will be excluded.<\/p>\n\n\n\n<ul class=\"ul-custom\"><li>Percentile Privacy<\/li><\/ul>\n\n\n\n<p>The second filter supported by NV Flare is the one referring to \u201cPercentile Privacy\u201d. This filter is based on the &#8220;largest percentile to share&#8221; privacy preserving policy which is presented by Shokri and Shmatikov [4]. The main idea is that participants train independently on their own datasets and share small subsets of their models\u2019 parameters during training. The number of shared models\u2019 parameters depends on the percentile variable of the filter, which acts as a threshold. Using the \u201cPercentile Privacy\u201d filter, the client can control the percentile of parameters desired to be shared.<\/p>\n\n\n\n<ul class=\"ul-custom\"><li>SVT Privacy<\/li><\/ul>\n\n\n\n<p>Another filter supported is the differential privacy method, which is provided by NV Flare through the Sparse Vector Technique (SVT)\u00a0[5]. This filter applies a fundamental method for satisfying differential privacy by adding noise to ML model weights. SVT takes a sequence of queries and a certain threshold <img loading=\"lazy\" decoding=\"async\" width=\"7\" height=\"15\" src=\"\">\u00a0and outputs a vector {\u22a5,\u22a4}<em><sup>\u2113<\/sup><\/em>, where <em>\u2113<\/em> is the number of queries answered, T specifies that the corresponding query answer is above the threshold, conversely \u22a5 indicates it is below [6]. This algorithm, after identifying the meaningful queries, adds standard differentially private noise from the Laplace distribution.<\/p>\n\n\n\n<ul class=\"ul-custom\"><li>Homomorphic encryption<\/li><\/ul>\n\n\n\n<p>Homomorphic encryption (HE) is also available on NV Flare as a privacy-preserving choice. In HE, the clients receive keys to homomorphically encrypt their model updates before sending them to the server. The server does not own a key, it only sees the encrypted model updates and can aggregate these encrypted weights. As soon as the weights are aggregated, the server sends the updated model back to the clients, where they can decrypt the model weights because they have the keys.<\/p>\n\n\n\n<p>In some cases, using the mechanisms mentioned above there is a trade-off between privacy and model performance. Specifically, in case of SVT privacy by adding noise to the ML model weights there is interference t\u03bf weights and therefore deterioration of the performance. Also, it is possible to combine 2 privacy preserving techniques at the same time. For example, a combination of \u201cSVT Privacy\u201d with \u201cHomomorphic encryption\u201d can be performed where the first adds differential privacy to weights, while the second homomorphically encrypts the model.<\/p>\n\n\n\n<p>FL is a distributed AI approach that has sparked great interest to realize privacy-enhancing and scalable IoT services and applications. Malicious attacks had awakened the interest of multiple researchers to investigate methods in order to prevent them. NV Flare, as described above, provides 4 different privacy preserving mechanisms to avert malicious attacks and support researchers in protecting data privacy in a multitude of application scenarios.<\/p>\n\n\n\n<hr class=\"wp-block-separator is-style-wide\">\n<p><meta charset=\"utf-8\"><strong>References<\/strong><\/p>\n\n\n\n[1] <a>\tT. F. Authors, &#8220;\u201cA friendly federated learning framework&#8221;,&#8221; [Online]. Available: https:\/\/flower.dev\/. [Accessed 07 Nov 2022].<\/a> <br>\n[2] <a>\tN. Developer, &#8220;&#8221;Nvidia Flare&#8221;,&#8221; [Online]. Available: https:\/\/developer.nvidia.com\/flare. [Accessed 7 Nov 2022].<\/a> <br>\n[3] <a>\t&#8220;\u201cTensorflow Federated\u201d,&#8221; TensorFlow, [Online]. Available: https:\/\/www.tensorflow.org\/federated. [Accessed 7 Nov 2022].<\/a> <br>\n[4] <a>\tR. Shokri, &#8220;Privacy-preserving deep learning,&#8221; in Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, 2015.<\/a>  <br>\n[5] <a>\tC. Dwork, Cynthia and Naor, Moni and Reingold, Omer and Rothblum, Guy N and Vadhan and Salil, &#8220;On the complexity of differentially private data release: efficient algorithms and hardness results,&#8221; Proceedings of the forty-first annual ACM symposium on Theory of computing, pp. 381&#8211;390, 2009.<\/a>  <br>\n[6] <a>\tLyu, Min and Su, Dong and Li and Ninghui, &#8220;Understanding the sparse vector technique for differential privacy,&#8221; arXiv preprint arXiv:1603.01699, 2016.<\/a>  <br>\n\n\n\n<div style=\"height:100px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n","protected":false},"excerpt":{"rendered":"<p>The Internet of Things (IoT) is penetrating in our daily lives with multiple intelligent services and applications empowered by Artificial Intelligence (AI).&nbsp; Traditionally, AI techniques require centralized data collection and processing that may not be feasible in realistic application scenarios &hellip;<\/p>\n","protected":false},"author":2,"featured_media":2679,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"footnotes":""},"categories":[17],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v17.6 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Privacy preserving mechanisms with NVIDIA FLARE - IOT NGIN<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Privacy preserving mechanisms with NVIDIA FLARE - IOT NGIN\" \/>\n<meta property=\"og:description\" content=\"The Internet of Things (IoT) is penetrating in our daily lives with multiple intelligent services and applications empowered by Artificial Intelligence (AI).&nbsp; Traditionally, AI techniques require centralized data collection and processing that may not be feasible in realistic application scenarios &hellip;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/\" \/>\n<meta property=\"og:site_name\" content=\"IOT NGIN\" \/>\n<meta property=\"article:published_time\" content=\"2022-11-11T12:45:11+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-11-11T12:46:50+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/iot-ngin.eu\/wp-content\/uploads\/2022\/11\/fly-d-mT7lXZPjk7U-unsplash-1024x683.jpeg\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"683\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Giannis Tsichlas\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/iot-ngin.eu\/#website\",\"url\":\"https:\/\/iot-ngin.eu\/\",\"name\":\"IOT NGIN\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/iot-ngin.eu\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/iot-ngin.eu\/wp-content\/uploads\/2022\/11\/fly-d-mT7lXZPjk7U-unsplash.jpeg\",\"contentUrl\":\"https:\/\/iot-ngin.eu\/wp-content\/uploads\/2022\/11\/fly-d-mT7lXZPjk7U-unsplash.jpeg\",\"width\":7952,\"height\":5304},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/#webpage\",\"url\":\"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/\",\"name\":\"Privacy preserving mechanisms with NVIDIA FLARE - IOT NGIN\",\"isPartOf\":{\"@id\":\"https:\/\/iot-ngin.eu\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/#primaryimage\"},\"datePublished\":\"2022-11-11T12:45:11+00:00\",\"dateModified\":\"2022-11-11T12:46:50+00:00\",\"author\":{\"@id\":\"https:\/\/iot-ngin.eu\/#\/schema\/person\/dac2bc3c10805a89083986685c5801d1\"},\"breadcrumb\":{\"@id\":\"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/iot-ngin.eu\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Privacy preserving mechanisms with NVIDIA FLARE\"}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/iot-ngin.eu\/#\/schema\/person\/dac2bc3c10805a89083986685c5801d1\",\"name\":\"Giannis Tsichlas\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/iot-ngin.eu\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/740c19c414d2291ee24fcbe553ee2609?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/740c19c414d2291ee24fcbe553ee2609?s=96&d=mm&r=g\",\"caption\":\"Giannis Tsichlas\"},\"url\":\"https:\/\/iot-ngin.eu\/index.php\/author\/tsichlas\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Privacy preserving mechanisms with NVIDIA FLARE - IOT NGIN","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/","og_locale":"en_US","og_type":"article","og_title":"Privacy preserving mechanisms with NVIDIA FLARE - IOT NGIN","og_description":"The Internet of Things (IoT) is penetrating in our daily lives with multiple intelligent services and applications empowered by Artificial Intelligence (AI).&nbsp; Traditionally, AI techniques require centralized data collection and processing that may not be feasible in realistic application scenarios &hellip;","og_url":"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/","og_site_name":"IOT NGIN","article_published_time":"2022-11-11T12:45:11+00:00","article_modified_time":"2022-11-11T12:46:50+00:00","og_image":[{"width":1024,"height":683,"url":"https:\/\/iot-ngin.eu\/wp-content\/uploads\/2022\/11\/fly-d-mT7lXZPjk7U-unsplash-1024x683.jpeg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Written by":"Giannis Tsichlas","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebSite","@id":"https:\/\/iot-ngin.eu\/#website","url":"https:\/\/iot-ngin.eu\/","name":"IOT NGIN","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/iot-ngin.eu\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"ImageObject","@id":"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/#primaryimage","inLanguage":"en-US","url":"https:\/\/iot-ngin.eu\/wp-content\/uploads\/2022\/11\/fly-d-mT7lXZPjk7U-unsplash.jpeg","contentUrl":"https:\/\/iot-ngin.eu\/wp-content\/uploads\/2022\/11\/fly-d-mT7lXZPjk7U-unsplash.jpeg","width":7952,"height":5304},{"@type":"WebPage","@id":"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/#webpage","url":"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/","name":"Privacy preserving mechanisms with NVIDIA FLARE - IOT NGIN","isPartOf":{"@id":"https:\/\/iot-ngin.eu\/#website"},"primaryImageOfPage":{"@id":"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/#primaryimage"},"datePublished":"2022-11-11T12:45:11+00:00","dateModified":"2022-11-11T12:46:50+00:00","author":{"@id":"https:\/\/iot-ngin.eu\/#\/schema\/person\/dac2bc3c10805a89083986685c5801d1"},"breadcrumb":{"@id":"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/iot-ngin.eu\/index.php\/2022\/11\/11\/privacy-preserving-mechanisms-with-nvidia-flare\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/iot-ngin.eu\/"},{"@type":"ListItem","position":2,"name":"Privacy preserving mechanisms with NVIDIA FLARE"}]},{"@type":"Person","@id":"https:\/\/iot-ngin.eu\/#\/schema\/person\/dac2bc3c10805a89083986685c5801d1","name":"Giannis Tsichlas","image":{"@type":"ImageObject","@id":"https:\/\/iot-ngin.eu\/#personlogo","inLanguage":"en-US","url":"https:\/\/secure.gravatar.com\/avatar\/740c19c414d2291ee24fcbe553ee2609?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/740c19c414d2291ee24fcbe553ee2609?s=96&d=mm&r=g","caption":"Giannis Tsichlas"},"url":"https:\/\/iot-ngin.eu\/index.php\/author\/tsichlas\/"}]}},"cc_featured_image_caption":{"caption_text":"","source_text":"","source_url":""},"_links":{"self":[{"href":"https:\/\/iot-ngin.eu\/index.php\/wp-json\/wp\/v2\/posts\/2676"}],"collection":[{"href":"https:\/\/iot-ngin.eu\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/iot-ngin.eu\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/iot-ngin.eu\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/iot-ngin.eu\/index.php\/wp-json\/wp\/v2\/comments?post=2676"}],"version-history":[{"count":6,"href":"https:\/\/iot-ngin.eu\/index.php\/wp-json\/wp\/v2\/posts\/2676\/revisions"}],"predecessor-version":[{"id":2685,"href":"https:\/\/iot-ngin.eu\/index.php\/wp-json\/wp\/v2\/posts\/2676\/revisions\/2685"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/iot-ngin.eu\/index.php\/wp-json\/wp\/v2\/media\/2679"}],"wp:attachment":[{"href":"https:\/\/iot-ngin.eu\/index.php\/wp-json\/wp\/v2\/media?parent=2676"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/iot-ngin.eu\/index.php\/wp-json\/wp\/v2\/categories?post=2676"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/iot-ngin.eu\/index.php\/wp-json\/wp\/v2\/tags?post=2676"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}