Site Policy Management¶
It is possible for each site to define its own policies in the following areas:
Resource Management: the configuration of system resources that are solely the decisions of local IT;
Authorization Policy: local authorization policy that determines what a user can or cannot do on the local site;
Privacy Policy: local policy that specifies what types of studies are allowed and how to add privacy protection to the learning results produced by the FL client on the local site.
Logging Configuration: each site can now define its own logging configuration for system generated log messages.
Workspace Structure¶
NVFLARE’s policy files are stored in the workspace. To support local site policies, a new “local” folder is added to the workspace. Here is the complete workspace structure, with the addition of the “local” folder:
{WSROOT}
startup
fed_server|client.json
Site cert, site private key, root certificate, and site
start.sh
…
local
resources.json.default
authorization.json.default
privacy.json.sample
log.config.default
resources.json
authorization.json
privacy.json
log.config
custom/
local_code.xyz
audit.txt
log.txt
1234567 (run)
log.txt
job_meta.json
app_xxx
fl_app.txt
config
config_fed_client.json
…
custom
xyz.py
234562 (run)
log.txt
job_meta.json
app_xxx
fl_app.txt
config
custom
Content highlighted in yellow is generated by the Provision process - the ZIP package generated by the Provision now contains two folders: startup and local. The “startup” folder contains security credentials needed for communication to the FL Server, as well as general system configuration information. The “local” folder contains default and/or samples for local policies. If the Org Admin wants to define his/her own policies, he/she can do so by creating separate files to override the default. These files are unhighlighted ones in the “local” folder.
The Org Admin can also install additional custom code in the “local/custom” folder. This makes it possible for the site to develop its own custom filters for privacy control.
Resource Management Policy¶
Configuration items in fed_server|client.json that should be local decisions are now moved to local resources.json.default.
Here is the example of resources.json.default for FL Server:
{
"format_version": 2,
"servers": [
{
"admin_storage": "transfer",
"max_num_clients": 100,
"heart_beat_timeout": 600,
"num_server_workers": 4,
"download_job_url": "http://download.server.com/",
"compression": "Gzip"
}
],
"snapshot_persistor": {
"path": "nvflare.app_common.state_persistors.storage_state_persistor.StorageStatePersistor",
"args": {
"uri_root": "/",
"storage": {
"path": "nvflare.app_common.storages.filesystem_storage.FilesystemStorage",
"args": {
"root_dir": "/tmp/nvflare/snapshot-storage",
"uri_root": "/"
}
}
}
},
"components": [
{
"id": "job_scheduler",
"path": "nvflare.app_common.job_schedulers.job_scheduler.DefaultJobScheduler",
"args": {
"max_jobs": 1
}
},
{
"id": "job_manager",
"path": "nvflare.apis.impl.job_def_manager.SimpleJobDefManager",
"args": {
"uri_root": "/tmp/nvflare/jobs-storage",
"job_store_id": "job_store"
}
},
{
"id": "job_store",
"path": "nvflare.app_common.storages.filesystem_storage.FilesystemStorage"
}
]
}
As you can see, the Org Admin can decide to change parameters or even use different Python objects for storage without having to go through another Provision process.
Here is the example of resources.json.default for a FL Client:
{
"format_version": 2,
"client": {
"retry_timeout": 30,
"compression": "Gzip"
},
"components": [
{
"id": "resource_manager",
"path": "nvflare.app_common.resource_managers.list_resource_manager.ListResourceManager",
"args": {
"resources": {
"gpu": [
0,
1,
2,
3
]
}
}
},
{
"id": "resource_consumer",
"path": "nvflare.app_common.resource_consumers.gpu_resource_consumer.GPUResourceConsumer",
"args": {
"gpu_resource_key": "gpu"
}
}
]
}
As you can see, the Org Admin of the FL client site can change the number of GPUs and other parameters without going through another Provision process.
Privacy Management¶
NVFLARE comes with a security enhancement that allows each site to define its own privacy protection policy to be applied to the learning results produced by the client.
Note that in this discussion, data privacy protection specifically refers to this threat: the receiver (Server) of the learning results produced by a sender (Client) could discover/reconstruct the learning data by reverse engineering the learning results.
As in previous versions of NVFLARE, the primary privacy protection technique is the filtering mechanism. There are two types of filters:
Task Data Filters - they are applied to the task data before invoking an executor to execute the task. Only the filtered task data is passed to the task executor.
Task Result Filters - they are applied to the task result produced by the task executor before sending back to the Server. Only the filtered result will be sent to the Server.
In previous versions of NVFLARE, only researchers can specify filters in the job configuration. However it may not be the best interest of the researchers to protect data privacy of FL clients. Protecting data privacy is the Org Admin’s interest.
NVFLARE allows the Org Admin to specify filters for data privacy protection. Unlike researcher-specified filters that are only applicable to a job, filters specified in the site’s privacy policies are applicable to all jobs! This is made possible by the concept of Scope.
A scope can be thought of as a space within which jobs are performed. For example, depending on the purpose of the FL project, the Project Admin may decide to conduct the study in two phases. First run jobs in a “public” scope that use some publicly available datasets and with relaxed data privacy protection. After algorithms are determined, then run jobs in a “private” scope where each site’s own datasets will be used with more strict data privacy protection.
Each scope has the following attributes:
Name - a scope must have a unique name. It is the Project Admin’s job to work with all sites to come up with the scopes and their names at the beginning of the project.
Properties - any key/values that define additional properties that could be useful for executors to execute tasks in the scope.
Task Data Filters - filters to be applied to task data for jobs in the scope.
Task Result Filters - filters to be applied to the task result for jobs in the scope.
The following is a sample policy:
{
"scopes": [
{
"name": "public",
"properties": {
"train_dataset": "/data/public/train",
"val_dataset": "/data/public/val"
},
"task_result_filters": [
{
"name": "AddNoiseToMinMax",
"args": {
"min_noise_level": 0.2,
"max_noise_level": 0.2
}
},
{
"name": "PercentilePrivacy",
"args": {
"percentile": 10,
"gamma": 0.02
}
}
],
"task_data_filters": [
{
"name": "BadModelDetector"
}
]
},
{
"name": "private",
"properties": {
"train_dataset": "/data/private/train",
"val_dataset": "/data/private/val"
},
"task_result_filters": [
{
"name": "AddNoiseToMinMax",
"args": {
"min_noise_level": 0.1,
"max_noise_level": 0.1
}
},
{
"name": "SVTPrivacy",
"args": {
"fraction": 0.1,
"epsilon": 0.2
}
}
]
}
],
"default_scope": "public"
}
The scope of the job is specified with the meta key “scope”. If the job doesn’t specify scope, the default scope is used.
Privacy Processing Rules¶
The following are the privacy processing rules built into NVFLARE:
If the site does not define privacy.json, then no privacy control is applied.
If a job does not explicitly specify a scope name, then the site-specified “default_scope” will be used as the scope of the job. If the site does not specify the default scope, then the job will be rejected. This rule is enforced at the Job Deploy time.
If the job-specified scope is not found in the scope list of the site, then the job is rejected. This rule is enforced at the Job Deploy time.
If a job’s scope is found (either as default scope, or explicitly defined in the site’s scope list), then the scope’s filters (if any) are applied before the job-specified filters (if any). This rule is enforced during task execution time.
Create Site Policies¶
To ensure system integrity and minimize chance of errors, please follow the following simple steps:
Make a copy of the file that you want to override and name the new file with a temporary name. For example: cp resources.json.default my_resources.json
Edit the new file with your own policy definition, and save
Rename the file to the right name: mv my_resources.json resources.json