nvflare.app_common.workflows.cross_site_model_eval module

class CrossSiteModelEval(task_check_period=0.5, cross_val_dir='cross_site_val', submit_model_timeout=600, validation_timeout: int = 6000, model_locator_id='', formatter_id='', submit_model_task_name='submit_model', validation_task_name='validate', cleanup_models=False, participating_clients=None, wait_for_clients_timeout=300)[source]

Bases: Controller

Cross Site Model Evaluation workflow.

# TODO: change validation to evaluation to reflect the real meaning

Parameters:

  • task_check_period (float, optional) – How often to check for new tasks or tasks being finished. Defaults to 0.5.

  • cross_val_dir (str, optional) – Path to cross site validation directory relative to run directory. Defaults to “cross_site_val”.

  • submit_model_timeout (int, optional) – Timeout of submit_model_task. Defaults to 600 secs.

  • validation_timeout (int, optional) – Timeout for validate_model task. Defaults to 6000 secs.

  • model_locator_id (str, optional) – ID for ModelLocator component. Defaults to “”.

  • formatter_id (str, optional) – ID for Formatter component. Defaults to “”.

  • submit_model_task_name (str, optional) – Name of submit_model task. Defaults to “”.

  • validation_task_name (str, optional) – Name of validate_model task. Defaults to “validate”.

  • cleanup_models (bool, optional) – Whether or not models should be deleted after run. Defaults to False.

  • participating_clients (list, optional) – List of participating client names. If not provided, defaults to all clients connected at start of controller.

  • wait_for_clients_timeout (int, optional) – Timeout for clients to appear. Defaults to 300 secs

control_flow(abort_signal: Signal, fl_ctx: FLContext)[source]

This is the control logic for the RUN.

NOTE: this is running in a separate thread, and its life is the duration of the RUN.

Parameters:

  • fl_ctx – the FL context

  • abort_signal – the abort signal. If triggered, this method stops waiting and returns to the caller.

handle_event(event_type: str, fl_ctx: FLContext)[source]

Handles events.

Parameters:

  • event_type (str) – event type fired by workflow.

  • fl_ctx (FLContext) – FLContext information.

process_result_of_unknown_task(client: Client, task_name: str, client_task_id: str, result: Shareable, fl_ctx: FLContext)[source]

Process result when no task is found for it.

This is called when a result submission is received from a client, but no standing task can be found for it (from the task queue)

This could happen when: - the client’s submission is too late - the task is already completed - the Controller lost the task, e.g. the Server is restarted

Parameters:

  • client – the client that the result comes from

  • task_name – the name of the task

  • client_task_id – ID of the task

  • result – the result from the client

  • fl_ctx – the FL context that comes with the client’s submission

start_controller(fl_ctx: FLContext)[source]

Starts the controller.

This method is called at the beginning of the RUN.

Parameters:

  • fl_ctx – the FL context. You can use this context to access services provided by the

  • example (framework. For)

  • your (you can get Command Register from it and register)

  • modules. (admin command)

stop_controller(fl_ctx: FLContext)[source]

Stops the controller.

This method is called right before the RUN is ended.

Parameters:

  • fl_ctx – the FL context. You can use this context to access services provided by the

  • example (framework. For)

  • your (you can get Command Register from it and unregister)

  • modules. (admin command)