nvflare.app_common.workflows.global_model_eval module

class GlobalModelEval(task_check_period=0.5, cross_val_dir='cross_site_val', validation_timeout: int = 6000, model_locator_id='', formatter_id='', validation_task_name='validate', cleanup_models=False, participating_clients=None, wait_for_clients_timeout=300)[source]

Bases: CrossSiteModelEval

Global Model Evaluation workflow.

# TODO: change validation to evaluation to reflect the real meaning

Parameters:

  • task_check_period (float, optional) – How often to check for new tasks or tasks being finished. Defaults to 0.5.

  • cross_val_dir (str, optional) – Path to cross site validation directory relative to run directory. Defaults to “cross_site_val”.

  • validation_timeout (int, optional) – Timeout for validate_model task. Defaults to 6000.

  • model_locator_id (str, optional) – ID for model_locator component. Defaults to None.

  • formatter_id (str, optional) – ID for formatter component. Defaults to None.

  • validation_task_name (str, optional) – Name of validate_model task. Defaults to “validate”.

  • cleanup_models (bool, optional) – Whether models should be deleted after run. Defaults to False.

  • participating_clients (list, optional) – List of participating client names. If not provided, defaults to all clients connected at start of controller.

  • wait_for_clients_timeout (int, optional) – Timeout for clients to appear. Defaults to 300 secs