nvflare.app_common.np.np_downloader module

class ArrayConsumer(arrays_received_cb, cb_kwargs)

Bases: ItemConsumer

consume_items(items: List[Any], result: Any) → Any

Process items and return updated result.

class ArrayDownloadable(arrays: dict[str, ndarray], max_chunk_size: int)

Bases: CacheableObject

Constructor of CacheableObject.

Parameters:

• obj – the object to be downloaded.

• max_chunk_size – max number of bytes for each chunk.

Notes: The object must be able to be divided into multiple items. A chunk is generated for each item.

get_item_count() → int

The subclass must implement this method to return the number of items the object contains.

Returns: the number of items the object contains

produce_item(index: int) → bytes

This method is called to produce the chunk for the specified item.

Parameters:

index – index of the item.

Returns: a chunk for the item

add_arrays(downloader: ObjectDownloader, arrays: dict[str, ndarray], max_chunk_size: int = 2097152) → str

Add arrays to be downloaded to the specified downloader.

Parameters:

• downloader – the downloader to add arrays to.

• arrays – arrays to be downloaded

• max_chunk_size – max chunk size

Returns: reference id for the arrays.

download_arrays(from_fqcn: str, ref_id: str, per_request_timeout: float, cell: Cell, secure=False, optional=False, abort_signal=None, arrays_received_cb=None, **cb_kwargs) → Tuple[str, dict[str, ndarray] | None]

Download the referenced arrays from the source.

Parameters:

• from_fqcn – FQCN of the data source.

• ref_id – reference ID of the arrays to be downloaded.

• per_request_timeout – timeout for requests sent to the data source.

• cell – cell to be used for communicating to the data source.

• secure – P2P private mode for communication

• optional – suppress log messages of communication

• abort_signal – signal for aborting download.

• arrays_received_cb – the callback to be called when one set of arrays are received

Returns: tuple of (error message if any, downloaded state dict).