nvflare.fuel.utils.fobs.decomposers.via_file module

class EncKey

Bases: object

DATA = 'data'

TYPE = 'type'

class EncType

Bases: object

NATIVE = 'native'

REF = 'ref'

class ViaFileDecomposer(min_size_for_file, config_var_prefix)

Bases: Decomposer, ABC

decompose(target: Any, manager: DatumManager | None = None) → Any

Decompose the target into types supported by msgpack or classes with decomposers registered.

Msgpack supports primitives, bytes, memoryview, lists, dicts.

Parameters:

• target – The instance to be serialized

• manager – Datum manager to store externalized datum

Returns:

The decomposed serializable objects

abstract dump_to_file(items: dict, path: str, fobs_ctx: dict)

Dump the items to the file with the specified path

Parameters:

• items – a dict of items of target object type to be dumped to file

• path – the path to the file.

• fobs_ctx – FOBS Context

Returns: a tuple of (file name, meta info)

The “path” is a temporary file name. You should create the file with the specified name. However, some frameworks (e.g. numpy) may add a special suffix to the name. In this case, you must return the modified name.

The “items” is a dict of target objects. The dict contains all objects of the target type in one payload. The dict could be very big. You must create a file to contain all the objects.

abstract get_bytes_dot() → int

Get the Datum Object Type to be used for bytes datum

Returns: the DOT for bytes datum

abstract get_file_dot() → int

Get the Datum Object Type to be used for file ref datum

Returns: the DOT for file ref datum

abstract load_from_file(path: str, fobs_ctx: dict, meta: dict | None = None) → dict

Load target object items from the specified file

Parameters:

• path – the absolute path to the file to be loaded.

• fobs_ctx – FOBS Context.

• meta – meta info of the file.

Returns: a dict of target objects.

You must not delete the file after loading. Management of the file is done by the ViaFile class.

abstract native_decompose(target: Any, manager: DatumManager | None = None) → bytes

abstract native_recompose(data: bytes, manager: DatumManager | None = None) → Any

process_datum(datum: Datum, manager: DatumManager)

This is called by the manager to process a datum that has a DOT. This happens before the recompose processing.

The datum contains information about where the data is: For bytes DOT, the data is included in the datum directly. For file DOT, the data is in a file, and the location of the file is further specified:

• If the location is local, then the file is on local file system;

• If the location is remote_cell, then the file is on a remote cell, and needs to be downloaded.

Parameters:

• datum – datum to be processed.

• manager – the datum manager.

Returns: None

recompose(data: Any, manager: DatumManager | None = None) → Any

Reconstruct the object from decomposed components.

Parameters:

• data – The decomposed component

• manager – Datum manager to internalize datum

Returns:

The reconstructed object

set_file_downloader_class(file_downloader_class)

set_min_size_for_file(size: int)

supported_dots()

Return the Datum Object Types supported by this decomposer. If a DOT is returned, this decomposer’s process_datum method will be called for any datum whose DOT matches this DOT.

Returns: None or list of DOTs