Function preprocess_transfer_tar

pub async fn preprocess_transfer_tar(
    location_cache: &DomainRegistryCache,
    worker_cfg: &WorkerConfig,
    proxy: Arc<ProxyClient>,
    use_case: &str,
    pc: Option<ProgramCounter>,
    workflow: Workflow,
    location: Location,
    dataname: DataName,
    prof: ProfileScopeHandle<'_>,
) -> Result<AccessKind, PreprocessError>

Function that preprocesses by downloading the given tar and extracting it.

Arguments

• location_cache: A cache that is used to efficiently resolve location IDs to registry addresses.
• worker_cfg: The configuration for this node’s environment. For us, contains the path where we may find certificates and where to download data & result files to.
• proxy: The proxy client we use to proxy the data transfer.
• use_case: A string denoting which use-case (registry) we’re using.
• pc: The ProgramCounter of the edge that provides context for this preprocessing. If omitted, should be interpreted as that the context is retrieving the workflow result instead.
• workflow: A Workflow that is given as context to the registry.
• location: The location to download the tarball from.
• dataname: The name of the dataset to download.
• prof: A ProfileScope to provide more detailled information about the time it takes to preprocess a TAR-file.

Returns

The AccessKind to access the extracted data.

Errors

This function can error for literally a million reasons - but they mostly relate to IO (file access, request success etc).