etl_lib.task.data_loading.ParquetLoad2Neo4jTask module

class ParquetLoad2Neo4jTask(context, file, model=None, error_file=None, batch_size=5000)[source]

Bases: Task

Load the output of a Parquet file to Neo4j sequentially.

Uses BatchProcessors to read and write data.

Parameters:

context (ETLContext)
file (Path)
model (Type[BaseModel] | None)
error_file (Path | None)
batch_size (int)

__init__(context, file, model=None, error_file=None, batch_size=5000)[source]

Construct a Task object.

Parameters:

context (ETLContext) – ETLContext instance. Will be available to subclasses.
file (Path)
model (Type[BaseModel] | None)
error_file (Path | None)
batch_size (int)

run_internal()[source]

Place to provide the logic to be performed.

This base class provides all the housekeeping and reporting, so that implementation must/should not need to care about them. Exceptions should not be captured by implementations. They are handled by this base class.

Parameters:: kwargs – will be passed to run_internal
Return type:: TaskReturn
Returns:: An instance of TaskReturn.