flytekit.core.shim_task

Directory

Classes

Variables

flytekit.core.shim_task.ExecutableTemplateShimTask

The canonical @task decorated Python function task is pretty simple to reason about. At execution time (either locally or on a Flyte cluster), the function runs.

This class, along with the ShimTaskExecutor class below, represents another execution pattern. This pattern, has two components:

• The TaskTemplate, or something like it like a FlyteTask.
• An executor, which can use information from the task template (including the custom field)

Basically at execution time (both locally and on a Flyte cluster), the task template is given to the executor, which is responsible for computing and returning the results.

This class also implements the dispatch_execute and execute functions to make it look like a PythonTask that the entrypoint.py can execute, even though this class doesn’t inherit from PythonTask.

class ExecutableTemplateShimTask(
    tt: _task_model.TaskTemplate,
    executor_type: Type[ShimTaskExecutor],
    args,
    kwargs,
)

Methods

dispatch_execute()

def dispatch_execute(
    ctx: FlyteContext,
    input_literal_map: _literal_models.LiteralMap,
) -> Union[_literal_models.LiteralMap, _dynamic_job.DynamicJobSpec]

This function is largely similar to the base PythonTask, with the exception that we have to infer the Python interface before executing. Also, we refer to self.task_template rather than just self similar to task classes that derive from the base PythonTask.

execute()

def execute(
    kwargs,
) -> Any

Rather than running here, send everything to the executor.

post_execute()

def post_execute(
    _: Optional[ExecutionParameters],
    rval: Any,
) -> Any

This function is a stub, just here to keep dispatch_execute compatibility between this class and PythonTask.

pre_execute()

def pre_execute(
    user_params: Optional[ExecutionParameters],
) -> Optional[ExecutionParameters]

This function is a stub, just here to keep dispatch_execute compatibility between this class and PythonTask.

Properties

flytekit.core.shim_task.ShimTaskExecutor

Please see the notes for the metaclass above first.

This functionality has two use-cases currently,

• Keep track of naming for non-function PythonAutoContainerTasks. That is, things like the flytekit.extras.sqlite3.task.SQLite3Task task.
• Task resolvers, because task resolvers are instances of flytekit.core.python_auto_container.TaskResolverMixin classes, not the classes themselves, which means we need to look on the left hand side of them to see how to find them at task execution time.

class ShimTaskExecutor(
    args,
    kwargs,
)

Methods

execute_from_model()

def execute_from_model(
    tt: _task_model.TaskTemplate,
    kwargs,
) -> n: Python native output values from the task.

This function must be overridden and is where all the business logic for running a task should live. Keep in mind that you’re only working with the TaskTemplate. You won’t have access to any information in the task that wasn’t serialized into the template.

find_lhs()

def find_lhs()

Properties