flytekit.tools.fast_registration

Directory

Classes

Errors

• FlyteDataNotFoundException

flytekit.tools.fast_registration.BarColumn

Renders a visual progress bar.

def BarColumn(
    bar_width: typing.Optional[int],
    style: typing.Union[str, ForwardRef('Style')],
    complete_style: typing.Union[str, ForwardRef('Style')],
    finished_style: typing.Union[str, ForwardRef('Style')],
    pulse_style: typing.Union[str, ForwardRef('Style')],
    table_column: typing.Optional[rich.table.Column],
):

Methods

get_table_column()

def get_table_column()

Get a table column, used to build tasks table.

render()

def render(
    task: Task,
):

Gets a progress bar widget for a task.

flytekit.tools.fast_registration.CopyFileDetection

Create a collection of name/value pairs.

Example enumeration:

class Color(Enum): … RED = 1 … BLUE = 2 … GREEN = 3

Access them by:

• attribute access:

Color.RED <Color.RED: 1>

• value lookup:

Color(1) <Color.RED: 1>

• name lookup:

Color[‘RED’] <Color.RED: 1>

Enumerations can be iterated over, and know how many members they have:

len(Color) 3

list(Color) [<Color.RED: 1>, <Color.BLUE: 2>, <Color.GREEN: 3>]

Methods can be added to enumerations, and members can have their own attributes – see the documentation for details.

flytekit.tools.fast_registration.DockerIgnore

Uses docker-py’s PatternMatcher to check whether a path is ignored.

def DockerIgnore(
    root: pathlib.Path,
):

Methods

is_ignored()

def is_ignored(
    path: str,
):

tar_filter()

def tar_filter(
    tarinfo: tarfile.TarInfo,
):

flytekit.tools.fast_registration.FastPackageOptions

FastPackageOptions is used to set configuration options when packaging files.

def FastPackageOptions(
    ignores: list[Ignore],
    keep_default_ignores: bool,
    copy_style: Optional[CopyFileDetection],
    show_files: bool,
):

flytekit.tools.fast_registration.FlyteContextManager

FlyteContextManager manages the execution context within Flytekit. It holds global state of either compilation or Execution. It is not thread-safe and can only be run as a single threaded application currently. Context’s within Flytekit is useful to manage compilation state and execution state. Refer to CompilationState and ExecutionState for more information. FlyteContextManager provides a singleton stack to manage these contexts.

Typical usage is

.. code-block:: python

FlyteContextManager.initialize() with FlyteContextManager.with_context(o) as ctx: pass

If required - not recommended you can use

FlyteContextManager.push_context()

but correspondingly a pop_context should be called

FlyteContextManager.pop_context()

Methods

add_signal_handler()

def add_signal_handler(
    handler: typing.Callable[[int, FrameType], typing.Any],
):

current_context()

def current_context()

get_origin_stackframe()

def get_origin_stackframe(
    limit,
):

initialize()

def initialize()

Re-initializes the context and erases the entire context

pop_context()

def pop_context()

push_context()

def push_context(
    ctx: FlyteContext,
    f: Optional[traceback.FrameSummary],
):

size()

def size()

with_context()

def with_context(
    b: FlyteContext.Builder,
):

flytekit.tools.fast_registration.FlyteDataNotFoundException

Inappropriate argument value (of correct type).

def FlyteDataNotFoundException(
    path: str,
):

Properties

flytekit.tools.fast_registration.FlyteIgnore

Uses a .flyteignore file to determine ignored files.

def FlyteIgnore(
    root: pathlib.Path,
):

Methods

is_ignored()

def is_ignored(
    path: str,
):

tar_filter()

def tar_filter(
    tarinfo: tarfile.TarInfo,
):

flytekit.tools.fast_registration.GitIgnore

Uses git cli (if available) to list all ignored files and compare with those.

def GitIgnore(
    root: pathlib.Path,
):

Methods

is_ignored()

def is_ignored(
    path: str,
):

tar_filter()

def tar_filter(
    tarinfo: tarfile.TarInfo,
):

flytekit.tools.fast_registration.Ignore

Base for Ignores, implements core logic. Children have to implement _is_ignored

def Ignore(
    root: str,
):

Methods

is_ignored()

def is_ignored(
    path: str,
):

tar_filter()

def tar_filter(
    tarinfo: tarfile.TarInfo,
):

flytekit.tools.fast_registration.IgnoreGroup

Groups multiple Ignores and checks a path against them. A file is ignored if any Ignore considers it ignored.

def IgnoreGroup(
    root: str,
    ignores: typing.List[typing.Type[flytekit.tools.ignore.Ignore]],
):

Methods

is_ignored()

def is_ignored(
    path: str,
):

list_ignored()

def list_ignored()

tar_filter()

def tar_filter(
    tarinfo: tarfile.TarInfo,
):

flytekit.tools.fast_registration.Progress

Renders an auto-updating progress bar(s).

def Progress(
    columns: typing.Union[str, rich.progress.ProgressColumn],
    console: typing.Optional[rich.console.Console],
    auto_refresh: bool,
    refresh_per_second: float,
    speed_estimate_period: float,
    transient: bool,
    redirect_stdout: bool,
    redirect_stderr: bool,
    get_time: typing.Optional[typing.Callable[[], float]],
    disable: bool,
    expand: bool,
):

Methods

add_task()

def add_task(
    description: str,
    start: bool,
    total: typing.Optional[float],
    completed: int,
    visible: bool,
    fields: typing.Any,
):

Add a new ’task’ to the Progress display.

advance()

def advance(
    task_id: rich.progress.TaskID,
    advance: float,
):

Advance task by a number of steps.

get_default_columns()

def get_default_columns()

Get the default columns used for a new Progress instance:

• a text column for the description (TextColumn)
• the bar itself (BarColumn)
• a text column showing completion percentage (TextColumn)
• an estimated-time-remaining column (TimeRemainingColumn) If the Progress instance is created without passing a columns argument, the default columns defined here will be used.

You can also create a Progress instance using custom columns before and/or after the defaults, as in this example:

progress = Progress( SpinnerColumn(), *Progress.get_default_columns(), “Elapsed:”, TimeElapsedColumn(), )

This code shows the creation of a Progress display, containing a spinner to the left, the default columns, and a labeled elapsed time column.

get_renderable()

def get_renderable()

Get a renderable for the progress display.

get_renderables()

def get_renderables()

Get a number of renderables for the progress display.

make_tasks_table()

def make_tasks_table(
    tasks: typing.Iterable[rich.progress.Task],
):

Get a table to render the Progress display.

open()

def open(
    file: typing.Union[str, ForwardRef('PathLike[str]'), bytes],
    mode: typing.Union[typing.Literal['rb'], typing.Literal['rt'], typing.Literal['r']],
    buffering: int,
    encoding: typing.Optional[str],
    errors: typing.Optional[str],
    newline: typing.Optional[str],
    total: typing.Optional[int],
    task_id: typing.Optional[rich.progress.TaskID],
    description: str,
):

Track progress while reading from a binary file.

refresh()

def refresh()

Refresh (render) the progress information.

remove_task()

def remove_task(
    task_id: rich.progress.TaskID,
):

Delete a task if it exists.

reset()

def reset(
    task_id: rich.progress.TaskID,
    start: bool,
    total: typing.Optional[float],
    completed: int,
    visible: typing.Optional[bool],
    description: typing.Optional[str],
    fields: typing.Any,
):

Reset a task so completed is 0 and the clock is reset.

start()

def start()

Start the progress display.

start_task()

def start_task(
    task_id: rich.progress.TaskID,
):

Start a task.

Starts a task (used when calculating elapsed time). You may need to call this manually, if you called add_task with start=False.

stop()

def stop()

Stop the progress display.

stop_task()

def stop_task(
    task_id: rich.progress.TaskID,
):

Stop a task.

This will freeze the elapsed time on the task.

track()

def track(
    sequence: typing.Union[typing.Iterable[~ProgressType], typing.Sequence[~ProgressType]],
    total: typing.Optional[float],
    completed: int,
    task_id: typing.Optional[rich.progress.TaskID],
    description: str,
    update_period: float,
):

Track progress by iterating over a sequence.

update()

def update(
    task_id: rich.progress.TaskID,
    total: typing.Optional[float],
    completed: typing.Optional[float],
    advance: typing.Optional[float],
    description: typing.Optional[str],
    visible: typing.Optional[bool],
    refresh: bool,
    fields: typing.Any,
):

Update information associated with a task.

wrap_file()

def wrap_file(
    file: typing.BinaryIO,
    total: typing.Optional[int],
    task_id: typing.Optional[rich.progress.TaskID],
    description: str,
):

Track progress file reading from a binary file.

Properties

flytekit.tools.fast_registration.StandardIgnore

Retains the standard ignore functionality that previously existed. Could in theory by fed with custom ignore patterns from cli.

def StandardIgnore(
    root: pathlib.Path,
    patterns: typing.Optional[typing.List[str]],
):

Methods

is_ignored()

def is_ignored(
    path: str,
):

tar_filter()

def tar_filter(
    tarinfo: tarfile.TarInfo,
):

flytekit.tools.fast_registration.TextColumn

A column containing text.

def TextColumn(
    text_format: str,
    style: typing.Union[str, ForwardRef('Style')],
    justify: typing.Literal['default', 'left', 'center', 'right', 'full'],
    markup: bool,
    highlighter: typing.Optional[rich.highlighter.Highlighter],
    table_column: typing.Optional[rich.table.Column],
):

Methods

get_table_column()

def get_table_column()

Get a table column, used to build tasks table.

render()

def render(
    task: Task,
):

Should return a renderable object.

flytekit.tools.fast_registration.TimeElapsedColumn

Renders time elapsed.

def TimeElapsedColumn(
    table_column: typing.Optional[rich.table.Column],
):

Methods

get_table_column()

def get_table_column()

Get a table column, used to build tasks table.

render()

def render(
    task: Task,
):

Show time elapsed.

flytekit.tools.fast_registration.Tree

A renderable for a tree structure.

Attributes: ASCII_GUIDES (GuideType): Guide lines used when Console.ascii_only is True. TREE_GUIDES (List[GuideType, GuideType, GuideType]): Default guide lines.

def Tree(
    label: typing.Union[rich.console.ConsoleRenderable, rich.console.RichCast, str],
    style: typing.Union[str, ForwardRef('Style')],
    guide_style: typing.Union[str, ForwardRef('Style')],
    expanded: bool,
    highlight: bool,
    hide_root: bool,
):

Methods

add()

def add(
    label: typing.Union[rich.console.ConsoleRenderable, rich.console.RichCast, str],
    style: typing.Union[str, ForwardRef('Style'), NoneType],
    guide_style: typing.Union[str, ForwardRef('Style'), NoneType],
    expanded: bool,
    highlight: typing.Optional[bool],
):

Add a child tree.

flytekit.tools.fast_registration.timeit

A context manager and a decorator that measures the execution time of the wrapped code block or functions. It will append a timing information to TimeLineDeck. For instance:

@timeit(“Function description”) def function()

with timeit(“Wrapped code block description”):

your code

def timeit(
    name: str,
):