flytekitplugins.omegaconf.dictconfig_transformer

Directory

Classes

Methods

Variables

Methods

check_if_valid_dictconfig()

def check_if_valid_dictconfig(
    python_val: omegaconf.dictconfig.DictConfig,
)

Validate the DictConfig to ensure it’s serializable.

create_struct()

def create_struct(
    type_map: typing.Dict[str, str],
    value_map: typing.Dict[str, typing.Any],
    base_config: typing.Type,
) -> google.protobuf.struct_pb2.Struct

Create a protobuf Struct object from type and value maps.

extract_type_and_value_maps()

def extract_type_and_value_maps(
    ctx: flytekit.core.context_manager.FlyteContext,
    python_val: omegaconf.dictconfig.DictConfig,
) -> (typing.Dict[str, str], typing.Dict[str, typing.Any])

Extract type and value maps from the DictConfig.

handle_base_dataclass()

def handle_base_dataclass(
    ctx: flytekit.core.context_manager.FlyteContext,
    nested_dict: typing.Dict[str, typing.Any],
    cfg_dict: typing.Dict[str, typing.Any],
) -> omegaconf.dictconfig.DictConfig

Handle the base dataclass and create the DictConfig.

is_flattenable()

def is_flattenable(
    d: omegaconf.dictconfig.DictConfig,
) -> bool

Check if a DictConfig can be properly flattened and unflattened, i.e. keys do not contain dots.

parse_node_value()

def parse_node_value(
    ctx: flytekit.core.context_manager.FlyteContext,
    key: str,
    type_desc: str,
    nested_dict: typing.Dict[str, typing.Any],
) -> typing.Any

Parse the node value from the nested dictionary.

parse_type_description()

def parse_type_description(
    type_desc: str,
) -> typing.Type

Parse the type description and return the corresponding type.

flytekitplugins.omegaconf.dictconfig_transformer.DictConfigTransformer

Base transformer type that should be implemented for every python native type that can be handled by flytekit

def DictConfigTransformer()

Construct DictConfigTransformer.

Methods

assert_type()

def assert_type(
    t: Type[T],
    v: T,
)

from_binary_idl()

def from_binary_idl(
    binary_idl_object: Binary,
    expected_python_type: Type[T],
) -> Optional[T]

This function primarily handles deserialization for untyped dicts, dataclasses, Pydantic BaseModels, and attribute access.｀

For untyped dict, dataclass, and pydantic basemodel: Life Cycle (Untyped Dict as example): python val -> msgpack bytes -> binary literal scalar -> msgpack bytes -> python val (to_literal) (from_binary_idl)

For attribute access: Life Cycle: python val -> msgpack bytes -> binary literal scalar -> resolved golang value -> binary literal scalar -> msgpack bytes -> python val (to_literal) (propeller attribute access) (from_binary_idl)

from_generic_idl()

def from_generic_idl(
    generic: Struct,
    expected_python_type: Type[T],
) -> Optional[T]

TODO: Support all Flyte Types. This is for dataclass attribute access from input created from the Flyte Console.

Note:

• This can be removed in the future when the Flyte Console support generate Binary IDL Scalar as input.

get_literal_type()

def get_literal_type(
    t: typing.Type[omegaconf.dictconfig.DictConfig],
) -> flytekit.models.types.LiteralType

Provide type hint for Flytekit type system.

To support the multivariate typing of nodes in a DictConfig, we encode them as binaries (no introspection) with multiple files.

guess_python_type()

def guess_python_type(
    literal_type: LiteralType,
) -> Type[T]

Converts the Flyte LiteralType to a python object type.

isinstance_generic()

def isinstance_generic(
    obj,
    generic_alias,
)

to_html()

def to_html(
    ctx: FlyteContext,
    python_val: T,
    expected_python_type: Type[T],
) -> str

Converts any python val (dataframe, int, float) to a html string, and it will be wrapped in the HTML div

to_literal()

def to_literal(
    ctx: flytekit.core.context_manager.FlyteContext,
    python_val: ~T,
    python_type: typing.Type[~T],
    expected: flytekit.models.types.LiteralType,
) -> flytekit.models.literals.Literal

Convert from given python type object DictConfig to the Literal representation.

to_python_value()

def to_python_value(
    ctx: flytekit.core.context_manager.FlyteContext,
    lv: flytekit.models.literals.Literal,
    expected_python_type: typing.Type[omegaconf.dictconfig.DictConfig],
) -> omegaconf.dictconfig.DictConfig

Re-hydrate the custom object from Flyte Literal value.

Properties