Decks

Decks lets you display customized data visualizations from within your task code. Decks are rendered as HTML and appear right in the Union.ai UI when you run your workflow.

Decks is an opt-in feature; to enable it, set enable_deck to True in the task parameters.

To begin, import the dependencies:

        
    
import union
from flytekit.deck.renderer import MarkdownRenderer
from sklearn.decomposition import PCA

We create a new deck named pca and render Markdown content along with a PCA plot.

Now, declare the required dependnecies in an ImageSpec:

        
    
custom_image = union.ImageSpec(
    packages=[
        "flytekitplugins-deck-standard",
        "markdown",
        "pandas",
        "pillow",
        "plotly",
        "pyarrow",
        "scikit-learn",
        "ydata_profiling",
    ],
    builder="union",
)

Next, we define the task that will construct the figure and create the Deck:

        
    
@union.task(enable_deck=True, container_image=custom_image)
def pca_plot():
    iris_df = px.data.iris()
    X = iris_df[["sepal_length", "sepal_width", "petal_length", "petal_width"]]
    pca = PCA(n_components=3)
    components = pca.fit_transform(X)
    total_var = pca.explained_variance_ratio_.sum() * 100
    fig = px.scatter_3d(
        components,
        x=0,
        y=1,
        z=2,
        color=iris_df["species"],
        title=f"Total Explained Variance: {total_var:.2f}%",
        labels={"0": "PC 1", "1": "PC 2", "2": "PC 3"},
    )
    main_deck = union.Deck("pca", MarkdownRenderer().to_html("### Principal Component Analysis"))
    main_deck.append(plotly.io.to_html(fig))

Note the usage of append to append the Plotly figure to the Markdown deck.

The following is the expected output containing the path to the deck.html file:

        
    
{"asctime": "2023-07-11 13:16:04,558", "name": "flytekit", "levelname": "INFO", "message": "pca_plot task creates flyte deck html to file:///var/folders/6f/xcgm46ds59j7g__gfxmkgdf80000gn/T/flyte-0_8qfjdd/sandbox/local_flytekit/c085853af5a175edb17b11cd338cbd61/deck.html"}

Union deck plot

Once you execute this task on the Union.ai instance, you can access the deck by going to the task view and clicking the Deck button:

Union deck button

Deck tabs

Each Deck has a minimum of three tabs: input, output and default. The input and output tabs are used to render the input and output data of the task, while the default deck can be used to creta cusom renderings such as line plots, scatter plots, Markdown text, etc. Additionally, you can create other tabs as well.

Deck renderers

Frame profiling renderer

The frame profiling render creates a profile report from a Pandas DataFrame.

        
    
import union
import pandas as pd
from flytekitplugins.deck.renderer import FrameProfilingRenderer


@union.task(enable_deck=True, container_image=custom_image)
def frame_renderer() -> None:
    df = pd.DataFrame(data={"col1": [1, 2], "col2": [3, 4]})
    union.Deck("Frame Renderer", FrameProfilingRenderer().to_html(df=df))

Frame renderer

Top-frame renderer

The top-fram renderer renders a DataFrame as an HTML table.

        
    
import union
from typing import Annotated
from flytekit.deck import TopFrameRenderer


@union.task(enable_deck=True, container_image=custom_image)
def top_frame_renderer() -> Annotated[pd.DataFrame, TopFrameRenderer(1)]:
    return pd.DataFrame(data={"col1": [1, 2], "col2": [3, 4]})

Top frame renderer

Markdown renderer

The Markdown renderer converts a Markdown string into HTML.

        
    
import union
from flytekit.deck import MarkdownRenderer


@union.task(enable_deck=True, container_image=custom_image)
def markdown_renderer() -> None:
    union.current_context().default_deck.append(
        MarkdownRenderer().to_html("You can install flytekit using this command: ```import flytekit```")
    )

Markdown renderer

Box renderer

The box renderer groups rows of a DataFrame together into a box-and-whisker mark to visualize their distribution.

Each box extends from the first quartile (Q1) to the third quartile (Q3). The median (Q2) is indicated by a line within the box. Typically, the whiskers extend to the edges of the box, plus or minus 1.5 times the interquartile range (IQR: Q3-Q1).

        
    
import union
from flytekitplugins.deck.renderer import BoxRenderer


@union.task(enable_deck=True, container_image=custom_image)
def box_renderer() -> None:
    iris_df = px.data.iris()
    union.Deck("Box Plot", BoxRenderer("sepal_length").to_html(iris_df))

Box renderer

Image renderer

The image renderer converts a FlyteFile or PIL.Image.Image object into an HTML displayable image, where the image data is encoded as a base64 string.

        
import union
from flytekitplugins.deck.renderer import ImageRenderer

@union.task(enable_deck=True, container_image=custom_image)
def image_renderer(image: union.FlyteFile) -> None:
    union.Deck("Image Renderer", ImageRenderer().to_html(image_src=image))

@union.workflow
def image_renderer_wf(image: union.FlyteFile = "https://bit.ly/3KZ95q4",) -> None:
    image_renderer(image=image)

Image renderer

Table renderer

The table renderer converts a Pandas DataFrame into an HTML table.

        
    
import union
from flytekitplugins.deck.renderer import TableRenderer


@union.task(enable_deck=True, container_image=custom_image)
def table_renderer() -> None:
    union.Deck(
        "Table Renderer",
        TableRenderer().to_html(df=pd.DataFrame(data={"col1": [1, 2], "col2": [3, 4]}), table_width=50),
    )

Table renderer

Custom renderers

YOU can also create your own cusome renderer. A renderer is essentially a class with a to_html method. Here we create custom renderer that summarizes the data from a Pandas DataFRame instead of showing raw values.

        
class DataFrameSummaryRenderer:

    def to_html(self, df: pd.DataFrame) -> str:
        assert isinstance(df, pd.DataFrame)
        return df.describe().to_html()

Then we can use the Annotated type to override the default renderer of the pandas.DataFrame type:

        
    
try:
    from typing import Annotated
except ImportError:
    from typing_extensions import Annotated


@task(enable_deck=True)
def iris_data(
    sample_frac: Optional[float] = None,
    random_state: Optional[int] = None,
) -> Annotated[pd.DataFrame, DataFrameSummaryRenderer()]:
    data = px.data.iris()
    if sample_frac is not None:
        data = data.sample(frac=sample_frac, random_state=random_state)

    md_text = (
        "# Iris Dataset\n"
        "This task loads the iris dataset using the  `plotly` package."
    )
    flytekit.current_context().default_deck.append(MarkdownRenderer().to_html(md_text))
    flytekit.Deck("box plot", BoxRenderer("sepal_length").to_html(data))
    return data

Streaming Decks

You can stream a Deck directly using Deck.publish():

        
import union

@task(enable_deck=True)
def t_deck():
    union.Deck.publish()

This will create a live deck that where you can click the refresh button and see the Deck update until the task succeeds.

Union Deck Succeed Video

When the task fails, you can also see the deck in the UI.

Union Deck Fail Video