Flyteidl

flyteidl/core/compiler.proto

CompiledLaunchPlan

Output of the compilation step. This object represents one LaunchPlan. We store more metadata at this layer

Field	Type	Label	Description
template	LaunchPlanTemplate		Completely contained LaunchPlan Template

CompiledTask

Output of the Compilation step. This object represent one Task. We store more metadata at this layer

Field	Type	Label	Description
template	TaskTemplate		Completely contained TaskTemplate

CompiledWorkflow

Output of the compilation Step. This object represents one workflow. We store more metadata at this layer

Field	Type	Label	Description
template	WorkflowTemplate		Completely contained Workflow Template
connections	ConnectionSet		For internal use only! This field is used by the system and must not be filled in. Any values set will be ignored.

CompiledWorkflowClosure

A Compiled Workflow Closure contains all the information required to start a new execution, or to visualize a workflow and its details. The CompiledWorkflowClosure should always contain a primary workflow, that is the main workflow that will being the execution. All subworkflows are denormalized. WorkflowNodes refer to the workflow identifiers of compiled subworkflows.

Field	Type	Label	Description
primary	CompiledWorkflow		+required
sub_workflows	CompiledWorkflow	repeated	Guaranteed that there will only exist one and only one workflow with a given id, i.e., every sub workflow has a unique identifier. Also every enclosed subworkflow is used either by a primary workflow or by a subworkflow as an inlined workflow +optional
tasks	CompiledTask	repeated	Guaranteed that there will only exist one and only one task with a given id, i.e., every task has a unique id +required (at least 1)
launch_plans	CompiledLaunchPlan	repeated	A collection of launch plans that are compiled. Guaranteed that there will only exist one and only one launch plan with a given id, i.e., every launch plan has a unique id.

ConnectionSet

Adjacency list for the workflow. This is created as part of the compilation process. Every process after the compilation step uses this created ConnectionSet

Field	Type	Label	Description
downstream	ConnectionSet.DownstreamEntry	repeated	A list of all the node ids that are downstream from a given node id
upstream	ConnectionSet.UpstreamEntry	repeated	A list of all the node ids, that are upstream of this node id

ConnectionSet.DownstreamEntry

Field	Type	Label	Description
key	string
value	ConnectionSet.IdList

ConnectionSet.IdList

Field	Type	Label	Description
ids	string	repeated

ConnectionSet.UpstreamEntry

Field	Type	Label	Description
key	string
value	ConnectionSet.IdList

flyteidl/core/interface.proto

Parameter

A parameter is used as input to a launch plan and has the special ability to have a default value or mark itself as required.

Field	Type	Label	Description
var	Variable		+required Variable. Defines the type of the variable backing this parameter.
default	Literal		Defines a default value that has to match the variable type defined.
required	bool		+optional, is this value required to be filled.
artifact_query	ArtifactQuery		This is an execution time search basically that should result in exactly one Artifact with a Type that matches the type of the variable.
artifact_id	ArtifactID

ParameterMap

A map of Parameters.

Field	Type	Label	Description
parameters	ParameterMap.ParametersEntry	repeated	Defines a map of parameter names to parameters.

ParameterMap.ParametersEntry

Field	Type	Label	Description
key	string
value	Parameter

TypedInterface

Defines strongly typed inputs and outputs.

Field	Type	Label	Description
inputs	VariableMap
outputs	VariableMap

Variable

Defines a strongly typed variable.

Field	Type	Label	Description
type	LiteralType		Variable literal type.
description	string		+optional string describing input variable
artifact_partial_id	ArtifactID		+optional This object allows the user to specify how Artifacts are created. name, tag, partitions can be specified. The other fields (version and project/domain) are ignored.
artifact_tag	ArtifactTag

VariableMap

A map of Variables

Field	Type	Label	Description
variables	VariableMap.VariablesEntry	repeated	Defines a map of variable names to variables.

VariableMap.VariablesEntry

Field	Type	Label	Description
key	string
value	Variable

flyteidl/core/catalog.proto

CatalogArtifactTag

Field	Type	Label	Description
artifact_id	string		Artifact ID is generated name
name	string		Flyte computes the tag automatically, as the hash of the values

CatalogMetadata

Catalog artifact information with specific metadata

Field	Type	Label	Description
dataset_id	Identifier		Dataset ID in the catalog
artifact_tag	CatalogArtifactTag		Artifact tag in the catalog
source_task_execution	TaskExecutionIdentifier		Today we only support TaskExecutionIdentifier as a source, as catalog caching only works for task executions

CatalogReservation

CatalogCacheStatus

Indicates the status of CatalogCaching. The reason why this is not embedded in TaskNodeMetadata is, that we may use for other types of nodes as well in the future

Name	Number	Description
CACHE_DISABLED	0	Used to indicate that caching was disabled
CACHE_MISS	1	Used to indicate that the cache lookup resulted in no matches
CACHE_HIT	2	used to indicate that the associated artifact was a result of a previous execution
CACHE_POPULATED	3	used to indicate that the resultant artifact was added to the cache
CACHE_LOOKUP_FAILURE	4	Used to indicate that cache lookup failed because of an error
CACHE_PUT_FAILURE	5	Used to indicate that cache lookup failed because of an error
CACHE_SKIPPED	6	Used to indicate the cache lookup was skipped
CACHE_EVICTED	7	Used to indicate that the cache was evicted

CatalogReservation.Status

Indicates the status of a catalog reservation operation.

Name	Number	Description
RESERVATION_DISABLED	0	Used to indicate that reservations are disabled
RESERVATION_ACQUIRED	1	Used to indicate that a reservation was successfully acquired or extended
RESERVATION_EXISTS	2	Used to indicate that an active reservation currently exists
RESERVATION_RELEASED	3	Used to indicate that the reservation has been successfully released
RESERVATION_FAILURE	4	Used to indicate that a reservation operation resulted in failure

flyteidl/core/literals.proto

Binary

A simple byte array with a tag to help different parts of the system communicate about what is in the byte array. It’s strongly advisable that consumers of this type define a unique tag and validate the tag before parsing the data.

Field	Type	Label	Description
value	bytes		Serialized data (MessagePack) for supported types like Dataclass, Pydantic BaseModel, and untyped dict.
tag	string		The serialization format identifier (e.g., MessagePack). Consumers must define unique tags and validate them before deserialization.

Binding

An input/output binding of a variable to either static value or a node output.

Field	Type	Label	Description
var	string		Variable name must match an input/output variable of the node.
binding	BindingData		Data to use to bind this variable.

BindingData

Specifies either a simple value or a reference to another output.

Field	Type	Label	Description
scalar	Scalar		A simple scalar value.
collection	BindingDataCollection		A collection of binding data. This allows nesting of binding data to any number of levels.
promise	OutputReference		References an output promised by another node.
map	BindingDataMap		A map of bindings. The key is always a string.
offloaded_metadata	LiteralOffloadedMetadata		Offloaded literal metadata When you deserialize the offloaded metadata, it would be of Literal and its type would be defined by LiteralType stored in offloaded_metadata. Used for nodes that don’t have promises from upstream nodes such as ArrayNode subNodes.
union	UnionInfo

BindingDataCollection

A collection of BindingData items.

Field	Type	Label	Description
bindings	BindingData	repeated

BindingDataMap

A map of BindingData items.

Field	Type	Label	Description
bindings	BindingDataMap.BindingsEntry	repeated

BindingDataMap.BindingsEntry

Field	Type	Label	Description
key	string
value	BindingData

Blob

Refers to an offloaded set of files. It encapsulates the type of the store and a unique uri for where the data is. There are no restrictions on how the uri is formatted since it will depend on how to interact with the store.

Field	Type	Label	Description
metadata	BlobMetadata
uri	string

BlobMetadata

Field	Type	Label	Description
type	BlobType

KeyValuePair

A generic key value pair.

Field	Type	Label	Description
key	string		required.
value	string		+optional.

Literal

A simple value. This supports any level of nesting (e.g. array of array of array of Blobs) as well as simple primitives.

Field	Type	Label	Description
scalar	Scalar		A simple value.
collection	LiteralCollection		A collection of literals to allow nesting.
map	LiteralMap		A map of strings to literals.
offloaded_metadata	LiteralOffloadedMetadata		Offloaded literal metadata When you deserialize the offloaded metadata, it would be of Literal and its type would be defined by LiteralType stored in offloaded_metadata.
hash	string		A hash representing this literal. This is used for caching purposes. For more details refer to RFC 1893 ( https://github.com/flyteorg/flyte/blob/master/rfc/system/1893-caching-of-offloaded-objects.md)
metadata	Literal.MetadataEntry	repeated	Additional metadata for literals.

Literal.MetadataEntry

Field	Type	Label	Description
key	string
value	string

LiteralCollection

A collection of literals. This is a workaround since oneofs in proto messages cannot contain a repeated field.

Field	Type	Label	Description
literals	Literal	repeated

LiteralMap

A map of literals. This is a workaround since oneofs in proto messages cannot contain a repeated field.

Field	Type	Label	Description
literals	LiteralMap.LiteralsEntry	repeated

LiteralMap.LiteralsEntry

Field	Type	Label	Description
key	string
value	Literal

LiteralOffloadedMetadata

A message that contains the metadata of the offloaded data.

Field	Type	Label	Description
uri	string		The location of the offloaded core.Literal.
size_bytes	uint64		The size of the offloaded data.
inferred_type	LiteralType		The inferred literal type of the offloaded data.

Primitive

Primitive Types

Field	Type	Label	Description
integer	int64
float_value	double
string_value	string
boolean	bool
datetime	google.protobuf.Timestamp
duration	google.protobuf.Duration

RetryStrategy

Retry strategy associated with an executable unit.

Field	Type	Label	Description
retries	uint32		Number of retries. Retries will be consumed when the job fails with a recoverable error. The number of retries must be less than or equals to 10.

Scalar

Field	Type	Label	Description
primitive	Primitive
blob	Blob
binary	Binary
schema	Schema
none_type	Void
error	Error
generic	google.protobuf.Struct
structured_dataset	StructuredDataset
union	Union

Schema

A strongly typed schema that defines the interface of data retrieved from the underlying storage medium.

Field	Type	Label	Description
uri	string
type	SchemaType

StructuredDataset

Field	Type	Label	Description
uri	string		String location uniquely identifying where the data is. Should start with the storage location (e.g. s3://, gs://, bq://, etc.)
metadata	StructuredDatasetMetadata

StructuredDatasetMetadata

Field	Type	Label	Description
structured_dataset_type	StructuredDatasetType		Bundle the type information along with the literal. This is here because StructuredDatasets can often be more defined at run time than at compile time. That is, at compile time you might only declare a task to return a pandas dataframe or a StructuredDataset, without any column information, but at run time, you might have that column information. flytekit python will copy this type information into the literal, from the type information, if not provided by the various plugins (encoders). Since this field is run time generated, it’s not used for any type checking.

Union

The runtime representation of a tagged union value. See UnionType for more details.

Field	Type	Label	Description
value	Literal
type	LiteralType

UnionInfo

Field	Type	Label	Description
targetType	LiteralType

Void

Used to denote a nil/null/None assignment to a scalar value. The underlying LiteralType for Void is intentionally undefined since it can be assigned to a scalar of any LiteralType.

flyteidl/core/tasks.proto

Container

Field	Type	Label	Description
image	string		Container image url. Eg: docker/redis:latest
command	string	repeated	Command to be executed, if not provided, the default entrypoint in the container image will be used.
args	string	repeated	These will default to Flyte given paths. If provided, the system will not append known paths. If the task still needs flyte’s inputs and outputs path, add $(FLYTE_INPUT_FILE), $(FLYTE_OUTPUT_FILE) wherever makes sense and the system will populate these before executing the container.
resources	Resources		Container resources requirement as specified by the container engine.
env	KeyValuePair	repeated	Environment variables will be set as the container is starting up.
config	KeyValuePair	repeated	Deprecated. Allows extra configs to be available for the container. TODO: elaborate on how configs will become available. Deprecated, please use TaskTemplate.config instead.
ports	ContainerPort	repeated	Ports to open in the container. This feature is not supported by all execution engines. (e.g. supported on K8s but not supported on AWS Batch) Only K8s
data_config	DataLoadingConfig		BETA: Optional configuration for DataLoading. If not specified, then default values are used. This makes it possible to to run a completely portable container, that uses inputs and outputs only from the local file-system and without having any reference to flyteidl. This is supported only on K8s at the moment. If data loading is enabled, then data will be mounted in accompanying directories specified in the DataLoadingConfig. If the directories are not specified, inputs will be mounted onto and outputs will be uploaded from a pre-determined file-system path. Refer to the documentation to understand the default paths. Only K8s
architecture	Container.Architecture

ContainerPort

Defines port properties for a container.

Field	Type	Label	Description
container_port	uint32		Number of port to expose on the pod’s IP address. This must be a valid port number, 0 <lt; x <lt; 65536.
name	string		Name of the port to expose on the pod’s IP address.

DataLoadingConfig

This configuration allows executing raw containers in Flyte using the Flyte CoPilot system. Flyte CoPilot, eliminates the needs of flytekit or sdk inside the container. Any inputs required by the users container are side-loaded in the input_path Any outputs generated by the user container - within output_path are automatically uploaded.

Field	Type	Label	Description
enabled	bool		Flag enables DataLoading Config. If this is not set, data loading will not be used!
input_path	string		File system path (start at root). This folder will contain all the inputs exploded to a separate file. Example, if the input interface needs (x: int, y: blob, z: multipart_blob) and the input path is ‘/var/flyte/inputs’, then the file system will look like /var/flyte/inputs/inputs.<lt;metadata format dependent ->gt; .pb .json .yaml>gt; ->gt; Format as defined previously. The Blob and Multipart blob will reference local filesystem instead of remote locations /var/flyte/inputs/x ->gt; X is a file that contains the value of x (integer) in string format /var/flyte/inputs/y ->gt; Y is a file in Binary format /var/flyte/inputs/z/… ->gt; Note Z itself is a directory More information about the protocol - refer to docs #TODO reference docs here
output_path	string		File system path (start at root). This folder should contain all the outputs for the task as individual files and/or an error text file
format	DataLoadingConfig.LiteralMapFormat		In the inputs folder, there will be an additional summary/metadata file that contains references to all files or inlined primitive values. This format decides the actual encoding for the data. Refer to the encoding to understand the specifics of the contents and the encoding
io_strategy	IOStrategy

ExtendedResources

Encapsulates all non-standard resources, not captured by v1.ResourceRequirements, to allocate to a task.

Field	Type	Label	Description
gpu_accelerator	GPUAccelerator		GPU accelerator to select for task. Contains information about device type, and for multi-instance GPUs, the partition size to use.
shared_memory	SharedMemory

GPUAccelerator

Metadata associated with the GPU accelerator to allocate to a task. Contains information about device type, and for multi-instance GPUs, the partition size to use.

Field	Type	Label	Description
device	string		This can be any arbitrary string, and should be informed by the labels or taints associated with the nodes in question. Default cloud provider labels typically use the following values: nvidia-tesla-t4, nvidia-tesla-a100, etc.
unpartitioned	bool
partition_size	string		Like device, this can be any arbitrary string, and should be informed by the labels or taints associated with the nodes in question. Default cloud provider labels typically use the following values: 1g.5gb, 2g.10gb, etc.

IOStrategy

Strategy to use when dealing with Blob, Schema, or multipart blob data (large datasets)

Field	Type	Label	Description
download_mode	IOStrategy.DownloadMode		Mode to use to manage downloads
upload_mode	IOStrategy.UploadMode		Mode to use to manage uploads

K8sObjectMetadata

Metadata for building a kubernetes object when a task is executed.

Field	Type	Label	Description
labels	K8sObjectMetadata.LabelsEntry	repeated	Optional labels to add to the pod definition.
annotations	K8sObjectMetadata.AnnotationsEntry	repeated	Optional annotations to add to the pod definition.

K8sObjectMetadata.AnnotationsEntry

Field	Type	Label	Description
key	string
value	string

K8sObjectMetadata.LabelsEntry

Field	Type	Label	Description
key	string
value	string

K8sPod

Defines a pod spec and additional pod metadata that is created when a task is executed.

Field	Type	Label	Description
metadata	K8sObjectMetadata		Contains additional metadata for building a kubernetes pod.
pod_spec	google.protobuf.Struct		Defines the primary pod spec created when a task is executed. This should be a JSON-marshalled pod spec, which can be defined in - go, using: https://github.com/kubernetes/api/blob/release-1.21/core/v1/types.go#L2936 - python: using https://github.com/kubernetes-client/python/blob/release-19.0/kubernetes/client/models/v1_pod_spec.py
data_config	DataLoadingConfig		BETA: Optional configuration for DataLoading. If not specified, then default values are used. This makes it possible to to run a completely portable container, that uses inputs and outputs only from the local file-system and without having any reference to flytekit. This is supported only on K8s at the moment. If data loading is enabled, then data will be mounted in accompanying directories specified in the DataLoadingConfig. If the directories are not specified, inputs will be mounted onto and outputs will be uploaded from a pre-determined file-system path. Refer to the documentation to understand the default paths. Only K8s
primary_container_name	string		Defines the primary container name when pod template override is executed.

Resources

A customizable interface to convey resources requested for a container. This can be interpreted differently for different container engines.

Field	Type	Label	Description
requests	Resources.ResourceEntry	repeated	The desired set of resources requested. ResourceNames must be unique within the list.
limits	Resources.ResourceEntry	repeated	Defines a set of bounds (e.g. min/max) within which the task can reliably run. ResourceNames must be unique within the list.

Resources.ResourceEntry

Encapsulates a resource name and value.

Field	Type	Label	Description
name	Resources.ResourceName		Resource name.
value	string		Value must be a valid k8s quantity. See https://github.com/kubernetes/apimachinery/blob/master/pkg/api/resource/quantity.go#L30-L80

RuntimeMetadata

Runtime information. This is loosely defined to allow for extensibility.

Field	Type	Label	Description
type	RuntimeMetadata.RuntimeType		Type of runtime.
version	string		Version of the runtime. All versions should be backward compatible. However, certain cases call for version checks to ensure tighter validation or setting expectations.
flavor	string		+optional It can be used to provide extra information about the runtime (e.g. python, golang… etc.).

SharedMemory

Metadata associated with configuring a shared memory volume for a task.

Field	Type	Label	Description
mount_path	string		Mount path to place in container
mount_name	string		Name for volume
size_limit	string		Size limit for shared memory. If not set, then the shared memory is equal to the allocated memory. +optional

Sql

Sql represents a generic sql workload with a statement and dialect.

Field	Type	Label	Description
statement	string		The actual query to run, the query can have templated parameters. We use Flyte’s Golang templating format for Query templating. For example, insert overwrite directory ‘{{ .rawOutputDataPrefix }}’ stored as parquet select * from my_table where ds = ‘{{ .Inputs.ds }}’
dialect	Sql.Dialect

TaskMetadata

Task Metadata

Field	Type	Label	Description
discoverable	bool		Indicates whether the system should attempt to lookup this task’s output to avoid duplication of work.
runtime	RuntimeMetadata		Runtime information about the task.
timeout	google.protobuf.Duration		The overall timeout of a task including user-triggered retries.
retries	RetryStrategy		Number of retries per task.
discovery_version	string		Indicates a logical version to apply to this task for the purpose of discovery.
deprecated_error_message	string		If set, this indicates that this task is deprecated. This will enable owners of tasks to notify consumers of the ending of support for a given task.
interruptible	bool
cache_serializable	bool		Indicates whether the system should attempt to execute discoverable instances in serial to avoid duplicate work
tags	TaskMetadata.TagsEntry	repeated	Arbitrary tags that allow users and the platform to store small but arbitrary labels
pod_template_name	string		pod_template_name is the unique name of a PodTemplate k8s resource to be used as the base configuration if this task creates a k8s Pod. If this value is set, the specified PodTemplate will be used instead of, but applied identically as, the default PodTemplate configured in FlytePropeller.
cache_ignore_input_vars	string	repeated	cache_ignore_input_vars is the input variables that should not be included when calculating hash for cache.
is_eager	bool		is_eager indicates whether the task is eager or not. This would be used by CreateTask endpoint.
generates_deck	google.protobuf.BoolValue		Indicates whether the task will generate a deck when it finishes executing. The BoolValue can have three states: - nil: The value is not set. - true: The task will generate a deck. - false: The task will not generate a deck.

TaskMetadata.TagsEntry

Field	Type	Label	Description
key	string
value	string

TaskTemplate

A Task structure that uniquely identifies a task in the system Tasks are registered as a first step in the system.

Field	Type	Label	Description
id	Identifier		Auto generated taskId by the system. Task Id uniquely identifies this task globally.
type	string		A predefined yet extensible Task type identifier. This can be used to customize any of the components. If no extensions are provided in the system, Flyte will resolve the this task to its TaskCategory and default the implementation registered for the TaskCategory.
metadata	TaskMetadata		Extra metadata about the task.
interface	TypedInterface		A strongly typed interface for the task. This enables others to use this task within a workflow and guarantees compile-time validation of the workflow to avoid costly runtime failures.
custom	google.protobuf.Struct		Custom data about the task. This is extensible to allow various plugins in the system.
container	Container
k8s_pod	K8sPod
sql	Sql
task_type_version	int32		This can be used to customize task handling at execution time for the same task type.
security_context	SecurityContext		security_context encapsulates security attributes requested to run this task.
extended_resources	ExtendedResources		Encapsulates all non-standard resources, not captured by v1.ResourceRequirements, to allocate to a task.
config	TaskTemplate.ConfigEntry	repeated	Metadata about the custom defined for this task. This is extensible to allow various plugins in the system to use as required. reserve the field numbers 1 through 15 for very frequently occurring message elements

TaskTemplate.ConfigEntry

Field	Type	Label	Description
key	string
value	string

Container.Architecture

Architecture-type the container image supports.

Name	Number	Description
UNKNOWN	0
AMD64	1
ARM64	2
ARM_V6	3
ARM_V7	4

DataLoadingConfig.LiteralMapFormat

LiteralMapFormat decides the encoding format in which the input metadata should be made available to the containers. If the user has access to the protocol buffer definitions, it is recommended to use the PROTO format. JSON and YAML do not need any protobuf definitions to read it All remote references in core.LiteralMap are replaced with local filesystem references (the data is downloaded to local filesystem)

Name	Number	Description
JSON	0	JSON / YAML for the metadata (which contains inlined primitive values). The representation is inline with the standard json specification as specified - https://www.json.org/json-en.html
YAML	1
PROTO	2	Proto is a serialized binary of core.LiteralMap defined in flyteidl/core

IOStrategy.DownloadMode

Mode to use for downloading

Name	Number	Description
DOWNLOAD_EAGER	0	All data will be downloaded before the main container is executed
DOWNLOAD_STREAM	1	Data will be downloaded as a stream and an End-Of-Stream marker will be written to indicate all data has been downloaded. Refer to protocol for details
DO_NOT_DOWNLOAD	2	Large objects (offloaded) will not be downloaded

IOStrategy.UploadMode

Mode to use for uploading

Name	Number	Description
UPLOAD_ON_EXIT	0	All data will be uploaded after the main container exits
UPLOAD_EAGER	1	Data will be uploaded as it appears. Refer to protocol specification for details
DO_NOT_UPLOAD	2	Data will not be uploaded, only references will be written

Resources.ResourceName

Known resource names.

Name	Number	Description
UNKNOWN	0
CPU	1
GPU	2
MEMORY	3
STORAGE	4
EPHEMERAL_STORAGE	5	For Kubernetes-based deployments, pods use ephemeral local storage for scratch space, caching, and for logs.

RuntimeMetadata.RuntimeType

Name	Number	Description
OTHER	0
FLYTE_SDK	1

Sql.Dialect

The dialect of the SQL statement. This is used to validate and parse SQL statements at compilation time to avoid expensive runtime operations. If set to an unsupported dialect, no validation will be done on the statement. We support the following dialect: ansi, hive.

Name	Number	Description
UNDEFINED	0
ANSI	1
HIVE	2
OTHER	3

flyteidl/core/metrics.proto

ExecutionMetricResult

ExecutionMetrics is a collection of metrics that are collected during the execution of a Flyte task.

Field	Type	Label	Description
metric	string		The metric this data represents. e.g. EXECUTION_METRIC_USED_CPU_AVG or EXECUTION_METRIC_USED_MEMORY_BYTES_AVG.
data	google.protobuf.Struct		The result data in prometheus range query result format https://prometheus.io/docs/prometheus/latest/querying/api/#expression-query-result-formats. This may include multiple time series, differentiated by their metric labels. Start time is greater of (execution attempt start, 48h ago) End time is lesser of (execution attempt end, now)

Span

Span represents a duration trace of Flyte execution. The id field denotes a Flyte execution entity or an operation which uniquely identifies the Span. The spans attribute allows this Span to be further broken down into more precise definitions.

Field	Type	Label	Description
start_time	google.protobuf.Timestamp		start_time defines the instance this span began.
end_time	google.protobuf.Timestamp		end_time defines the instance this span completed.
workflow_id	WorkflowExecutionIdentifier		workflow_id is the id of the workflow execution this Span represents.
node_id	NodeExecutionIdentifier		node_id is the id of the node execution this Span represents.
task_id	TaskExecutionIdentifier		task_id is the id of the task execution this Span represents.
operation_id	string		operation_id is the id of a unique operation that this Span represents.
spans	Span	repeated	spans defines a collection of Spans that breakdown this execution.

flyteidl/core/errors.proto

ContainerError

Error message to propagate detailed errors from container executions to the execution engine.

Field	Type	Label	Description
code	string		A simplified code for errors, so that we can provide a glossary of all possible errors.
message	string		A detailed error message.
kind	ContainerError.Kind		An abstract error kind for this error. Defaults to Non_Recoverable if not specified.
origin	ExecutionError.ErrorKind		Defines the origin of the error (system, user, unknown).
timestamp	google.protobuf.Timestamp		Timestamp of the error
worker	string		Worker that generated the error

ErrorDocument

Defines the errors.pb file format the container can produce to communicate failure reasons to the execution engine.

Field	Type	Label	Description
error	ContainerError		The error raised during execution.

ContainerError.Kind

Defines a generic error type that dictates the behavior of the retry strategy.

Name	Number	Description
NON_RECOVERABLE	0
RECOVERABLE	1

flyteidl/core/identifier.proto

Identifier

Encapsulation of fields that uniquely identifies a Flyte resource.

Field	Type	Label	Description
resource_type	ResourceType		Identifies the specific type of resource that this identifier corresponds to.
project	string		Name of the project the resource belongs to.
domain	string		Name of the domain the resource belongs to. A domain can be considered as a subset within a specific project.
name	string		User provided value for the resource.
version	string		Specific version of the resource.
org	string		Optional, org key applied to the resource.

NodeExecutionIdentifier

Encapsulation of fields that identify a Flyte node execution entity.

Field	Type	Label	Description
node_id	string
execution_id	WorkflowExecutionIdentifier

SignalIdentifier

Encapsulation of fields the uniquely identify a signal.

Field	Type	Label	Description
signal_id	string		Unique identifier for a signal.
execution_id	WorkflowExecutionIdentifier		Identifies the Flyte workflow execution this signal belongs to.

TaskExecutionIdentifier

Encapsulation of fields that identify a Flyte task execution entity.

Field	Type	Label	Description
task_id	Identifier
node_execution_id	NodeExecutionIdentifier
retry_attempt	uint32

WorkflowExecutionIdentifier

Encapsulation of fields that uniquely identifies a Flyte workflow execution

Field	Type	Label	Description
project	string		Name of the project the resource belongs to.
domain	string		Name of the domain the resource belongs to. A domain can be considered as a subset within a specific project.
name	string		User or system provided value for the resource.
org	string		Optional, org key applied to the resource.

ResourceType

Indicates a resource type within Flyte.

Name	Number	Description
UNSPECIFIED	0
TASK	1
WORKFLOW	2
LAUNCH_PLAN	3
DATASET	4	A dataset represents an entity modeled in Flyte DataCatalog. A Dataset is also a versioned entity and can be a compilation of multiple individual objects. Eventually all Catalog objects should be modeled similar to Flyte Objects. The Dataset entities makes it possible for the UI and CLI to act on the objects in a similar manner to other Flyte objects

flyteidl/core/artifact_id.proto

ArtifactBindingData

Only valid for triggers

Field	Type	Label	Description
partition_key	string
bind_to_time_partition	bool
time_transform	TimeTransform		This is only relevant in the time partition case

ArtifactID

Field	Type	Label	Description
artifact_key	ArtifactKey
version	string
partitions	Partitions		Think of a partition as a tag on an Artifact, except it’s a key-value pair. Different partitions naturally have different versions (execution ids).
time_partition	TimePartition		There is no such thing as an empty time partition - if it’s not set, then there is no time partition.

ArtifactKey

Field	Type	Label	Description
project	string		Project and domain and suffix needs to be unique across a given artifact store.
domain	string
name	string
org	string

ArtifactQuery

Uniqueness constraints for Artifacts

• project, domain, name, version, partitions Option 2 (tags are standalone, point to an individual artifact id):
• project, domain, name, alias (points to one partition if partitioned)
• project, domain, name, partition key, partition value

Field	Type	Label	Description
artifact_id	ArtifactID
artifact_tag	ArtifactTag
uri	string
binding	ArtifactBindingData		This is used in the trigger case, where a user specifies a value for an input that is one of the triggering artifacts, or a partition value derived from a triggering artifact.

ArtifactTag

Field	Type	Label	Description
artifact_key	ArtifactKey
value	LabelValue

InputBindingData

Field	Type	Label	Description
var	string

LabelValue

Field	Type	Label	Description
static_value	string		The string static value is for use in the Partitions object
time_value	google.protobuf.Timestamp		The time value is for use in the TimePartition case
triggered_binding	ArtifactBindingData
input_binding	InputBindingData
runtime_binding	RuntimeBinding

Partitions

Field	Type	Label	Description
value	Partitions.ValueEntry	repeated

Partitions.ValueEntry

Field	Type	Label	Description
key	string
value	LabelValue

RuntimeBinding

TimePartition

Field	Type	Label	Description
value	LabelValue
granularity	Granularity

TimeTransform

Field	Type	Label	Description
transform	string
op	Operator

Granularity

Name	Number	Description
UNSET	0
MINUTE	1
HOUR	2
DAY	3	default
MONTH	4

Operator

Name	Number	Description
MINUS	0
PLUS	1

flyteidl/core/types.proto

BlobType

Defines type behavior for blob objects

Field	Type	Label	Description
format	string		Format can be a free form string understood by SDK/UI etc like csv, parquet etc
dimensionality	BlobType.BlobDimensionality

EnumType

Enables declaring enum types, with predefined string values For len(values) >gt; 0, the first value in the ordered list is regarded as the default value. If you wish To provide no defaults, make the first value as undefined.

Field	Type	Label	Description
values	string	repeated	Predefined set of enum values.

Error

Represents an error thrown from a node.

Field	Type	Label	Description
failed_node_id	string		The node id that threw the error.
message	string		Error message thrown.

LiteralType

Defines a strong type to allow type checking between interfaces.

Field	Type	Label	Description
simple	SimpleType		A simple type that can be compared one-to-one with another.
schema	SchemaType		A complex type that requires matching of inner fields.
collection_type	LiteralType		Defines the type of the value of a collection. Only homogeneous collections are allowed.
map_value_type	LiteralType		Defines the type of the value of a map type. The type of the key is always a string.
blob	BlobType		A blob might have specialized implementation details depending on associated metadata.
enum_type	EnumType		Defines an enum with pre-defined string values.
structured_dataset_type	StructuredDatasetType		Generalized schema support
union_type	UnionType		Defines an union type with pre-defined LiteralTypes.
metadata	google.protobuf.Struct		This field contains type metadata that is descriptive of the type, but is NOT considered in type-checking. This might be used by consumers to identify special behavior or display extended information for the type.
annotation	TypeAnnotation		This field contains arbitrary data that might have special semantic meaning for the client but does not effect internal flyte behavior.
structure	TypeStructure		Hints to improve type matching.

OutputReference

A reference to an output produced by a node. The type can be retrieved -and validated- from the underlying interface of the node.

Field	Type	Label	Description
node_id	string		Node id must exist at the graph layer.
var	string		Variable name must refer to an output variable for the node.
attr_path	PromiseAttribute	repeated

PromiseAttribute

Field	Type	Label	Description
string_value	string
int_value	int32

SchemaType

Defines schema columns and types to strongly type-validate schemas interoperability.

Field	Type	Label	Description
columns	SchemaType.SchemaColumn	repeated	A list of ordered columns this schema comprises of.

SchemaType.SchemaColumn

Field	Type	Label	Description
name	string		A unique name -within the schema type- for the column
type	SchemaType.SchemaColumn.SchemaColumnType		The column type. This allows a limited set of types currently.

StructuredDatasetType

Field	Type	Label	Description
columns	StructuredDatasetType.DatasetColumn	repeated	A list of ordered columns this schema comprises of.
format	string		This is the storage format, the format of the bits at rest parquet, feather, csv, etc. For two types to be compatible, the format will need to be an exact match.
external_schema_type	string		This is a string representing the type that the bytes in external_schema_bytes are formatted in. This is an optional field that will not be used for type checking.
external_schema_bytes	bytes		The serialized bytes of a third-party schema library like Arrow. This is an optional field that will not be used for type checking.

StructuredDatasetType.DatasetColumn

Field	Type	Label	Description
name	string		A unique name within the schema type for the column.
literal_type	LiteralType		The column type.

TypeAnnotation

TypeAnnotation encapsulates registration time information about a type. This can be used for various control-plane operations. TypeAnnotation will not be available at runtime when a task runs.

Field	Type	Label	Description
annotations	google.protobuf.Struct		A arbitrary JSON payload to describe a type.

TypeStructure

Hints to improve type matching e.g. allows distinguishing output from custom type transformers even if the underlying IDL serialization matches.

Field	Type	Label	Description
tag	string		Must exactly match for types to be castable
dataclass_type	TypeStructure.DataclassTypeEntry	repeated	dataclass_type only exists for dataclasses. This is used to resolve the type of the fields of dataclass The key is the field name, and the value is the literal type of the field e.g. For dataclass Foo, with fields a, and a is a string Foo.a will be resolved as a literal type of string from dataclass_type

TypeStructure.DataclassTypeEntry

Field	Type	Label	Description
key	string
value	LiteralType

UnionType

Defines a tagged union type, also known as a variant (and formally as the sum type).

A sum type S is defined by a sequence of types (A, B, C, …), each tagged by a string tag A value of type S is constructed from a value of any of the variant types. The specific choice of type is recorded by storing the varaint’s tag with the literal value and can be examined in runtime.

Type S is typically written as S := Apple A | Banana B | Cantaloupe C | …

Notably, a nullable (optional) type is a sum type between some type X and the singleton type representing a null-value: Optional X := X | Null

See also: https://en.wikipedia.org/wiki/Tagged_union

Field	Type	Label	Description
variants	LiteralType	repeated	Predefined set of variants in union.

BlobType.BlobDimensionality

Name	Number	Description
SINGLE	0
MULTIPART	1

SchemaType.SchemaColumn.SchemaColumnType

Name	Number	Description
INTEGER	0
FLOAT	1
STRING	2
BOOLEAN	3
DATETIME	4
DURATION	5

SimpleType

Define a set of simple types.

Name	Number	Description
NONE	0
INTEGER	1
FLOAT	2
STRING	3
BOOLEAN	4
DATETIME	5
DURATION	6
BINARY	7
ERROR	8
STRUCT	9

flyteidl/core/execution_envs.proto

ExecutionEnv

ExecutionEnv is a message that is used to specify the execution environment.

Field	Type	Label	Description
name	string		name is a human-readable identifier for the execution environment. This is combined with the project, domain, and version to uniquely identify an execution environment.
type	string		type is the type of the execution environment.
extant	google.protobuf.Struct		extant is a reference to an existing environment.
spec	google.protobuf.Struct		spec is a specification of the environment.
version	string		version is the version of the execution environment. This may be used differently by each individual environment type (ex. auto-generated or manually provided), but is intended to allow variance in environment specifications with the same ID.

ExecutionEnvAssignment

ExecutionEnvAssignment is a message that is used to assign an execution environment to a set of nodes.

Field	Type	Label	Description
node_ids	string	repeated	node_ids is a list of node ids that are being assigned the execution environment.
task_type	string		task_type is the type of task that is being assigned. This is used to override which Flyte plugin will be used during execution.
execution_env	ExecutionEnv		execution_env is the environment that is being assigned to the nodes.

flyteidl/core/execution.proto

ExecutionError

Represents the error message from the execution.

Field	Type	Label	Description
code	string		Error code indicates a grouping of a type of error. More Info: <lt;Link>gt;
message	string		Detailed description of the error - including stack trace.
error_uri	string		Full error contents accessible via a URI
kind	ExecutionError.ErrorKind
timestamp	google.protobuf.Timestamp		Timestamp of the error
worker	string		Worker that generated the error

NodeExecution

Indicates various phases of Node Execution that only include the time spent to run the nodes/workflows

QualityOfService

Indicates the priority of an execution.

Field	Type	Label	Description
tier	QualityOfService.Tier
spec	QualityOfServiceSpec

QualityOfServiceSpec

Represents customized execution run-time attributes.

Field	Type	Label	Description
queueing_budget	google.protobuf.Duration		Indicates how much queueing delay an execution can tolerate.

TaskExecution

Phases that task plugins can go through. Not all phases may be applicable to a specific plugin task, but this is the cumulative list that customers may want to know about for their task.

TaskLog

Log information for the task that is specific to a log sink When our log story is flushed out, we may have more metadata here like log link expiry

Field	Type	Label	Description
uri	string
name	string
message_format	TaskLog.MessageFormat
ttl	google.protobuf.Duration
ShowWhilePending	bool
HideOnceFinished	bool

WorkflowExecution

Indicates various phases of Workflow Execution

ExecutionError.ErrorKind

Error type: System or User

Name	Number	Description
UNKNOWN	0
USER	1
SYSTEM	2

NodeExecution.Phase

Name	Number	Description
UNDEFINED	0
QUEUED	1
RUNNING	2
SUCCEEDED	3
FAILING	4
FAILED	5
ABORTED	6
SKIPPED	7
TIMED_OUT	8
DYNAMIC_RUNNING	9
RECOVERED	10

QualityOfService.Tier

Name	Number	Description
UNDEFINED	0	Default: no quality of service specified.
HIGH	1
MEDIUM	2
LOW	3

TaskExecution.Phase

Name	Number	Description
UNDEFINED	0
QUEUED	1
RUNNING	2
SUCCEEDED	3
ABORTED	4
FAILED	5
INITIALIZING	6	To indicate cases where task is initializing, like: ErrImagePull, ContainerCreating, PodInitializing
WAITING_FOR_RESOURCES	7	To address cases, where underlying resource is not available: Backoff error, Resource quota exceeded

TaskLog.MessageFormat

Name	Number	Description
UNKNOWN	0
CSV	1
JSON	2

WorkflowExecution.Phase

Name	Number	Description
UNDEFINED	0
QUEUED	1
RUNNING	2
SUCCEEDING	3
SUCCEEDED	4
FAILING	5
FAILED	6
ABORTED	7
TIMED_OUT	8
ABORTING	9

flyteidl/core/security.proto

Identity

Identity encapsulates the various security identities a task can run as. It’s up to the underlying plugin to pick the right identity for the execution environment.

Field	Type	Label	Description
iam_role	string		iam_role references the fully qualified name of Identity & Access Management role to impersonate.
k8s_service_account	string		k8s_service_account references a kubernetes service account to impersonate.
oauth2_client	OAuth2Client		oauth2_client references an oauth2 client. Backend plugins can use this information to impersonate the client when making external calls.
execution_identity	string		execution_identity references the subject who makes the execution

OAuth2Client

OAuth2Client encapsulates OAuth2 Client Credentials to be used when making calls on behalf of that task.

Field	Type	Label	Description
client_id	string		client_id is the public id for the client to use. The system will not perform any pre-auth validation that the secret requested matches the client_id indicated here. +required
client_secret	Secret		client_secret is a reference to the secret used to authenticate the OAuth2 client. +required

OAuth2TokenRequest

OAuth2TokenRequest encapsulates information needed to request an OAuth2 token. FLYTE_TOKENS_ENV_PREFIX will be passed to indicate the prefix of the environment variables that will be present if tokens are passed through environment variables. FLYTE_TOKENS_PATH_PREFIX will be passed to indicate the prefix of the path where secrets will be mounted if tokens are passed through file mounts.

Field	Type	Label	Description
name	string		name indicates a unique id for the token request within this task token requests. It’ll be used as a suffix for environment variables and as a filename for mounting tokens as files. +required
type	OAuth2TokenRequest.Type		type indicates the type of the request to make. Defaults to CLIENT_CREDENTIALS. +required
client	OAuth2Client		client references the client_id/secret to use to request the OAuth2 token. +required
idp_discovery_endpoint	string		idp_discovery_endpoint references the discovery endpoint used to retrieve token endpoint and other related information. +optional
token_endpoint	string		token_endpoint references the token issuance endpoint. If idp_discovery_endpoint is not provided, this parameter is mandatory. +optional

Secret

Secret encapsulates information about the secret a task needs to proceed. An environment variable FLYTE_SECRETS_ENV_PREFIX will be passed to indicate the prefix of the environment variables that will be present if secrets are passed through environment variables. FLYTE_SECRETS_DEFAULT_DIR will be passed to indicate the prefix of the path where secrets will be mounted if secrets are passed through file mounts.

Field	Type	Label	Description
group	string		The name of the secret group where to find the key referenced below. For K8s secrets, this should be the name of the v1/secret object. For Confidant, this should be the Credential name. For Vault, this should be the secret name. For AWS Secret Manager, this should be the name of the secret. +required
group_version	string		The group version to fetch. This is not supported in all secret management systems. It’ll be ignored for the ones that do not support it. +optional
key	string		The name of the secret to mount. This has to match an existing secret in the system. It’s up to the implementation of the secret management system to require case sensitivity. For K8s secrets, Confidant and Vault, this should match one of the keys inside the secret. For AWS Secret Manager, it’s ignored. +optional
mount_requirement	Secret.MountType		mount_requirement is optional. Indicates where the secret has to be mounted. If provided, the execution will fail if the underlying key management system cannot satisfy that requirement. If not provided, the default location will depend on the key management system. +optional
env_var	string		env_var is optional. Custom environment variable to set the value of the secret. If mount_requirement is ENV_VAR, then the value is the secret itself. If mount_requirement is FILE, then the value is the path to the secret file. +optional

SecurityContext

SecurityContext holds security attributes that apply to tasks.

Field	Type	Label	Description
run_as	Identity		run_as encapsulates the identity a pod should run as. If the task fills in multiple fields here, it’ll be up to the backend plugin to choose the appropriate identity for the execution engine the task will run on.
secrets	Secret	repeated	secrets indicate the list of secrets the task needs in order to proceed. Secrets will be mounted/passed to the pod as it starts. If the plugin responsible for kicking of the task will not run it on a flyte cluster (e.g. AWS Batch), it’s the responsibility of the plugin to fetch the secret (which means propeller identity will need access to the secret) and to pass it to the remote execution engine.
tokens	OAuth2TokenRequest	repeated	tokens indicate the list of token requests the task needs in order to proceed. Tokens will be mounted/passed to the pod as it starts. If the plugin responsible for kicking of the task will not run it on a flyte cluster (e.g. AWS Batch), it’s the responsibility of the plugin to fetch the secret (which means propeller identity will need access to the secret) and to pass it to the remote execution engine.

OAuth2TokenRequest.Type

Type of the token requested.

Name	Number	Description
CLIENT_CREDENTIALS	0	CLIENT_CREDENTIALS indicates a 2-legged OAuth token requested using client credentials.

Secret.MountType

Name	Number	Description
ANY	0	Default case, indicates the client can tolerate either mounting options.
ENV_VAR	1	ENV_VAR indicates the secret needs to be mounted as an environment variable.
FILE	2	FILE indicates the secret needs to be mounted as a file.

flyteidl/core/workflow.proto

Alias

Links a variable to an alias.

Field	Type	Label	Description
var	string		Must match one of the output variable names on a node.
alias	string		A workflow-level unique alias that downstream nodes can refer to in their input.

ApproveCondition

ApproveCondition represents a dependency on an external approval. During execution, this will manifest as a boolean signal with the provided signal_id.

Field	Type	Label	Description
signal_id	string		A unique identifier for the requested boolean signal.

ArrayNode

ArrayNode is a Flyte node type that simplifies the execution of a sub-node over a list of input values. An ArrayNode can be executed with configurable parallelism (separate from the parent workflow) and can be configured to succeed when a certain number of sub-nodes succeed.

Field	Type	Label	Description
node	Node		node is the sub-node that will be executed for each element in the array.
parallelism	uint32		parallelism defines the minimum number of instances to bring up concurrently at any given point. Note that this is an optimistic restriction and that, due to network partitioning or other failures, the actual number of currently running instances might be more. This has to be a positive number if assigned. Default value is size.
min_successes	uint32		min_successes is an absolute number of the minimum number of successful completions of sub-nodes. As soon as this criteria is met, the ArrayNode will be marked as successful and outputs will be computed. This has to be a non-negative number if assigned. Default value is size (if specified).
min_success_ratio	float		If the array job size is not known beforehand, the min_success_ratio can instead be used to determine when an ArrayNode can be marked successful.
execution_mode	ArrayNode.ExecutionMode		execution_mode determines the execution path for ArrayNode.
is_original_sub_node_interface	google.protobuf.BoolValue		Indicates whether the sub node’s original interface was altered
data_mode	ArrayNode.DataMode		data_mode determines how input data is passed to the sub-nodes
bound_inputs	string	repeated	+optional. Specifies input bindings that are not mapped over for the node.

BranchNode

BranchNode is a special node that alter the flow of the workflow graph. It allows the control flow to branch at runtime based on a series of conditions that get evaluated on various parameters (e.g. inputs, primitives).

Field	Type	Label	Description
if_else	IfElseBlock		+required

GateNode

GateNode refers to the condition that is required for the gate to successfully complete.

Field	Type	Label	Description
approve	ApproveCondition		ApproveCondition represents a dependency on an external approval provided by a boolean signal.
signal	SignalCondition		SignalCondition represents a dependency on an signal.
sleep	SleepCondition		SleepCondition represents a dependency on waiting for the specified duration.

IfBlock

Defines a condition and the execution unit that should be executed if the condition is satisfied.

Field	Type	Label	Description
condition	BooleanExpression
then_node	Node

IfElseBlock

Defines a series of if/else blocks. The first branch whose condition evaluates to true is the one to execute. If no conditions were satisfied, the else_node or the error will execute.

Field	Type	Label	Description
case	IfBlock		+required. First condition to evaluate.
other	IfBlock	repeated	+optional. Additional branches to evaluate.
else_node	Node		The node to execute in case none of the branches were taken.
error	Error		An error to throw in case none of the branches were taken.

LaunchPlanTemplate

A structure that uniquely identifies a launch plan in the system.

Field	Type	Label	Description
id	Identifier		A globally unique identifier for the launch plan.
interface	TypedInterface		The input and output interface for the launch plan
fixed_inputs	LiteralMap		A collection of input literals that are fixed for the launch plan

Node

A Workflow graph Node. One unit of execution in the graph. Each node can be linked to a Task, a Workflow or a branch node.

Field	Type	Label	Description
id	string		A workflow-level unique identifier that identifies this node in the workflow. ‘inputs’ and ‘outputs’ are reserved node ids that cannot be used by other nodes.
metadata	NodeMetadata		Extra metadata about the node.
inputs	Binding	repeated	Specifies how to bind the underlying interface’s inputs. All required inputs specified in the underlying interface must be fulfilled.
upstream_node_ids	string	repeated	+optional Specifies execution dependency for this node ensuring it will only get scheduled to run after all its upstream nodes have completed. This node will have an implicit dependency on any node that appears in inputs field.
output_aliases	Alias	repeated	+optional. A node can define aliases for a subset of its outputs. This is particularly useful if different nodes need to conform to the same interface (e.g. all branches in a branch node). Downstream nodes must refer to this nodes outputs using the alias if one’s specified.
task_node	TaskNode		Information about the Task to execute in this node.
workflow_node	WorkflowNode		Information about the Workflow to execute in this mode.
branch_node	BranchNode		Information about the branch node to evaluate in this node.
gate_node	GateNode		Information about the condition to evaluate in this node.
array_node	ArrayNode		Information about the sub-node executions for each value in the list of this nodes inputs values.

NodeMetadata

Defines extra information about the Node.

Field	Type	Label	Description
name	string		A friendly name for the Node
timeout	google.protobuf.Duration		The overall timeout of a task.
retries	RetryStrategy		Number of retries per task.
interruptible	bool
cacheable	bool
cache_version	string
cache_serializable	bool
config	NodeMetadata.ConfigEntry	repeated	Config is a bag of properties that can be used to instruct propeller on how to execute the node.

NodeMetadata.ConfigEntry

Field	Type	Label	Description
key	string
value	string

SignalCondition

SignalCondition represents a dependency on an signal.

Field	Type	Label	Description
signal_id	string		A unique identifier for the requested signal.
type	LiteralType		A type denoting the required value type for this signal.
output_variable_name	string		The variable name for the signal value in this nodes outputs.

SleepCondition

SleepCondition represents a dependency on waiting for the specified duration.

Field	Type	Label	Description
duration	google.protobuf.Duration		The overall duration for this sleep.

TaskNode

Refers to the task that the Node is to execute.

Field	Type	Label	Description
reference_id	Identifier		A globally unique identifier for the task.
overrides	TaskNodeOverrides		Optional overrides applied at task execution time.

TaskNodeOverrides

Optional task node overrides that will be applied at task execution time.

Field	Type	Label	Description
resources	Resources		A customizable interface to convey resources requested for a task container.
extended_resources	ExtendedResources		Overrides for all non-standard resources, not captured by v1.ResourceRequirements, to allocate to a task.
container_image	string		Override for the image used by task pods.
pod_template	K8sPod		Override for the pod template used by task pods +optional

WorkflowMetadata

This is workflow layer metadata. These settings are only applicable to the workflow as a whole, and do not percolate down to child entities (like tasks) launched by the workflow.

Field	Type	Label	Description
quality_of_service	QualityOfService		Indicates the runtime priority of workflow executions.
on_failure	WorkflowMetadata.OnFailurePolicy		Defines how the system should behave when a failure is detected in the workflow execution.
tags	WorkflowMetadata.TagsEntry	repeated	Arbitrary tags that allow users and the platform to store small but arbitrary labels

WorkflowMetadata.TagsEntry

Field	Type	Label	Description
key	string
value	string

WorkflowMetadataDefaults

The difference between these settings and the WorkflowMetadata ones is that these are meant to be passed down to a workflow’s underlying entities (like tasks). For instance, ‘interruptible’ has no meaning at the workflow layer, it is only relevant when a task executes. The settings here are the defaults that are passed to all nodes unless explicitly overridden at the node layer. If you are adding a setting that applies to both the Workflow itself, and everything underneath it, it should be added to both this object and the WorkflowMetadata object above.

Field	Type	Label	Description
interruptible	bool		Whether child nodes of the workflow are interruptible.

WorkflowNode

Refers to a the workflow the node is to execute.

Field	Type	Label	Description
launchplan_ref	Identifier		A globally unique identifier for the launch plan.
sub_workflow_ref	Identifier		Reference to a subworkflow, that should be defined with the compiler context

WorkflowTemplate

Flyte Workflow Structure that encapsulates task, branch and subworkflow nodes to form a statically analyzable, directed acyclic graph.

Field	Type	Label	Description
id	Identifier		A globally unique identifier for the workflow.
metadata	WorkflowMetadata		Extra metadata about the workflow.
interface	TypedInterface		Defines a strongly typed interface for the Workflow. This can include some optional parameters.
nodes	Node	repeated	A list of nodes. In addition, ‘globals’ is a special reserved node id that can be used to consume workflow inputs.
outputs	Binding	repeated	A list of output bindings that specify how to construct workflow outputs. Bindings can pull node outputs or specify literals. All workflow outputs specified in the interface field must be bound in order for the workflow to be validated. A workflow has an implicit dependency on all of its nodes to execute successfully in order to bind final outputs. Most of these outputs will be Binding’s with a BindingData of type OutputReference. That is, your workflow can just have an output of some constant (Output(5)), but usually, the workflow will be pulling outputs from the output of a task.
failure_node	Node		+optional A catch-all node. This node is executed whenever the execution engine determines the workflow has failed. The interface of this node must match the Workflow interface with an additional input named ’error’ of type pb.lyft.flyte.core.Error.
metadata_defaults	WorkflowMetadataDefaults		workflow defaults

ArrayNode.DataMode

Name	Number	Description
SINGLE_INPUT_FILE	0	Indicates the ArrayNode’s input is a list of input values that map to subNode executions. The file path set for the subNode will be the ArrayNode’s input file, but the in-memory value utilized in propeller will be the individual value for each subNode execution. SubNode executions need to be able to read in and parse the individual value to execute correctly.
INDIVIDUAL_INPUT_FILES	1	Indicates the ArrayNode’s input is a list of input values that map to subNode executions. Propeller will create input files for each ArrayNode subNode by parsing the inputs and setting the InputBindings on each subNodeSpec. Both the file path and in-memory input values will be the individual value for each subNode execution.

ArrayNode.ExecutionMode

Name	Number	Description
MINIMAL_STATE	0	Indicates the ArrayNode will store minimal state for the sub-nodes. This is more efficient, but only supports a subset of Flyte entities.
FULL_STATE	1	Indicates the ArrayNode will store full state for the sub-nodes. This supports a wider range of Flyte entities.

WorkflowMetadata.OnFailurePolicy

Failure Handling Strategy

Name	Number	Description
FAIL_IMMEDIATELY	0	FAIL_IMMEDIATELY instructs the system to fail as soon as a node fails in the workflow. It’ll automatically abort all currently running nodes and clean up resources before finally marking the workflow executions as failed.
FAIL_AFTER_EXECUTABLE_NODES_COMPLETE	1	FAIL_AFTER_EXECUTABLE_NODES_COMPLETE instructs the system to make as much progress as it can. The system will not alter the dependencies of the execution graph so any node that depend on the failed node will not be run. Other nodes that will be executed to completion before cleaning up resources and marking the workflow execution as failed.

flyteidl/core/workflow_closure.proto

WorkflowClosure

Defines an enclosed package of workflow and tasks it references.

Field	Type	Label	Description
workflow	WorkflowTemplate		required. Workflow template.
tasks	TaskTemplate	repeated	optional. A collection of tasks referenced by the workflow. Only needed if the workflow references tasks.

flyteidl/core/condition.proto

BooleanExpression

Defines a boolean expression tree. It can be a simple or a conjunction expression. Multiple expressions can be combined using a conjunction or a disjunction to result in a final boolean result.

Field	Type	Label	Description
conjunction	ConjunctionExpression
comparison	ComparisonExpression

ComparisonExpression

Defines a 2-level tree where the root is a comparison operator and Operands are primitives or known variables. Each expression results in a boolean result.

Field	Type	Label	Description
operator	ComparisonExpression.Operator
left_value	Operand
right_value	Operand

ConjunctionExpression

Defines a conjunction expression of two boolean expressions.

Field	Type	Label	Description
operator	ConjunctionExpression.LogicalOperator
left_expression	BooleanExpression
right_expression	BooleanExpression

Operand

Defines an operand to a comparison expression.

Field	Type	Label	Description
primitive	Primitive		Deprecated. Can be a constant
var	string		Or one of this node’s input variables
scalar	Scalar		Replace the primitive field

ComparisonExpression.Operator

Binary Operator for each expression

Name	Number	Description
EQ	0
NEQ	1
GT	2	Greater Than
GTE	3
LT	4	Less Than
LTE	5

ConjunctionExpression.LogicalOperator

Nested conditions. They can be conjoined using AND / OR Order of evaluation is not important as the operators are Commutative

Name	Number	Description
AND	0	Conjunction
OR	1

flyteidl/core/dynamic_job.proto

DynamicJobSpec

Describes a set of tasks to execute and how the final outputs are produced.

Field	Type	Label	Description
nodes	Node	repeated	A collection of nodes to execute.
min_successes	int64		An absolute number of successful completions of nodes required to mark this job as succeeded. As soon as this criteria is met, the dynamic job will be marked as successful and outputs will be computed. If this number becomes impossible to reach (e.g. number of currently running tasks + number of already succeeded tasks <lt; min_successes) the task will be aborted immediately and marked as failed. The default value of this field, if not specified, is the count of nodes repeated field.
outputs	Binding	repeated	Describes how to bind the final output of the dynamic job from the outputs of executed nodes. The referenced ids in bindings should have the generated id for the subtask.
tasks	TaskTemplate	repeated	[Optional] A complete list of task specs referenced in nodes.
subworkflows	WorkflowTemplate	repeated	[Optional] A complete list of task specs referenced in nodes.

flyteidl/plugins/presto.proto

PrestoQuery

This message works with the ‘presto’ task type in the SDK and is the object that will be in the ‘custom’ field of a Presto task’s TaskTemplate

Field	Type	Label	Description
routing_group	string
catalog	string
schema	string
statement	string

flyteidl/plugins/qubole.proto

HiveQuery

Defines a query to execute on a hive cluster.

Field	Type	Label	Description
query	string
timeout_sec	uint32
retryCount	uint32

HiveQueryCollection

Defines a collection of hive queries.

Field	Type	Label	Description
queries	HiveQuery	repeated

QuboleHiveJob

This message works with the ‘hive’ task type in the SDK and is the object that will be in the ‘custom’ field of a hive task’s TaskTemplate

Field	Type	Label	Description
cluster_label	string
query_collection	HiveQueryCollection		Deprecated.
tags	string	repeated
query	HiveQuery

flyteidl/plugins/ray.proto

HeadGroupSpec

HeadGroupSpec are the spec for the head pod

Field	Type	Label	Description
ray_start_params	HeadGroupSpec.RayStartParamsEntry	repeated	Optional. RayStartParams are the params of the start command: address, object-store-memory. Refer to https://docs.ray.io/en/latest/ray-core/package-ref.html#ray-start
k8s_pod	flyteidl.core.K8sPod		Pod Spec for the ray head pod

HeadGroupSpec.RayStartParamsEntry

Field	Type	Label	Description
key	string
value	string

RayCluster

Define Ray cluster defines the desired state of RayCluster

Field	Type	Label	Description
head_group_spec	HeadGroupSpec		HeadGroupSpecs are the spec for the head pod
worker_group_spec	WorkerGroupSpec	repeated	WorkerGroupSpecs are the specs for the worker pods
enable_autoscaling	bool		Whether to enable autoscaling.

RayJob

RayJobSpec defines the desired state of RayJob

Field	Type	Label	Description
ray_cluster	RayCluster		RayClusterSpec is the cluster template to run the job
runtime_env	string		Deprecated. runtime_env is base64 encoded. Ray runtime environments: https://docs.ray.io/en/latest/ray-core/handling-dependencies.html#runtime-environments
shutdown_after_job_finishes	bool		shutdown_after_job_finishes specifies whether the RayCluster should be deleted after the RayJob finishes.
ttl_seconds_after_finished	int32		ttl_seconds_after_finished specifies the number of seconds after which the RayCluster will be deleted after the RayJob finishes.
runtime_env_yaml	string		RuntimeEnvYAML represents the runtime environment configuration provided as a multi-line YAML string.

WorkerGroupSpec

WorkerGroupSpec are the specs for the worker pods

Field	Type	Label	Description
group_name	string		Required. RayCluster can have multiple worker groups, and it distinguishes them by name
replicas	int32		Required. Desired replicas of the worker group. Defaults to 1.
min_replicas	int32		Optional. Min replicas of the worker group. MinReplicas defaults to 1.
max_replicas	int32		Optional. Max replicas of the worker group. MaxReplicas defaults to maxInt32
ray_start_params	WorkerGroupSpec.RayStartParamsEntry	repeated	Optional. RayStartParams are the params of the start command: address, object-store-memory. Refer to https://docs.ray.io/en/latest/ray-core/package-ref.html#ray-start
k8s_pod	flyteidl.core.K8sPod		Pod Spec for ray worker pods

WorkerGroupSpec.RayStartParamsEntry

Field	Type	Label	Description
key	string
value	string

flyteidl/plugins/spark.proto

SparkApplication

SparkJob

Custom Proto for Spark Plugin.

Field	Type	Label	Description
applicationType	SparkApplication.Type
mainApplicationFile	string
mainClass	string
sparkConf	SparkJob.SparkConfEntry	repeated
hadoopConf	SparkJob.HadoopConfEntry	repeated
executorPath	string		Executor path for Python jobs.
databricksConf	google.protobuf.Struct		Databricks job configuration. Config structure can be found here. https://docs.databricks.com/dev-tools/api/2.0/jobs.html#request-structure.
databricksToken	string		Databricks access token. https://docs.databricks.com/dev-tools/api/latest/authentication.html This token can be set in either flytepropeller or flytekit.
databricksInstance	string		Domain name of your deployment. Use the form <lt;account>gt;.cloud.databricks.com. This instance name can be set in either flytepropeller or flytekit.
driverPod	flyteidl.core.K8sPod		Pod Spec for the Spark driver pod
executorPod	flyteidl.core.K8sPod		Pod Spec for the Spark executor pod

SparkJob.HadoopConfEntry

Field	Type	Label	Description
key	string
value	string

SparkJob.SparkConfEntry

Field	Type	Label	Description
key	string
value	string

SparkApplication.Type

Name	Number	Description
PYTHON	0
JAVA	1
SCALA	2
R	3

flyteidl/plugins/array_job.proto

ArrayJob

Describes a job that can process independent pieces of data concurrently. Multiple copies of the runnable component will be executed concurrently.

Field	Type	Label	Description
parallelism	int64		Defines the maximum number of instances to bring up concurrently at any given point. Note that this is an optimistic restriction and that, due to network partitioning or other failures, the actual number of currently running instances might be more. This has to be a positive number if assigned. Default value is size.
size	int64		Defines the number of instances to launch at most. This number should match the size of the input if the job requires processing of all input data. This has to be a positive number. In the case this is not defined, the back-end will determine the size at run-time by reading the inputs.
min_successes	int64		An absolute number of the minimum number of successful completions of subtasks. As soon as this criteria is met, the array job will be marked as successful and outputs will be computed. This has to be a non-negative number if assigned. Default value is size (if specified).
min_success_ratio	float		If the array job size is not known beforehand, the min_success_ratio can instead be used to determine when an array job can be marked successful.

flyteidl/plugins/waitable.proto

Waitable

Represents an Execution that was launched and could be waited on.

Field	Type	Label	Description
wf_exec_id	flyteidl.core.WorkflowExecutionIdentifier
phase	flyteidl.core.WorkflowExecution.Phase
workflow_id	string

flyteidl/plugins/dask.proto

DaskJob

Custom Proto for Dask Plugin.

Field	Type	Label	Description
scheduler	DaskScheduler		Spec for the scheduler pod.
workers	DaskWorkerGroup		Spec of the default worker group.

DaskScheduler

Specification for the scheduler pod.

Field	Type	Label	Description
image	string		Optional image to use. If unset, will use the default image.
resources	flyteidl.core.Resources		Resources assigned to the scheduler pod.

DaskWorkerGroup

Field	Type	Label	Description
number_of_workers	uint32		Number of workers in the group.
image	string		Optional image to use for the pods of the worker group. If unset, will use the default image.
resources	flyteidl.core.Resources		Resources assigned to the all pods of the worker group. As per https://kubernetes.dask.org/en/latest/kubecluster.html?highlight=limit#best-practices it is advised to only set limits. If requests are not explicitly set, the plugin will make sure to set requests==limits. The plugin sets --memory-limit as well as --nthreads for the workers according to the limit.

flyteidl/plugins/mpi.proto

DistributedMPITrainingTask

MPI operator proposal https://github.com/kubeflow/community/blob/master/proposals/mpi-operator-proposal.md Custom proto for plugin that enables distributed training using https://github.com/kubeflow/mpi-operator

Field	Type	Label	Description
num_workers	int32		number of worker spawned in the cluster for this job
num_launcher_replicas	int32		number of launcher replicas spawned in the cluster for this job The launcher pod invokes mpirun and communicates with worker pods through MPI.
slots	int32		number of slots per worker used in hostfile. The available slots (GPUs) in each pod.

flyteidl/plugins/pytorch.proto

DistributedPyTorchTrainingTask

Custom proto for plugin that enables distributed training using https://github.com/kubeflow/pytorch-operator

Field	Type	Label	Description
workers	int32		number of worker replicas spawned in the cluster for this job
elastic_config	ElasticConfig		config for an elastic pytorch job

ElasticConfig

Custom proto for torch elastic config for distributed training using https://github.com/kubeflow/training-operator/blob/master/pkg/apis/kubeflow.org/v1/pytorch_types.go

Field	Type	Label	Description
rdzv_backend	string
min_replicas	int32
max_replicas	int32
nproc_per_node	int32
max_restarts	int32

flyteidl/plugins/kubeflow/mpi.proto

DistributedMPITrainingReplicaSpec

Replica specification for distributed MPI training

Field	Type	Label	Description
replicas	int32		Deprecated. 1~4 deprecated. Use common instead. Number of replicas
image	string		Deprecated. Image used for the replica group
resources	flyteidl.core.Resources		Deprecated. Resources required for the replica group
restart_policy	flyteidl.plugins.RestartPolicy		Deprecated. Restart policy determines whether pods will be restarted when they exit
command	string	repeated	MPI sometimes requires different command set for different replica groups
common	flyteidl.plugins.CommonReplicaSpec		The common replica spec

DistributedMPITrainingTask

Proto for plugin that enables distributed training using https://github.com/kubeflow/mpi-operator

Field	Type	Label	Description
worker_replicas	DistributedMPITrainingReplicaSpec		Worker replicas spec
launcher_replicas	DistributedMPITrainingReplicaSpec		Master replicas spec
run_policy	RunPolicy		RunPolicy encapsulates various runtime policies of the distributed training job, for example how to clean up resources and how long the job can stay active.
slots	int32		Number of slots per worker

flyteidl/plugins/kubeflow/pytorch.proto

DistributedPyTorchTrainingReplicaSpec

Field	Type	Label	Description
replicas	int32		Deprecated. 1~4 deprecated. Use common instead. Number of replicas
image	string		Deprecated. Image used for the replica group
resources	flyteidl.core.Resources		Deprecated. Resources required for the replica group
restart_policy	flyteidl.plugins.RestartPolicy		Deprecated. Restart policy determines whether pods will be restarted when they exit
common	flyteidl.plugins.CommonReplicaSpec		The common replica spec

DistributedPyTorchTrainingTask

Proto for plugin that enables distributed training using https://github.com/kubeflow/pytorch-operator

Field	Type	Label	Description
worker_replicas	DistributedPyTorchTrainingReplicaSpec		Worker replicas spec
master_replicas	DistributedPyTorchTrainingReplicaSpec		Master replicas spec, master replicas can only have 1 replica
run_policy	RunPolicy		RunPolicy encapsulates various runtime policies of the distributed training job, for example how to clean up resources and how long the job can stay active.
elastic_config	ElasticConfig		config for an elastic pytorch job

ElasticConfig

Custom proto for torch elastic config for distributed training using https://github.com/kubeflow/training-operator/blob/master/pkg/apis/kubeflow.org/v1/pytorch_types.go

Field	Type	Label	Description
rdzv_backend	string
min_replicas	int32
max_replicas	int32
nproc_per_node	int32
max_restarts	int32

flyteidl/plugins/kubeflow/tensorflow.proto

DistributedTensorflowTrainingReplicaSpec

Field	Type	Label	Description
replicas	int32		Deprecated. 1~4 deprecated. Use common instead. Number of replicas
image	string		Deprecated. Image used for the replica group
resources	flyteidl.core.Resources		Deprecated. Resources required for the replica group
restart_policy	flyteidl.plugins.RestartPolicy		Deprecated. Restart policy determines whether pods will be restarted when they exit
common	flyteidl.plugins.CommonReplicaSpec		The common replica spec

DistributedTensorflowTrainingTask

Proto for plugin that enables distributed training using https://github.com/kubeflow/tf-operator

Field	Type	Label	Description
worker_replicas	DistributedTensorflowTrainingReplicaSpec		Worker replicas spec
ps_replicas	DistributedTensorflowTrainingReplicaSpec		Parameter server replicas spec
chief_replicas	DistributedTensorflowTrainingReplicaSpec		Chief replicas spec
run_policy	RunPolicy		RunPolicy encapsulates various runtime policies of the distributed training job, for example how to clean up resources and how long the job can stay active.
evaluator_replicas	DistributedTensorflowTrainingReplicaSpec		Evaluator replicas spec

flyteidl/plugins/kubeflow/common.proto

RunPolicy

Field	Type	Label	Description
clean_pod_policy	CleanPodPolicy		Defines the policy to kill pods after the job completes. Default to None.
ttl_seconds_after_finished	int32		TTL to clean up jobs. Default to infinite.
active_deadline_seconds	int32		Specifies the duration in seconds relative to the startTime that the job may be active before the system tries to terminate it; value must be positive integer.
backoff_limit	int32		Number of retries before marking this job failed.

CleanPodPolicy

Name	Number	Description
CLEANPOD_POLICY_NONE	0
CLEANPOD_POLICY_RUNNING	1
CLEANPOD_POLICY_ALL	2

flyteidl/plugins/tensorflow.proto

DistributedTensorflowTrainingTask

Custom proto for plugin that enables distributed training using https://github.com/kubeflow/tf-operator

Field	Type	Label	Description
workers	int32		number of worker replicas spawned in the cluster for this job
ps_replicas	int32		PS ->gt; Parameter server number of ps replicas spawned in the cluster for this job
chief_replicas	int32		number of chief replicas spawned in the cluster for this job
evaluator_replicas	int32		number of evaluator replicas spawned in the cluster for this job

flyteidl/plugins/common.proto

CommonReplicaSpec

Field	Type	Label	Description
replicas	int32		Number of replicas
image	string		Image used for the replica group
resources	flyteidl.core.Resources		Resources required for the replica group
restart_policy	RestartPolicy		RestartPolicy determines whether pods will be restarted when they exit

RestartPolicy

Name	Number	Description
RESTART_POLICY_NEVER	0
RESTART_POLICY_ON_FAILURE	1
RESTART_POLICY_ALWAYS	2

flyteidl/admin/schedule.proto

CronSchedule

Options for schedules to run according to a cron expression.

Field	Type	Label	Description
schedule	string		Standard/default cron implementation as described by https://en.wikipedia.org/wiki/Cron#CRON_expression; Also supports nonstandard predefined scheduling definitions as described by https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html#CronExpressions except @reboot
offset	string		ISO 8601 duration as described by https://en.wikipedia.org/wiki/ISO_8601#Durations

FixedRate

Option for schedules run at a certain frequency e.g. every 2 minutes.

Field	Type	Label	Description
value	uint32
unit	FixedRateUnit

Schedule

Defines complete set of information required to trigger an execution on a schedule.

Field	Type	Label	Description
cron_expression	string		Deprecated. Uses AWS syntax: Minutes Hours Day-of-month Month Day-of-week Year e.g. for a schedule that runs every 15 minutes: 0/15 * * * ? *
rate	FixedRate
cron_schedule	CronSchedule
kickoff_time_input_arg	string		Name of the input variable that the kickoff time will be supplied to when the workflow is kicked off.

FixedRateUnit

Represents a frequency at which to run a schedule.

Name	Number	Description
MINUTE	0
HOUR	1
DAY	2

flyteidl/admin/project.proto

Domain

Namespace within a project commonly used to differentiate between different service instances. e.g. “production”, “development”, etc.

Field	Type	Label	Description
id	string		Globally unique domain name.
name	string		Display name.

GetDomainRequest

Empty request for GetDomain

GetDomainsResponse

Represents a list of domains.

Field	Type	Label	Description
domains	Domain	repeated

InactiveProject

Error returned for inactive projects

Field	Type	Label	Description
id	string		Indicates a unique project. +required
org	string		Optional, org key applied to the resource.

Project

Top-level namespace used to classify different entities like workflows and executions.

Field	Type	Label	Description
id	string		Globally unique project name.
name	string		Display name.
domains	Domain	repeated
description	string
labels	Labels		Leverage Labels from flyteidl.admin.common.proto to tag projects with ownership information.
state	Project.ProjectState
org	string		Optional, org key applied to the resource.

ProjectGetRequest

Field	Type	Label	Description
id	string		Indicates a unique project. +required
org	string		Optional, org key applied to the resource.

ProjectListRequest

Request to retrieve a list of projects matching specified filters. See :ref:ref_flyteidl.admin.Project for more details

Field	Type	Label	Description
limit	uint32		Indicates the number of projects to be returned. +required
token	string		In the case of multiple pages of results, this server-provided token can be used to fetch the next page in a query. +optional
filters	string		Indicates a list of filters passed as string. More info on constructing filters : <lt;Link>gt; +optional
sort_by	Sort		Sort ordering. +optional
org	string		Optional, org filter applied to list project requests.

ProjectRegisterRequest

Adds a new user-project within the Flyte deployment. See :ref:ref_flyteidl.admin.Project for more details

Field	Type	Label	Description
project	Project		+required

ProjectRegisterResponse

Purposefully empty, may be updated in the future.

ProjectUpdateResponse

Purposefully empty, may be updated in the future.

Projects

Represents a list of projects. See :ref:ref_flyteidl.admin.Project for more details

Field	Type	Label	Description
projects	Project	repeated
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

Project.ProjectState

The state of the project is used to control its visibility in the UI and validity.

Name	Number	Description
ACTIVE	0	By default, all projects are considered active.
ARCHIVED	1	Archived projects are no longer visible in the UI and no longer valid.
SYSTEM_GENERATED	2	System generated projects that aren’t explicitly created or managed by a user.
SYSTEM_ARCHIVED	3	System archived projects that aren’t explicitly archived by a user.

flyteidl/admin/cluster_assignment.proto

ClusterAssignment

Encapsulates specifications for routing an execution onto a specific cluster.

Field	Type	Label	Description
cluster_pool_name	string

flyteidl/admin/notification.proto

EmailMessage

Represents the Email object that is sent to a publisher/subscriber to forward the notification. Note: This is internal to Admin and doesn’t need to be exposed to other components.

Field	Type	Label	Description
recipients_email	string	repeated	The list of email addresses to receive an email with the content populated in the other fields. Currently, each email recipient will receive its own email. This populates the TO field.
sender_email	string		The email of the sender. This populates the FROM field.
subject_line	string		The content of the subject line. This populates the SUBJECT field.
body	string		The content of the email body. This populates the BODY field.

flyteidl/admin/task.proto

Task

Flyte workflows are composed of many ordered tasks. That is small, reusable, self-contained logical blocks arranged to process workflow inputs and produce a deterministic set of outputs. Tasks can come in many varieties tuned for specialized behavior.

Field	Type	Label	Description
id	flyteidl.core.Identifier		id represents the unique identifier of the task.
closure	TaskClosure		closure encapsulates all the fields that maps to a compiled version of the task.
short_description	string		One-liner overview of the entity.

TaskClosure

Compute task attributes which include values derived from the TaskSpec, as well as plugin-specific data and task metadata.

Field	Type	Label	Description
compiled_task	flyteidl.core.CompiledTask		Represents the compiled representation of the task from the specification provided.
created_at	google.protobuf.Timestamp		Time at which the task was created.

TaskCreateRequest

Represents a request structure to create a revision of a task. See :ref:ref_flyteidl.admin.Task for more details

Field	Type	Label	Description
id	flyteidl.core.Identifier		id represents the unique identifier of the task. +required
spec	TaskSpec		Represents the specification for task. +required

TaskCreateResponse

Represents a response structure if task creation succeeds.

Purposefully empty, may be populated in the future.

TaskList

Represents a list of tasks returned from the admin. See :ref:ref_flyteidl.admin.Task for more details

Field	Type	Label	Description
tasks	Task	repeated	A list of tasks returned based on the request.
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

TaskSpec

Represents a structure that encapsulates the user-configured specification of the task.

Field	Type	Label	Description
template	flyteidl.core.TaskTemplate		Template of the task that encapsulates all the metadata of the task.
description	DescriptionEntity		Represents the specification for description entity.

flyteidl/admin/launch_plan.proto

ActiveLaunchPlanListRequest

Represents a request structure to list active launch plans within a project/domain and optional org. See :ref:ref_flyteidl.admin.LaunchPlan for more details

Field	Type	Label	Description
project	string		Name of the project that contains the identifiers. +required.
domain	string		Name of the domain the identifiers belongs to within the project. +required.
limit	uint32		Indicates the number of resources to be returned. +required.
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional
sort_by	Sort		Sort ordering. +optional
org	string		Optional, org key applied to the resource.

ActiveLaunchPlanRequest

Represents a request struct for finding an active launch plan for a given NamedEntityIdentifier See :ref:ref_flyteidl.admin.LaunchPlan for more details

Field	Type	Label	Description
id	NamedEntityIdentifier		+required.

Auth

Defines permissions associated with executions created by this launch plan spec. Use either of these roles when they have permissions required by your workflow execution. Deprecated.

Field	Type	Label	Description
assumable_iam_role	string		Defines an optional iam role which will be used for tasks run in executions created with this launch plan.
kubernetes_service_account	string		Defines an optional kubernetes service account which will be used for tasks run in executions created with this launch plan.

LaunchPlan

A LaunchPlan provides the capability to templatize workflow executions. Launch plans simplify associating one or more schedules, inputs and notifications with your workflows. Launch plans can be shared and used to trigger executions with predefined inputs even when a workflow definition doesn’t necessarily have a default value for said input.

Field	Type	Label	Description
id	flyteidl.core.Identifier		Uniquely identifies a launch plan entity.
spec	LaunchPlanSpec		User-provided launch plan details, including reference workflow, inputs and other metadata.
closure	LaunchPlanClosure		Values computed by the flyte platform after launch plan registration.

LaunchPlanClosure

Values computed by the flyte platform after launch plan registration. These include expected_inputs required to be present in a CreateExecutionRequest to launch the reference workflow as well timestamp values associated with the launch plan.

Field	Type	Label	Description
state	LaunchPlanState		Indicate the Launch plan state.
expected_inputs	flyteidl.core.ParameterMap		Indicates the set of inputs expected when creating an execution with the Launch plan
expected_outputs	flyteidl.core.VariableMap		Indicates the set of outputs expected to be produced by creating an execution with the Launch plan
created_at	google.protobuf.Timestamp		Time at which the launch plan was created.
updated_at	google.protobuf.Timestamp		Time at which the launch plan was last updated.

LaunchPlanCreateRequest

Request to register a launch plan. The included LaunchPlanSpec may have a complete or incomplete set of inputs required to launch a workflow execution. By default all launch plans are registered in state INACTIVE. If you wish to set the state to ACTIVE, you must submit a LaunchPlanUpdateRequest, after you have successfully created a launch plan.

Field	Type	Label	Description
id	flyteidl.core.Identifier		Uniquely identifies a launch plan entity.
spec	LaunchPlanSpec		User-provided launch plan details, including reference workflow, inputs and other metadata.

LaunchPlanCreateResponse

Purposefully empty, may be populated in the future.

LaunchPlanList

Response object for list launch plan requests. See :ref:ref_flyteidl.admin.LaunchPlan for more details

Field	Type	Label	Description
launch_plans	LaunchPlan	repeated
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

LaunchPlanMetadata

Additional launch plan attributes included in the LaunchPlanSpec not strictly required to launch the reference workflow.

Field	Type	Label	Description
schedule	Schedule		Schedule to execute the Launch Plan
notifications	Notification	repeated	List of notifications based on Execution status transitions
launch_conditions	google.protobuf.Any		Additional metadata for how to launch the launch plan

LaunchPlanSpec

User-provided launch plan definition and configuration values.

Field	Type	Label	Description
workflow_id	flyteidl.core.Identifier		Reference to the Workflow template that the launch plan references
entity_metadata	LaunchPlanMetadata		Metadata for the Launch Plan
default_inputs	flyteidl.core.ParameterMap		Input values to be passed for the execution. These can be overridden when an execution is created with this launch plan.
fixed_inputs	flyteidl.core.LiteralMap		Fixed, non-overridable inputs for the Launch Plan. These can not be overridden when an execution is created with this launch plan.
role	string		Deprecated. String to indicate the role to use to execute the workflow underneath
labels	Labels		Custom labels to be applied to the execution resource.
annotations	Annotations		Custom annotations to be applied to the execution resource.
auth	Auth		Deprecated. Indicates the permission associated with workflow executions triggered with this launch plan.
auth_role	AuthRole		Deprecated.
security_context	flyteidl.core.SecurityContext		Indicates security context for permissions triggered with this launch plan
quality_of_service	flyteidl.core.QualityOfService		Indicates the runtime priority of the execution.
raw_output_data_config	RawOutputDataConfig		Encapsulates user settings pertaining to offloaded data (i.e. Blobs, Schema, query data, etc.).
max_parallelism	int32		Controls the maximum number of tasknodes that can be run in parallel for the entire workflow. This is useful to achieve fairness. Note: MapTasks are regarded as one unit, and parallelism/concurrency of MapTasks is independent from this.
interruptible	google.protobuf.BoolValue		Allows for the interruptible flag of a workflow to be overwritten for a single execution. Omitting this field uses the workflow’s value as a default. As we need to distinguish between the field not being provided and its default value false, we have to use a wrapper around the bool field.
overwrite_cache	bool		Allows for all cached values of a workflow and its tasks to be overwritten for a single execution. If enabled, all calculations are performed even if cached results would be available, overwriting the stored data once execution finishes successfully.
envs	Envs		Environment variables to be set for the execution.
execution_env_assignments	flyteidl.core.ExecutionEnvAssignment	repeated	Execution environment assignments to be set for the execution.
cluster_assignment	ClusterAssignment		ClusterAssignment controls how to select an available cluster on which executions of this LaunchPlan should run. This can be overwritten at execution creation level.

LaunchPlanUpdateRequest

Request to set the referenced launch plan state to the configured value. See :ref:ref_flyteidl.admin.LaunchPlan for more details

Field	Type	Label	Description
id	flyteidl.core.Identifier		Identifier of launch plan for which to change state. +required.
state	LaunchPlanState		Desired state to apply to the launch plan. +required.

LaunchPlanUpdateResponse

Purposefully empty, may be populated in the future.

LaunchPlanState

By default any launch plan regardless of state can be used to launch a workflow execution. However, at most one version of a launch plan (e.g. a NamedEntityIdentifier set of shared project, domain and name values) can be active at a time in regards to schedules. That is, at most one schedule in a NamedEntityIdentifier group will be observed and trigger executions at a defined cadence.

Name	Number	Description
INACTIVE	0
ACTIVE	1

flyteidl/admin/signal.proto

Signal

Signal encapsulates a unique identifier, associated metadata, and a value for a single Flyte signal. Signals may exist either without a set value (representing a signal request) or with a populated value (indicating the signal has been given).

Field	Type	Label	Description
id	flyteidl.core.SignalIdentifier		A unique identifier for the requested signal.
type	flyteidl.core.LiteralType		A type denoting the required value type for this signal.
value	flyteidl.core.Literal		The value of the signal. This is only available if the signal has been “set” and must match the defined the type.

SignalGetOrCreateRequest

SignalGetOrCreateRequest represents a request structure to retrieve or create a signal. See :ref:ref_flyteidl.admin.Signal for more details

Field	Type	Label	Description
id	flyteidl.core.SignalIdentifier		A unique identifier for the requested signal.
type	flyteidl.core.LiteralType		A type denoting the required value type for this signal.

SignalList

SignalList represents collection of signals along with the token of the last result. See :ref:ref_flyteidl.admin.Signal for more details

Field	Type	Label	Description
signals	Signal	repeated	A list of signals matching the input filters.
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

SignalListRequest

SignalListRequest represents a request structure to retrieve a collection of signals. See :ref:ref_flyteidl.admin.Signal for more details

Field	Type	Label	Description
workflow_execution_id	flyteidl.core.WorkflowExecutionIdentifier		Indicates the workflow execution to filter by. +required
limit	uint32		Indicates the number of resources to be returned. +required
token	string		In the case of multiple pages of results, the, server-provided token can be used to fetch the next page in a query. +optional
filters	string		Indicates a list of filters passed as string. +optional
sort_by	Sort		Sort ordering. +optional

SignalSetRequest

SignalSetRequest represents a request structure to set the value on a signal. Setting a signal effetively satisfies the signal condition within a Flyte workflow. See :ref:ref_flyteidl.admin.Signal for more details

Field	Type	Label	Description
id	flyteidl.core.SignalIdentifier		A unique identifier for the requested signal.
value	flyteidl.core.Literal		The value of this signal, must match the defining signal type.

SignalSetResponse

SignalSetResponse represents a response structure if signal setting succeeds.

Purposefully empty, may be populated in the future.

flyteidl/admin/task_execution.proto

Reason

Reason is a single message annotated with a timestamp to indicate the instant the reason occurred.

Field	Type	Label	Description
occurred_at	google.protobuf.Timestamp		occurred_at is the timestamp indicating the instant that this reason happened.
message	string		message is the explanation for the most recent phase transition or status update.

TaskExecution

Encapsulates all details for a single task execution entity. A task execution represents an instantiated task, including all inputs and additional metadata as well as computed results included state, outputs, and duration-based attributes.

Field	Type	Label	Description
id	flyteidl.core.TaskExecutionIdentifier		Unique identifier for the task execution.
input_uri	string		Path to remote data store where input blob is stored.
closure	TaskExecutionClosure		Task execution details and results.
is_parent	bool		Whether this task spawned nodes.

TaskExecutionClosure

Container for task execution details and results.

Field	Type	Label	Description
output_uri	string		Deprecated. Path to remote data store where output blob is stored if the execution succeeded (and produced outputs). DEPRECATED. Use GetTaskExecutionData to fetch output data instead.
error	flyteidl.core.ExecutionError		Error information for the task execution. Populated if the execution failed.
output_data	flyteidl.core.LiteralMap		Deprecated. Raw output data produced by this task execution. DEPRECATED. Use GetTaskExecutionData to fetch output data instead.
phase	flyteidl.core.TaskExecution.Phase		The last recorded phase for this task execution.
logs	flyteidl.core.TaskLog	repeated	Detailed log information output by the task execution.
started_at	google.protobuf.Timestamp		Time at which the task execution began running.
duration	google.protobuf.Duration		The amount of time the task execution spent running.
created_at	google.protobuf.Timestamp		Time at which the task execution was created.
updated_at	google.protobuf.Timestamp		Time at which the task execution was last updated.
custom_info	google.protobuf.Struct		Custom data specific to the task plugin.
reason	string		If there is an explanation for the most recent phase transition, the reason will capture it.
task_type	string		A predefined yet extensible Task type identifier.
metadata	flyteidl.event.TaskExecutionMetadata		Metadata around how a task was executed.
event_version	int32		The event version is used to indicate versioned changes in how data is maintained using this proto message. For example, event_verison >gt; 0 means that maps tasks logs use the TaskExecutionMetadata ExternalResourceInfo fields for each subtask rather than the TaskLog in this message.
reasons	Reason	repeated	A time-series of the phase transition or update explanations. This, when compared to storing a singular reason as previously done, is much more valuable in visualizing and understanding historical evaluations.

TaskExecutionGetDataRequest

Request structure to fetch inputs and output for a task execution. By default this data is not returned inline in :ref:ref_flyteidl.admin.TaskExecutionGetRequest

Field	Type	Label	Description
id	flyteidl.core.TaskExecutionIdentifier		The identifier of the task execution for which to fetch inputs and outputs. +required

TaskExecutionGetDataResponse

Response structure for TaskExecutionGetDataRequest which contains inputs and outputs for a task execution.

Field	Type	Label	Description
inputs	UrlBlob		Deprecated. Signed url to fetch a core.LiteralMap of task execution inputs. Deprecated: Please use full_inputs instead.
outputs	UrlBlob		Deprecated. Signed url to fetch a core.LiteralMap of task execution outputs. Deprecated: Please use full_outputs instead.
full_inputs	flyteidl.core.LiteralMap		Full_inputs will only be populated if they are under a configured size threshold.
full_outputs	flyteidl.core.LiteralMap		Full_outputs will only be populated if they are under a configured size threshold.
flyte_urls	FlyteURLs		flyte tiny url to fetch a core.LiteralMap of task execution’s IO Deck will be empty for task

TaskExecutionGetRequest

A message used to fetch a single task execution entity. See :ref:ref_flyteidl.admin.TaskExecution for more details

Field	Type	Label	Description
id	flyteidl.core.TaskExecutionIdentifier		Unique identifier for the task execution. +required

TaskExecutionList

Response structure for a query to list of task execution entities. See :ref:ref_flyteidl.admin.TaskExecution for more details

Field	Type	Label	Description
task_executions	TaskExecution	repeated
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

TaskExecutionListRequest

Represents a request structure to retrieve a list of task execution entities yielded by a specific node execution. See :ref:ref_flyteidl.admin.TaskExecution for more details

Field	Type	Label	Description
node_execution_id	flyteidl.core.NodeExecutionIdentifier		Indicates the node execution to filter by. +required
limit	uint32		Indicates the number of resources to be returned. +required
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional
filters	string		Indicates a list of filters passed as string. More info on constructing filters : <lt;Link>gt; +optional
sort_by	Sort		Sort ordering for returned list. +optional

flyteidl/admin/node_execution.proto

DynamicNodeWorkflowResponse

Field	Type	Label	Description
compiled_workflow	flyteidl.core.CompiledWorkflowClosure

DynamicWorkflowNodeMetadata

For dynamic workflow nodes we capture information about the dynamic workflow definition that gets generated.

Field	Type	Label	Description
id	flyteidl.core.Identifier		id represents the unique identifier of the workflow.
compiled_workflow	flyteidl.core.CompiledWorkflowClosure		Represents the compiled representation of the embedded dynamic workflow.
dynamic_job_spec_uri	string		dynamic_job_spec_uri is the location of the DynamicJobSpec proto message for this DynamicWorkflow. This is required to correctly recover partially completed executions where the subworkflow has already been compiled.

GetDynamicNodeWorkflowRequest

Field	Type	Label	Description
id	flyteidl.core.NodeExecutionIdentifier

NodeExecution

Encapsulates all details for a single node execution entity. A node represents a component in the overall workflow graph. A node launch a task, multiple tasks, an entire nested sub-workflow, or even a separate child-workflow execution. The same task can be called repeatedly in a single workflow but each node is unique.

Field	Type	Label	Description
id	flyteidl.core.NodeExecutionIdentifier		Uniquely identifies an individual node execution.
input_uri	string		Path to remote data store where input blob is stored.
closure	NodeExecutionClosure		Computed results associated with this node execution.
metadata	NodeExecutionMetaData		Metadata for Node Execution

NodeExecutionClosure

Container for node execution details and results.

Field	Type	Label	Description
output_uri	string		Deprecated. Links to a remotely stored, serialized core.LiteralMap of node execution outputs. DEPRECATED. Use GetNodeExecutionData to fetch output data instead.
error	flyteidl.core.ExecutionError		Error information for the Node
output_data	flyteidl.core.LiteralMap		Deprecated. Raw output data produced by this node execution. DEPRECATED. Use GetNodeExecutionData to fetch output data instead.
phase	flyteidl.core.NodeExecution.Phase		The last recorded phase for this node execution.
started_at	google.protobuf.Timestamp		Time at which the node execution began running.
duration	google.protobuf.Duration		The amount of time the node execution spent running.
created_at	google.protobuf.Timestamp		Time at which the node execution was created.
updated_at	google.protobuf.Timestamp		Time at which the node execution was last updated.
workflow_node_metadata	WorkflowNodeMetadata
task_node_metadata	TaskNodeMetadata
deck_uri	string		String location uniquely identifying where the deck HTML file is. NativeUrl specifies the url in the format of the configured storage provider (e.g. s3://my-bucket/randomstring/suffix.tar)
dynamic_job_spec_uri	string		dynamic_job_spec_uri is the location of the DynamicJobSpec proto message for a DynamicWorkflow. This is required to correctly recover partially completed executions where the subworkflow has already been compiled.

NodeExecutionForTaskListRequest

Represents a request structure to retrieve a list of node execution entities launched by a specific task. This can arise when a task yields a subworkflow.

Field	Type	Label	Description
task_execution_id	flyteidl.core.TaskExecutionIdentifier		Indicates the node execution to filter by. +required
limit	uint32		Indicates the number of resources to be returned. +required
token	string		In the case of multiple pages of results, the, server-provided token can be used to fetch the next page in a query. +optional
filters	string		Indicates a list of filters passed as string. More info on constructing filters : <lt;Link>gt; +optional
sort_by	Sort		Sort ordering. +optional

NodeExecutionGetDataRequest

Request structure to fetch inputs and output for a node execution. By default, these are not returned in :ref:ref_flyteidl.admin.NodeExecutionGetRequest

Field	Type	Label	Description
id	flyteidl.core.NodeExecutionIdentifier		The identifier of the node execution for which to fetch inputs and outputs.

NodeExecutionGetDataResponse

Response structure for NodeExecutionGetDataRequest which contains inputs and outputs for a node execution.

Field	Type	Label	Description
inputs	UrlBlob		Deprecated. Signed url to fetch a core.LiteralMap of node execution inputs. Deprecated: Please use full_inputs instead.
outputs	UrlBlob		Deprecated. Signed url to fetch a core.LiteralMap of node execution outputs. Deprecated: Please use full_outputs instead.
full_inputs	flyteidl.core.LiteralMap		Full_inputs will only be populated if they are under a configured size threshold.
full_outputs	flyteidl.core.LiteralMap		Full_outputs will only be populated if they are under a configured size threshold.
dynamic_workflow	DynamicWorkflowNodeMetadata		Optional Workflow closure for a dynamically generated workflow, in the case this node yields a dynamic workflow we return its structure here.
flyte_urls	FlyteURLs

NodeExecutionGetRequest

A message used to fetch a single node execution entity. See :ref:ref_flyteidl.admin.NodeExecution for more details

Field	Type	Label	Description
id	flyteidl.core.NodeExecutionIdentifier		Uniquely identifies an individual node execution. +required

NodeExecutionList

Request structure to retrieve a list of node execution entities. See :ref:ref_flyteidl.admin.NodeExecution for more details

Field	Type	Label	Description
node_executions	NodeExecution	repeated
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

NodeExecutionListRequest

Represents a request structure to retrieve a list of node execution entities. See :ref:ref_flyteidl.admin.NodeExecution for more details

Field	Type	Label	Description
workflow_execution_id	flyteidl.core.WorkflowExecutionIdentifier		Indicates the workflow execution to filter by. +required
limit	uint32		Indicates the number of resources to be returned. +required
token	string
filters	string		Indicates a list of filters passed as string. More info on constructing filters : <lt;Link>gt; +optional
sort_by	Sort		Sort ordering. +optional
unique_parent_id	string		Unique identifier of the parent node in the execution +optional

NodeExecutionMetaData

Represents additional attributes related to a Node Execution

Field	Type	Label	Description
retry_group	string		Node executions are grouped depending on retries of the parent Retry group is unique within the context of a parent node.
is_parent_node	bool		Boolean flag indicating if the node has child nodes under it This can be true when a node contains a dynamic workflow which then produces child nodes.
spec_node_id	string		Node id of the node in the original workflow This maps to value of WorkflowTemplate.nodes[X].id
is_dynamic	bool		Boolean flag indicating if the node has contains a dynamic workflow which then produces child nodes. This is to distinguish between subworkflows and dynamic workflows which can both have is_parent_node as true.
is_array	bool		Boolean flag indicating if the node is an array node. This is intended to uniquely identify array nodes from other nodes which can have is_parent_node as true.
is_eager	bool		Whether this node is an eager node.

TaskNodeMetadata

Metadata for the case in which the node is a TaskNode

Field	Type	Label	Description
cache_status	flyteidl.core.CatalogCacheStatus		Captures the status of caching for this execution.
catalog_key	flyteidl.core.CatalogMetadata		This structure carries the catalog artifact information
checkpoint_uri	string		The latest checkpoint location

WorkflowNodeMetadata

Metadata for a WorkflowNode

Field	Type	Label	Description
executionId	flyteidl.core.WorkflowExecutionIdentifier		The identifier for a workflow execution launched by a node.

flyteidl/admin/execution.proto

AbortMetadata

Specifies metadata around an aborted workflow execution.

Field	Type	Label	Description
cause	string		In the case of a user-specified abort, this will pass along the user-supplied cause.
principal	string		Identifies the entity (if any) responsible for terminating the execution

Execution

A workflow execution represents an instantiated workflow, including all inputs and additional metadata as well as computed results included state, outputs, and duration-based attributes. Used as a response object used in Get and List execution requests.

Field	Type	Label	Description
id	flyteidl.core.WorkflowExecutionIdentifier		Unique identifier of the workflow execution.
spec	ExecutionSpec		User-provided configuration and inputs for launching the execution.
closure	ExecutionClosure		Execution results.

ExecutionClosure

Encapsulates the results of the Execution

Field	Type	Label	Description
outputs	LiteralMapBlob		Deprecated. Output URI in the case of a successful execution. DEPRECATED. Use GetExecutionData to fetch output data instead.
error	flyteidl.core.ExecutionError		Error information in the case of a failed execution.
abort_cause	string		Deprecated. In the case of a user-specified abort, this will pass along the user-supplied cause.
abort_metadata	AbortMetadata		In the case of a user-specified abort, this will pass along the user and their supplied cause.
output_data	flyteidl.core.LiteralMap		Deprecated. Raw output data produced by this execution. DEPRECATED. Use GetExecutionData to fetch output data instead.
computed_inputs	flyteidl.core.LiteralMap		Deprecated. Inputs computed and passed for execution. computed_inputs depends on inputs in ExecutionSpec, fixed and default inputs in launch plan
phase	flyteidl.core.WorkflowExecution.Phase		Most recent recorded phase for the execution.
started_at	google.protobuf.Timestamp		Reported time at which the execution began running.
duration	google.protobuf.Duration		The amount of time the execution spent running.
created_at	google.protobuf.Timestamp		Reported time at which the execution was created.
updated_at	google.protobuf.Timestamp		Reported time at which the execution was last updated.
notifications	Notification	repeated	The notification settings to use after merging the CreateExecutionRequest and the launch plan notification settings. An execution launched with notifications will always prefer that definition to notifications defined statically in a launch plan.
workflow_id	flyteidl.core.Identifier		Identifies the workflow definition for this execution.
state_change_details	ExecutionStateChangeDetails		Provides the details of the last stage change

ExecutionCreateRequest

Request to launch an execution with the given project, domain and optionally-assigned name.

Field	Type	Label	Description
project	string		Name of the project the execution belongs to. +required
domain	string		Name of the domain the execution belongs to. A domain can be considered as a subset within a specific project. +required
name	string		User provided value for the resource. If none is provided the system will generate a unique string. +optional
spec	ExecutionSpec		Additional fields necessary to launch the execution. +optional
inputs	flyteidl.core.LiteralMap		The inputs required to start the execution. All required inputs must be included in this map. If not required and not provided, defaults apply. +optional
org	string		Optional, org key applied to the resource.

ExecutionCreateResponse

The unique identifier for a successfully created execution. If the name was not specified in the create request, this identifier will include a generated name.

Field	Type	Label	Description
id	flyteidl.core.WorkflowExecutionIdentifier

ExecutionList

Used as a response for request to list executions. See :ref:ref_flyteidl.admin.Execution for more details

Field	Type	Label	Description
executions	Execution	repeated
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

ExecutionMetadata

Represents attributes about an execution which are not required to launch the execution but are useful to record. These attributes are assigned at launch time and do not change.

Field	Type	Label	Description
mode	ExecutionMetadata.ExecutionMode
principal	string		Identifier of the entity that triggered this execution. For systems using back-end authentication any value set here will be discarded in favor of the authenticated user context.
nesting	uint32		Indicates the nestedness of this execution. If a user launches a workflow execution, the default nesting is 0. If this execution further launches a workflow (child workflow), the nesting level is incremented by 0 =>gt; 1 Generally, if workflow at nesting level k launches a workflow then the child workflow will have nesting = k + 1.
scheduled_at	google.protobuf.Timestamp		For scheduled executions, the requested time for execution for this specific schedule invocation.
parent_node_execution	flyteidl.core.NodeExecutionIdentifier		Which subworkflow node (if any) launched this execution
reference_execution	flyteidl.core.WorkflowExecutionIdentifier		Optional, a reference workflow execution related to this execution. In the case of a relaunch, this references the original workflow execution.
system_metadata	SystemMetadata		Optional, platform-specific metadata about the execution. In this the future this may be gated behind an ACL or some sort of authorization.
artifact_ids	flyteidl.core.ArtifactID	repeated	Save a list of the artifacts used in this execution for now. This is a list only rather than a mapping since we don’t have a structure to handle nested ones anyways.

ExecutionRecoverRequest

Request to recover the referenced execution.

Field	Type	Label	Description
id	flyteidl.core.WorkflowExecutionIdentifier		Identifier of the workflow execution to recover.
name	string		User provided value for the recovered execution. If none is provided the system will generate a unique string. +optional
metadata	ExecutionMetadata		Additional metadata which will be used to overwrite any metadata in the reference execution when triggering a recovery execution.

ExecutionRelaunchRequest

Request to relaunch the referenced execution.

Field	Type	Label	Description
id	flyteidl.core.WorkflowExecutionIdentifier		Identifier of the workflow execution to relaunch. +required
name	string		User provided value for the relaunched execution. If none is provided the system will generate a unique string. +optional
overwrite_cache	bool		Allows for all cached values of a workflow and its tasks to be overwritten for a single execution. If enabled, all calculations are performed even if cached results would be available, overwriting the stored data once execution finishes successfully.

ExecutionSpec

An ExecutionSpec encompasses all data used to launch this execution. The Spec does not change over the lifetime of an execution as it progresses across phase changes.

Field	Type	Label	Description
launch_plan	flyteidl.core.Identifier		Launch plan to be executed
inputs	flyteidl.core.LiteralMap		Deprecated. Input values to be passed for the execution
metadata	ExecutionMetadata		Metadata for the execution
notifications	NotificationList		List of notifications based on Execution status transitions When this list is not empty it is used rather than any notifications defined in the referenced launch plan. When this list is empty, the notifications defined for the launch plan will be applied.
disable_all	bool		This should be set to true if all notifications are intended to be disabled for this execution.
labels	Labels		Labels to apply to the execution resource.
annotations	Annotations		Annotations to apply to the execution resource.
security_context	flyteidl.core.SecurityContext		Optional: security context override to apply this execution.
auth_role	AuthRole		Deprecated. Optional: auth override to apply this execution.
quality_of_service	flyteidl.core.QualityOfService		Indicates the runtime priority of the execution.
max_parallelism	int32		Controls the maximum number of task nodes that can be run in parallel for the entire workflow. This is useful to achieve fairness. Note: MapTasks are regarded as one unit, and parallelism/concurrency of MapTasks is independent from this.
raw_output_data_config	RawOutputDataConfig		User setting to configure where to store offloaded data (i.e. Blobs, structured datasets, query data, etc.). This should be a prefix like s3://my-bucket/my-data
cluster_assignment	ClusterAssignment		Controls how to select an available cluster on which this execution should run.
interruptible	google.protobuf.BoolValue		Allows for the interruptible flag of a workflow to be overwritten for a single execution. Omitting this field uses the workflow’s value as a default. As we need to distinguish between the field not being provided and its default value false, we have to use a wrapper around the bool field.
overwrite_cache	bool		Allows for all cached values of a workflow and its tasks to be overwritten for a single execution. If enabled, all calculations are performed even if cached results would be available, overwriting the stored data once execution finishes successfully.
envs	Envs		Environment variables to be set for the execution.
tags	string	repeated	Deprecated. Tags to be set for the execution.
execution_cluster_label	ExecutionClusterLabel		Execution cluster label to be set for the execution.
execution_env_assignments	flyteidl.core.ExecutionEnvAssignment	repeated	Execution environment assignments to be set for the execution.

ExecutionStateChangeDetails

Field	Type	Label	Description
state	ExecutionState		The state of the execution is used to control its visibility in the UI/CLI.
occurred_at	google.protobuf.Timestamp		This timestamp represents when the state changed.
principal	string		Identifies the entity (if any) responsible for causing the state change of the execution

ExecutionTerminateRequest

Request to terminate an in-progress execution. This action is irreversible. If an execution is already terminated, this request will simply be a no-op. This request will fail if it references a non-existent execution. If the request succeeds the phase “ABORTED” will be recorded for the termination with the optional cause added to the output_result.

Field	Type	Label	Description
id	flyteidl.core.WorkflowExecutionIdentifier		Uniquely identifies the individual workflow execution to be terminated.
cause	string		Optional reason for aborting.

ExecutionTerminateResponse

Purposefully empty, may be populated in the future.

ExecutionUpdateRequest

Field	Type	Label	Description
id	flyteidl.core.WorkflowExecutionIdentifier		Identifier of the execution to update
state	ExecutionState		State to set as the new value active/archive

ExecutionUpdateResponse

LiteralMapBlob

Input/output data can represented by actual values or a link to where values are stored

Field	Type	Label	Description
values	flyteidl.core.LiteralMap		Deprecated. Data in LiteralMap format
uri	string		In the event that the map is too large, we return a uri to the data

NotificationList

Field	Type	Label	Description
notifications	Notification	repeated

SystemMetadata

Represents system, rather than user-facing, metadata about an execution.

Field	Type	Label	Description
execution_cluster	string		Which execution cluster this execution ran on.
namespace	string		Which kubernetes namespace the execution ran under.

WorkflowExecutionGetDataRequest

Request structure to fetch inputs, output and other data produced by an execution. By default this data is not returned inline in :ref:ref_flyteidl.admin.WorkflowExecutionGetRequest

Field	Type	Label	Description
id	flyteidl.core.WorkflowExecutionIdentifier		The identifier of the execution for which to fetch inputs and outputs.

WorkflowExecutionGetDataResponse

Response structure for WorkflowExecutionGetDataRequest which contains inputs and outputs for an execution.

Field	Type	Label	Description
outputs	UrlBlob		Deprecated. Signed url to fetch a core.LiteralMap of execution outputs. Deprecated: Please use full_outputs instead.
inputs	UrlBlob		Deprecated. Signed url to fetch a core.LiteralMap of execution inputs. Deprecated: Please use full_inputs instead.
full_inputs	flyteidl.core.LiteralMap		Full_inputs will only be populated if they are under a configured size threshold.
full_outputs	flyteidl.core.LiteralMap		Full_outputs will only be populated if they are under a configured size threshold.

WorkflowExecutionGetMetricsRequest

WorkflowExecutionGetMetricsRequest represents a request to retrieve metrics for the specified workflow execution.

Field	Type	Label	Description
id	flyteidl.core.WorkflowExecutionIdentifier		id defines the workflow execution to query for.
depth	int32		depth defines the number of Flyte entity levels to traverse when breaking down execution details.

WorkflowExecutionGetMetricsResponse

WorkflowExecutionGetMetricsResponse represents the response containing metrics for the specified workflow execution.

Field	Type	Label	Description
span	flyteidl.core.Span		Span defines the top-level breakdown of the workflows execution. More precise information is nested in a hierarchical structure using Flyte entity references.

WorkflowExecutionGetRequest

A message used to fetch a single workflow execution entity. See :ref:ref_flyteidl.admin.Execution for more details

Field	Type	Label	Description
id	flyteidl.core.WorkflowExecutionIdentifier		Uniquely identifies an individual workflow execution.

ExecutionMetadata.ExecutionMode

The method by which this execution was launched.

Name	Number	Description
MANUAL	0	The default execution mode, MANUAL implies that an execution was launched by an individual.
SCHEDULED	1	A schedule triggered this execution launch.
SYSTEM	2	A system process was responsible for launching this execution rather an individual.
RELAUNCH	3	This execution was launched with identical inputs as a previous execution.
CHILD_WORKFLOW	4	This execution was triggered by another execution.
RECOVERED	5	This execution was recovered from another execution.
TRIGGER	6	Execution was kicked off by the artifact trigger system

ExecutionState

The state of the execution is used to control its visibility in the UI/CLI.

Name	Number	Description
EXECUTION_ACTIVE	0	By default, all executions are considered active.
EXECUTION_ARCHIVED	1	Archived executions are no longer visible in the UI.

flyteidl/admin/workflow_attributes.proto

WorkflowAttributes

Defines a set of custom matching attributes which defines resource defaults for a project, domain and workflow. For more info on matchable attributes, see :ref:ref_flyteidl.admin.MatchableAttributesConfiguration

Field	Type	Label	Description
project	string		Unique project id for which this set of attributes will be applied.
domain	string		Unique domain id for which this set of attributes will be applied.
workflow	string		Workflow name for which this set of attributes will be applied.
matching_attributes	MatchingAttributes
org	string		Optional, org key applied to the attributes.

WorkflowAttributesDeleteRequest

Request to delete a set matchable workflow attribute override. For more info on matchable attributes, see :ref:ref_flyteidl.admin.MatchableAttributesConfiguration

Field	Type	Label	Description
project	string		Unique project id which this set of attributes references. +required
domain	string		Unique domain id which this set of attributes references. +required
workflow	string		Workflow name which this set of attributes references. +required
resource_type	MatchableResource		Which type of matchable attributes to delete. +required
org	string		Optional, org key applied to the attributes.

WorkflowAttributesDeleteResponse

Purposefully empty, may be populated in the future.

WorkflowAttributesGetRequest

Request to get an individual workflow attribute override. For more info on matchable attributes, see :ref:ref_flyteidl.admin.MatchableAttributesConfiguration

Field	Type	Label	Description
project	string		Unique project id which this set of attributes references. +required
domain	string		Unique domain id which this set of attributes references. +required
workflow	string		Workflow name which this set of attributes references. +required
resource_type	MatchableResource		Which type of matchable attributes to return. +required
org	string		Optional, org key applied to the attributes.

WorkflowAttributesGetResponse

Response to get an individual workflow attribute override.

Field	Type	Label	Description
attributes	WorkflowAttributes

WorkflowAttributesUpdateRequest

Sets custom attributes for a project, domain and workflow combination. For more info on matchable attributes, see :ref:ref_flyteidl.admin.MatchableAttributesConfiguration

Field	Type	Label	Description
attributes	WorkflowAttributes

WorkflowAttributesUpdateResponse

Purposefully empty, may be populated in the future.

flyteidl/admin/event.proto

EventErrorAlreadyInTerminalState

Indicates that a sent event was not used to update execution state due to the referenced execution already being terminated (and therefore ineligible for further state transitions).

Field	Type	Label	Description
current_phase	string		+required

EventErrorIncompatibleCluster

Indicates an event was rejected because it came from a different cluster than is on record as running the execution.

Field	Type	Label	Description
cluster	string		The cluster which has been recorded as processing the execution. +required

EventFailureReason

Indicates why a sent event was not used to update execution.

Field	Type	Label	Description
already_in_terminal_state	EventErrorAlreadyInTerminalState
incompatible_cluster	EventErrorIncompatibleCluster

NodeExecutionEventRequest

Request to send a notification that a node execution event has occurred.

Field	Type	Label	Description
request_id	string		Unique ID for this request that can be traced between services
event	flyteidl.event.NodeExecutionEvent		Details about the event that occurred.

NodeExecutionEventResponse

Purposefully empty, may be populated in the future.

TaskExecutionEventRequest

Request to send a notification that a task execution event has occurred.

Field	Type	Label	Description
request_id	string		Unique ID for this request that can be traced between services
event	flyteidl.event.TaskExecutionEvent		Details about the event that occurred.

TaskExecutionEventResponse

Purposefully empty, may be populated in the future.

WorkflowExecutionEventRequest

Request to send a notification that a workflow execution event has occurred.

Field	Type	Label	Description
request_id	string		Unique ID for this request that can be traced between services
event	flyteidl.event.WorkflowExecutionEvent		Details about the event that occurred.

WorkflowExecutionEventResponse

Purposefully empty, may be populated in the future.

flyteidl/admin/matchable_resource.proto

ClusterResourceAttributes

Field	Type	Label	Description
attributes	ClusterResourceAttributes.AttributesEntry	repeated	Custom resource attributes which will be applied in cluster resource creation (e.g. quotas). Map keys are the case-sensitive names of variables in templatized resource files. Map values should be the custom values which get substituted during resource creation.

ClusterResourceAttributes.AttributesEntry

Field	Type	Label	Description
key	string
value	string

ExecutionClusterLabel

Field	Type	Label	Description
value	string		Label value to determine where the execution will be run

ExecutionQueueAttributes

Field	Type	Label	Description
tags	string	repeated	Tags used for assigning execution queues for tasks defined within this project.

ListMatchableAttributesRequest

Request all matching resource attributes for a resource type. See :ref:ref_flyteidl.admin.MatchableAttributesConfiguration for more details

Field	Type	Label	Description
resource_type	MatchableResource		+required
org	string		Optional, org filter applied to list project requests.

ListMatchableAttributesResponse

Response for a request for all matching resource attributes for a resource type. See :ref:ref_flyteidl.admin.MatchableAttributesConfiguration for more details

Field	Type	Label	Description
configurations	MatchableAttributesConfiguration	repeated

MatchableAttributesConfiguration

Represents a custom set of attributes applied for either a domain (and optional org); a domain and project (and optional org); or domain, project and workflow name (and optional org). These are used to override system level defaults for kubernetes cluster resource management, default execution values, and more all across different levels of specificity.

Field	Type	Label	Description
attributes	MatchingAttributes
domain	string
project	string
workflow	string
launch_plan	string
org	string		Optional, org key applied to the resource.

MatchingAttributes

Generic container for encapsulating all types of the above attributes messages.

Field	Type	Label	Description
task_resource_attributes	TaskResourceAttributes
cluster_resource_attributes	ClusterResourceAttributes
execution_queue_attributes	ExecutionQueueAttributes
execution_cluster_label	ExecutionClusterLabel
quality_of_service	flyteidl.core.QualityOfService
plugin_overrides	PluginOverrides
workflow_execution_config	WorkflowExecutionConfig
cluster_assignment	ClusterAssignment

PluginOverride

This MatchableAttribute configures selecting alternate plugin implementations for a given task type. In addition to an override implementation a selection of fallbacks can be provided or other modes for handling cases where the desired plugin override is not enabled in a given Flyte deployment.

Field	Type	Label	Description
task_type	string		A predefined yet extensible Task type identifier.
plugin_id	string	repeated	A set of plugin ids which should handle tasks of this type instead of the default registered plugin. The list will be tried in order until a plugin is found with that id.
missing_plugin_behavior	PluginOverride.MissingPluginBehavior		Defines the behavior when no plugin from the plugin_id list is not found.

PluginOverrides

Field	Type	Label	Description
overrides	PluginOverride	repeated

TaskResourceAttributes

Defines task resource defaults and limits that will be applied at task registration.

Field	Type	Label	Description
defaults	TaskResourceSpec
limits	TaskResourceSpec

TaskResourceSpec

Defines a set of overridable task resource attributes set during task registration.

Field	Type	Label	Description
cpu	string
gpu	string
memory	string
storage	string
ephemeral_storage	string

WorkflowExecutionConfig

Adds defaults for customizable workflow-execution specifications and overrides.

Field	Type	Label	Description
max_parallelism	int32		Can be used to control the number of parallel nodes to run within the workflow. This is useful to achieve fairness.
security_context	flyteidl.core.SecurityContext		Indicates security context permissions for executions triggered with this matchable attribute.
raw_output_data_config	RawOutputDataConfig		Encapsulates user settings pertaining to offloaded data (i.e. Blobs, Schema, query data, etc.).
labels	Labels		Custom labels to be applied to a triggered execution resource.
annotations	Annotations		Custom annotations to be applied to a triggered execution resource.
interruptible	google.protobuf.BoolValue		Allows for the interruptible flag of a workflow to be overwritten for a single execution. Omitting this field uses the workflow’s value as a default. As we need to distinguish between the field not being provided and its default value false, we have to use a wrapper around the bool field.
overwrite_cache	bool		Allows for all cached values of a workflow and its tasks to be overwritten for a single execution. If enabled, all calculations are performed even if cached results would be available, overwriting the stored data once execution finishes successfully.
envs	Envs		Environment variables to be set for the execution.
execution_env_assignments	flyteidl.core.ExecutionEnvAssignment	repeated	Execution environment assignments to be set for the execution.

MatchableResource

Defines a resource that can be configured by customizable Project-, ProjectDomain- or WorkflowAttributes based on matching tags.

Name	Number	Description
TASK_RESOURCE	0	Applies to customizable task resource requests and limits.
CLUSTER_RESOURCE	1	Applies to configuring templated kubernetes cluster resources.
EXECUTION_QUEUE	2	Configures task and dynamic task execution queue assignment.
EXECUTION_CLUSTER_LABEL	3	Configures the K8s cluster label to be used for execution to be run
QUALITY_OF_SERVICE_SPECIFICATION	4	Configures default quality of service when undefined in an execution spec.
PLUGIN_OVERRIDE	5	Selects configurable plugin implementation behavior for a given task type.
WORKFLOW_EXECUTION_CONFIG	6	Adds defaults for customizable workflow-execution specifications and overrides.
CLUSTER_ASSIGNMENT	7	Controls how to select an available cluster on which this execution should run.

PluginOverride.MissingPluginBehavior

Name	Number	Description
FAIL	0	By default, if this plugin is not enabled for a Flyte deployment then execution will fail.
USE_DEFAULT	1	Uses the system-configured default implementation.

flyteidl/admin/project_attributes.proto

ProjectAttributes

Defines a set of custom matching attributes at the project level. For more info on matchable attributes, see :ref:ref_flyteidl.admin.MatchableAttributesConfiguration

Field	Type	Label	Description
project	string		Unique project id for which this set of attributes will be applied.
matching_attributes	MatchingAttributes
org	string		Optional, org key applied to the project.

ProjectAttributesDeleteRequest

Request to delete a set matchable project level attribute override. For more info on matchable attributes, see :ref:ref_flyteidl.admin.MatchableAttributesConfiguration

Field	Type	Label	Description
project	string		Unique project id which this set of attributes references. +required
resource_type	MatchableResource		Which type of matchable attributes to delete. +required
org	string		Optional, org key applied to the project.

ProjectAttributesDeleteResponse

Purposefully empty, may be populated in the future.

ProjectAttributesGetRequest

Request to get an individual project level attribute override. For more info on matchable attributes, see :ref:ref_flyteidl.admin.MatchableAttributesConfiguration

Field	Type	Label	Description
project	string		Unique project id which this set of attributes references. +required
resource_type	MatchableResource		Which type of matchable attributes to return. +required
org	string		Optional, org key applied to the project.

ProjectAttributesGetResponse

Response to get an individual project level attribute override. For more info on matchable attributes, see :ref:ref_flyteidl.admin.MatchableAttributesConfiguration

Field	Type	Label	Description
attributes	ProjectAttributes

ProjectAttributesUpdateRequest

Sets custom attributes for a project For more info on matchable attributes, see :ref:ref_flyteidl.admin.MatchableAttributesConfiguration

Field	Type	Label	Description
attributes	ProjectAttributes		+required

ProjectAttributesUpdateResponse

Purposefully empty, may be populated in the future.

flyteidl/admin/version.proto

GetVersionRequest

Empty request for GetVersion

GetVersionResponse

Response for the GetVersion API

Field	Type	Label	Description
control_plane_version	Version		The control plane version information. FlyteAdmin and related components form the control plane of Flyte

Version

Provides Version information for a component

Field	Type	Label	Description
Build	string		Specifies the GIT sha of the build
Version	string		Version for the build, should follow a semver
BuildTime	string		Build timestamp

flyteidl/admin/workflow.proto

CreateWorkflowFailureReason

When a CreateWorkflowRequest fails due to matching id

Field	Type	Label	Description
exists_different_structure	WorkflowErrorExistsDifferentStructure
exists_identical_structure	WorkflowErrorExistsIdenticalStructure

Workflow

Represents the workflow structure stored in the Admin A workflow is created by ordering tasks and associating outputs to inputs in order to produce a directed-acyclic execution graph.

Field	Type	Label	Description
id	flyteidl.core.Identifier		id represents the unique identifier of the workflow.
closure	WorkflowClosure		closure encapsulates all the fields that maps to a compiled version of the workflow.
short_description	string		One-liner overview of the entity.

WorkflowClosure

A container holding the compiled workflow produced from the WorkflowSpec and additional metadata.

Field	Type	Label	Description
compiled_workflow	flyteidl.core.CompiledWorkflowClosure		Represents the compiled representation of the workflow from the specification provided.
created_at	google.protobuf.Timestamp		Time at which the workflow was created.

WorkflowCreateRequest

Represents a request structure to create a revision of a workflow. See :ref:ref_flyteidl.admin.Workflow for more details

Field	Type	Label	Description
id	flyteidl.core.Identifier		id represents the unique identifier of the workflow. +required
spec	WorkflowSpec		Represents the specification for workflow. +required

WorkflowCreateResponse

Purposefully empty, may be populated in the future.

WorkflowErrorExistsDifferentStructure

The workflow id is already used and the structure is different

Field	Type	Label	Description
id	flyteidl.core.Identifier

WorkflowErrorExistsIdenticalStructure

The workflow id is already used with an identical sctructure

Field	Type	Label	Description
id	flyteidl.core.Identifier

WorkflowList

Represents a list of workflows returned from the admin. See :ref:ref_flyteidl.admin.Workflow for more details

Field	Type	Label	Description
workflows	Workflow	repeated	A list of workflows returned based on the request.
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

WorkflowSpec

Represents a structure that encapsulates the specification of the workflow.

Field	Type	Label	Description
template	flyteidl.core.WorkflowTemplate		Template of the task that encapsulates all the metadata of the workflow.
sub_workflows	flyteidl.core.WorkflowTemplate	repeated	Workflows that are embedded into other workflows need to be passed alongside the parent workflow to the propeller compiler (since the compiler doesn’t have any knowledge of other workflows - ie, it doesn’t reach out to Admin to see other registered workflows). In fact, subworkflows do not even need to be registered.
description	DescriptionEntity		Represents the specification for description entity.

flyteidl/admin/description_entity.proto

Description

Full user description with formatting preserved. This can be rendered by clients, such as the console or command line tools with in-tact formatting.

Field	Type	Label	Description
value	string		long description - no more than 4KB
uri	string		if the description sizes exceed some threshold we can offload the entire description proto altogether to an external data store, like S3 rather than store inline in the db
format	DescriptionFormat		Format of the long description
icon_link	string		Optional link to an icon for the entity

DescriptionEntity

DescriptionEntity contains detailed description for the task/workflow. Documentation could provide insight into the algorithms, business use case, etc.

Field	Type	Label	Description
id	flyteidl.core.Identifier		id represents the unique identifier of the description entity.
short_description	string		One-liner overview of the entity.
long_description	Description		Full user description with formatting preserved.
source_code	SourceCode		Optional link to source code used to define this entity.
tags	string	repeated	User-specified tags. These are arbitrary and can be used for searching filtering and discovering tasks.

DescriptionEntityList

Represents a list of DescriptionEntities returned from the admin. See :ref:ref_flyteidl.admin.DescriptionEntity for more details

Field	Type	Label	Description
descriptionEntities	DescriptionEntity	repeated	A list of DescriptionEntities returned based on the request.
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

DescriptionEntityListRequest

Represents a request structure to retrieve a list of DescriptionEntities. See :ref:ref_flyteidl.admin.DescriptionEntity for more details

Field	Type	Label	Description
resource_type	flyteidl.core.ResourceType		Identifies the specific type of resource that this identifier corresponds to.
id	NamedEntityIdentifier		The identifier for the description entity. +required
limit	uint32		Indicates the number of resources to be returned. +required
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional
filters	string		Indicates a list of filters passed as string. More info on constructing filters : <lt;Link>gt; +optional
sort_by	Sort		Sort ordering for returned list. +optional

SourceCode

Link to source code used to define this entity

Field	Type	Label	Description
link	string

DescriptionFormat

The format of the long description

Name	Number	Description
DESCRIPTION_FORMAT_UNKNOWN	0
DESCRIPTION_FORMAT_MARKDOWN	1
DESCRIPTION_FORMAT_HTML	2
DESCRIPTION_FORMAT_RST	3	python default documentation - comments is rst

flyteidl/admin/project_domain_attributes.proto

ProjectDomainAttributes

Defines a set of custom matching attributes which defines resource defaults for a project and domain. For more info on matchable attributes, see :ref:ref_flyteidl.admin.MatchableAttributesConfiguration

Field	Type	Label	Description
project	string		Unique project id for which this set of attributes will be applied.
domain	string		Unique domain id for which this set of attributes will be applied.
matching_attributes	MatchingAttributes
org	string		Optional, org key applied to the attributes.

ProjectDomainAttributesDeleteRequest

Request to delete a set matchable project domain attribute override. For more info on matchable attributes, see :ref:ref_flyteidl.admin.MatchableAttributesConfiguration

Field	Type	Label	Description
project	string		Unique project id which this set of attributes references. +required
domain	string		Unique domain id which this set of attributes references. +required
resource_type	MatchableResource		Which type of matchable attributes to delete. +required
org	string		Optional, org key applied to the attributes.

ProjectDomainAttributesDeleteResponse

Purposefully empty, may be populated in the future.

ProjectDomainAttributesGetRequest

Request to get an individual project domain attribute override. For more info on matchable attributes, see :ref:ref_flyteidl.admin.MatchableAttributesConfiguration

Field	Type	Label	Description
project	string		Unique project id which this set of attributes references. +required
domain	string		Unique domain id which this set of attributes references. +required
resource_type	MatchableResource		Which type of matchable attributes to return. +required
org	string		Optional, org key applied to the attributes.

ProjectDomainAttributesGetResponse

Response to get an individual project domain attribute override. For more info on matchable attributes, see :ref:ref_flyteidl.admin.MatchableAttributesConfiguration

Field	Type	Label	Description
attributes	ProjectDomainAttributes

ProjectDomainAttributesUpdateRequest

Sets custom attributes for a project-domain combination. For more info on matchable attributes, see :ref:ref_flyteidl.admin.MatchableAttributesConfiguration

Field	Type	Label	Description
attributes	ProjectDomainAttributes		+required

ProjectDomainAttributesUpdateResponse

Purposefully empty, may be populated in the future.

flyteidl/admin/agent.proto

Agent

A message containing the agent metadata.

Field	Type	Label	Description
name	string		Name is the developer-assigned name of the agent.
supported_task_types	string	repeated	Deprecated. SupportedTaskTypes are the types of the tasks that the agent can handle.
is_sync	bool		IsSync indicates whether this agent is a sync agent. Sync agents are expected to return their results synchronously when called by propeller. Given that sync agents can affect the performance of the system, it’s important to enforce strict timeout policies. An Async agent, on the other hand, is required to be able to identify jobs by an identifier and query for job statuses as jobs progress.
supported_task_categories	TaskCategory	repeated	Supported_task_categories are the categories of the tasks that the agent can handle.

AgentError

Error message to propagate detailed errors from agent executions to the execution engine.

Field	Type	Label	Description
code	string		A simplified code for errors, so that we can provide a glossary of all possible errors.
kind	AgentError.Kind		An abstract error kind for this error. Defaults to Non_Recoverable if not specified.
origin	flyteidl.core.ExecutionError.ErrorKind		Defines the origin of the error (system, user, unknown).

CreateRequestHeader

Field	Type	Label	Description
template	flyteidl.core.TaskTemplate		Template of the task that encapsulates all the metadata of the task.
output_prefix	string		Prefix for where task output data will be written. (e.g. s3://my-bucket/randomstring)
task_execution_metadata	TaskExecutionMetadata		subset of runtime task execution metadata.
max_dataset_size_bytes	int64		MaxDatasetSizeBytes is the maximum size of the dataset that can be generated by the task.

CreateTaskRequest

Represents a request structure to create task.

Field	Type	Label	Description
inputs	flyteidl.core.LiteralMap		The inputs required to start the execution. All required inputs must be included in this map. If not required and not provided, defaults apply. +optional
template	flyteidl.core.TaskTemplate		Template of the task that encapsulates all the metadata of the task.
output_prefix	string		Prefix for where task output data will be written. (e.g. s3://my-bucket/randomstring)
task_execution_metadata	TaskExecutionMetadata		subset of runtime task execution metadata.

CreateTaskResponse

Represents a create response structure.

Field	Type	Label	Description
resource_meta	bytes		ResourceMeta is created by the agent. It could be a string (jobId) or a dict (more complex metadata).

DeleteTaskRequest

A message used to delete a task.

Field	Type	Label	Description
task_type	string		Deprecated. A predefined yet extensible Task type identifier.
resource_meta	bytes		Metadata about the resource to be pass to the agent.
task_category	TaskCategory		A predefined yet extensible Task type identifier.

DeleteTaskResponse

Response to delete a task.

ExecuteTaskSyncRequest

Field	Type	Label	Description
header	CreateRequestHeader
inputs	flyteidl.core.LiteralMap

ExecuteTaskSyncResponse

Field	Type	Label	Description
header	ExecuteTaskSyncResponseHeader
outputs	flyteidl.core.LiteralMap

ExecuteTaskSyncResponseHeader

Field	Type	Label	Description
resource	Resource

GetAgentRequest

A request to get an agent.

Field	Type	Label	Description
name	string		The name of the agent.

GetAgentResponse

A response containing an agent.

Field	Type	Label	Description
agent	Agent

GetTaskLogsRequest

A request to get the log from a task execution.

Field	Type	Label	Description
task_type	string		Deprecated. A predefined yet extensible Task type identifier.
resource_meta	bytes		Metadata is created by the agent. It could be a string (jobId) or a dict (more complex metadata).
lines	uint64		Number of lines to return.
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.
task_category	TaskCategory		A predefined yet extensible Task type identifier.

GetTaskLogsResponse

A response containing the logs for a task execution.

Field	Type	Label	Description
header	GetTaskLogsResponseHeader
body	GetTaskLogsResponseBody

GetTaskLogsResponseBody

Field	Type	Label	Description
results	string	repeated	Deprecated. The execution log results.
structured_lines	LogLine	repeated	Each line is separated by either CRLF, CR or LF, which are included at the ends of the lines. This lets clients know whether log emitter wanted to overwrite the previous line (LF) or append a new line (CRLF).

GetTaskLogsResponseHeader

Field	Type	Label	Description
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

GetTaskMetricsRequest

A request to get the metrics from a task execution.

Field	Type	Label	Description
task_type	string		Deprecated. A predefined yet extensible Task type identifier.
resource_meta	bytes		Metadata is created by the agent. It could be a string (jobId) or a dict (more complex metadata).
queries	string	repeated	The metrics to query. If empty, will return a default set of metrics. e.g. EXECUTION_METRIC_USED_CPU_AVG or EXECUTION_METRIC_USED_MEMORY_BYTES_AVG
start_time	google.protobuf.Timestamp		Start timestamp, inclusive.
end_time	google.protobuf.Timestamp		End timestamp, inclusive..
step	google.protobuf.Duration		Query resolution step width in duration format or float number of seconds.
task_category	TaskCategory		A predefined yet extensible Task type identifier.

GetTaskMetricsResponse

A response containing a list of metrics for a task execution.

Field	Type	Label	Description
results	flyteidl.core.ExecutionMetricResult	repeated	The execution metric results.

GetTaskRequest

A message used to fetch a job resource from flyte agent server.

Field	Type	Label	Description
task_type	string		Deprecated. A predefined yet extensible Task type identifier.
resource_meta	bytes		Metadata about the resource to be pass to the agent.
task_category	TaskCategory		A predefined yet extensible Task type identifier.
output_prefix	string		Prefix for where task output data will be written. (e.g. s3://my-bucket/randomstring)

GetTaskResponse

Response to get an individual task resource.

Field	Type	Label	Description
resource	Resource

ListAgentsRequest

A request to list all agents.

ListAgentsResponse

A response containing a list of agents.

Field	Type	Label	Description
agents	Agent	repeated

LogLine

Field	Type	Label	Description
timestamp	google.protobuf.Timestamp
message	string		Each line is separated by either CRLF, CR or LF, which are included at the ends of the lines. This lets clients know whether log emitter wanted to overwrite the previous line (LF) or append a new line (CRLF).
originator	LogLineOriginator

Resource

Field	Type	Label	Description
state	State		Deprecated. DEPRECATED. The state of the execution is used to control its visibility in the UI/CLI.
outputs	flyteidl.core.LiteralMap		The outputs of the execution. It’s typically used by sql task. Agent service will create a Structured dataset pointing to the query result table. +optional
message	string		A descriptive message for the current state. e.g. waiting for cluster.
log_links	flyteidl.core.TaskLog	repeated	log information for the task execution.
phase	flyteidl.core.TaskExecution.Phase		The phase of the execution is used to determine the phase of the plugin’s execution.
custom_info	google.protobuf.Struct		Custom data specific to the agent.
agent_error	AgentError		The error raised during execution

TaskCategory

Field	Type	Label	Description
name	string		The name of the task type.
version	int32		The version of the task type.

TaskExecutionMetadata

Represents a subset of runtime task execution metadata that are relevant to external plugins.

ID of the task execution

Field	Type	Label	Description
task_execution_id	flyteidl.core.TaskExecutionIdentifier
namespace	string		k8s namespace where the task is executed in
labels	TaskExecutionMetadata.LabelsEntry	repeated	Labels attached to the task execution
annotations	TaskExecutionMetadata.AnnotationsEntry	repeated	Annotations attached to the task execution
k8s_service_account	string		k8s service account associated with the task execution
environment_variables	TaskExecutionMetadata.EnvironmentVariablesEntry	repeated	Environment variables attached to the task execution
max_attempts	int32		Represents the maximum number of attempts allowed for a task. If a task fails, it can be retried up to this maximum number of attempts.
interruptible	bool		Indicates whether the task execution can be interrupted. If set to true, the task can be stopped before completion.
interruptible_failure_threshold	int32		Specifies the threshold for failure count at which the interruptible property will take effect. If the number of consecutive task failures exceeds this threshold, interruptible behavior will be activated.
overrides	flyteidl.core.TaskNodeOverrides		Overrides for specific properties of the task node. These overrides can be used to customize the behavior of the task node.
identity	flyteidl.core.Identity		Identity of user running this task execution

TaskExecutionMetadata.AnnotationsEntry

Field	Type	Label	Description
key	string
value	string

TaskExecutionMetadata.EnvironmentVariablesEntry

Field	Type	Label	Description
key	string
value	string

TaskExecutionMetadata.LabelsEntry

Field	Type	Label	Description
key	string
value	string

AgentError.Kind

Defines a generic error type that dictates the behavior of the retry strategy.

Name	Number	Description
NON_RECOVERABLE	0
RECOVERABLE	1

LogLineOriginator

Name	Number	Description
UNKNOWN	0	The originator of the log line is unknown.
USER	1	The originator of the log line is the user application.
SYSTEM	2	The originator of the log line is the system.

State

The state of the execution is used to control its visibility in the UI/CLI.

Name	Number	Description
RETRYABLE_FAILURE	0
PERMANENT_FAILURE	1
PENDING	2
RUNNING	3
SUCCEEDED	4

flyteidl/admin/common.proto

Annotations

Annotation values to be applied to an execution resource. In the future a mode (e.g. OVERRIDE, APPEND, etc) can be defined to specify how to merge annotations defined at registration and execution time.

Field	Type	Label	Description
values	Annotations.ValuesEntry	repeated	Map of custom annotations to be applied to the execution resource.

Annotations.ValuesEntry

Field	Type	Label	Description
key	string
value	string

AuthRole

Defines permissions associated with executions created by this launch plan spec. Use either of these roles when they have permissions required by your workflow execution. Deprecated.

Field	Type	Label	Description
assumable_iam_role	string		Defines an optional iam role which will be used for tasks run in executions created with this launch plan.
kubernetes_service_account	string		Defines an optional kubernetes service account which will be used for tasks run in executions created with this launch plan.

EmailNotification

Defines an email notification specification.

Field	Type	Label	Description
recipients_email	string	repeated	The list of email addresses recipients for this notification. +required
template	string		The template to use for this notification. +optional

Envs

Environment variable values to be applied to an execution resource. In the future a mode (e.g. OVERRIDE, APPEND, etc) can be defined to specify how to merge environment variables defined at registration and execution time.

Field	Type	Label	Description
values	flyteidl.core.KeyValuePair	repeated	Map of custom environment variables to be applied to the execution resource.

FlyteURLs

These URLs are returned as part of node and task execution data requests.

Field	Type	Label	Description
inputs	string
outputs	string
deck	string

Labels

Label values to be applied to an execution resource. In the future a mode (e.g. OVERRIDE, APPEND, etc) can be defined to specify how to merge labels defined at registration and execution time.

Field	Type	Label	Description
values	Labels.ValuesEntry	repeated	Map of custom labels to be applied to the execution resource.

Labels.ValuesEntry

Field	Type	Label	Description
key	string
value	string

NamedEntity

Encapsulates information common to a NamedEntity, a Flyte resource such as a task, workflow or launch plan. A NamedEntity is exclusively identified by its resource type and identifier.

Field	Type	Label	Description
resource_type	flyteidl.core.ResourceType		Resource type of the named entity. One of Task, Workflow or LaunchPlan.
id	NamedEntityIdentifier
metadata	NamedEntityMetadata		Additional metadata around a named entity.

NamedEntityGetRequest

A request to retrieve the metadata associated with a NamedEntityIdentifier

Field	Type	Label	Description
resource_type	flyteidl.core.ResourceType		Resource type of the metadata to get. One of Task, Workflow or LaunchPlan. +required
id	NamedEntityIdentifier		The identifier for the named entity for which to fetch metadata. +required

NamedEntityIdentifier

Encapsulation of fields that identifies a Flyte resource. A Flyte resource can be a task, workflow or launch plan. A resource can internally have multiple versions and is uniquely identified by project, domain, and name.

Field	Type	Label	Description
project	string		Name of the project the resource belongs to.
domain	string		Name of the domain the resource belongs to. A domain can be considered as a subset within a specific project.
name	string		User provided value for the resource. The combination of project + domain + name uniquely identifies the resource. +optional - in certain contexts - like ‘List API’, ‘Launch plans’
org	string		Optional, org key applied to the resource.

NamedEntityIdentifierList

Represents a list of NamedEntityIdentifiers.

Field	Type	Label	Description
entities	NamedEntityIdentifier	repeated	A list of identifiers.
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

NamedEntityIdentifierListRequest

Represents a request structure to list NamedEntityIdentifiers.

Field	Type	Label	Description
project	string		Name of the project that contains the identifiers. +required
domain	string		Name of the domain the identifiers belongs to within the project. +required
limit	uint32		Indicates the number of resources to be returned. +required
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional
sort_by	Sort		Specifies how listed entities should be sorted in the response. +optional
filters	string		Indicates a list of filters passed as string. +optional
org	string		Optional, org key applied to the resource.

NamedEntityList

Represents a list of NamedEntityIdentifiers.

Field	Type	Label	Description
entities	NamedEntity	repeated	A list of NamedEntity objects
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. If there are no more results, this value will be empty.

NamedEntityListRequest

Represents a request structure to list NamedEntity objects

Field	Type	Label	Description
resource_type	flyteidl.core.ResourceType		Resource type of the metadata to query. One of Task, Workflow or LaunchPlan. +required
project	string		Name of the project that contains the identifiers. +required
domain	string		Name of the domain the identifiers belongs to within the project.
limit	uint32		Indicates the number of resources to be returned.
token	string		In the case of multiple pages of results, the server-provided token can be used to fetch the next page in a query. +optional
sort_by	Sort		Specifies how listed entities should be sorted in the response. +optional
filters	string		Indicates a list of filters passed as string. +optional
org	string		Optional, org key applied to the resource.

NamedEntityMetadata

Additional metadata around a named entity.

Field	Type	Label	Description
description	string		Common description across all versions of the entity +optional
state	NamedEntityState		Shared state across all version of the entity At this point in time, only workflow entities can have their state archived.

NamedEntityUpdateRequest

Request to set the referenced named entity state to the configured value.

Field	Type	Label	Description
resource_type	flyteidl.core.ResourceType		Resource type of the metadata to update +required
id	NamedEntityIdentifier		Identifier of the metadata to update +required
metadata	NamedEntityMetadata		Metadata object to set as the new value +required

NamedEntityUpdateResponse

Purposefully empty, may be populated in the future.

Notification

Represents a structure for notifications based on execution status. The notification content is configured within flyte admin but can be templatized. Future iterations could expose configuring notifications with custom content.

Field	Type	Label	Description
phases	flyteidl.core.WorkflowExecution.Phase	repeated	A list of phases to which users can associate the notifications to. +required
email	EmailNotification
pager_duty	PagerDutyNotification
slack	SlackNotification

ObjectGetRequest

Shared request structure to fetch a single resource. Resources include: Task, Workflow, LaunchPlan

Field	Type	Label	Description
id	flyteidl.core.Identifier		Indicates a unique version of resource. +required

PagerDutyNotification

Defines a pager duty notification specification.

Field	Type	Label	Description
recipients_email	string	repeated	Currently, PagerDuty notifications leverage email to trigger a notification. +required
template	string		The template to use for this notification. +optional

RawOutputDataConfig

Encapsulates user settings pertaining to offloaded data (i.e. Blobs, Schema, query data, etc.). See https://github.com/flyteorg/flyte/issues/211 for more background information.

Field	Type	Label	Description
output_location_prefix	string		Prefix for where offloaded data from user workflows will be written e.g. s3://bucket/key or s3://bucket/

ResourceListRequest

Shared request structure to retrieve a list of resources. Resources include: Task, Workflow, LaunchPlan

Field	Type	Label	Description
id	NamedEntityIdentifier		id represents the unique identifier of the resource. +required
limit	uint32		Indicates the number of resources to be returned. +required
token	string		In the case of multiple pages of results, this server-provided token can be used to fetch the next page in a query. +optional
filters	string		Indicates a list of filters passed as string. More info on constructing filters : <lt;Link>gt; +optional
sort_by	Sort		Sort ordering. +optional

SlackNotification

Defines a slack notification specification.

Field	Type	Label	Description
recipients_email	string	repeated	Currently, Slack notifications leverage email to trigger a notification. +required
template	string		The template to use for this notification. +optional

Sort

Specifies sort ordering in a list request.

Field	Type	Label	Description
key	string		Indicates an attribute to sort the response values. +required
direction	Sort.Direction		Indicates the direction to apply sort key for response values. +optional

UrlBlob

Represents a string url and associated metadata used throughout the platform.

Field	Type	Label	Description
url	string		Actual url value.
bytes	int64		Represents the size of the file accessible at the above url.

NamedEntityState

The status of the named entity is used to control its visibility in the UI.

Name	Number	Description
NAMED_ENTITY_ACTIVE	0	By default, all named entities are considered active and under development.
NAMED_ENTITY_ARCHIVED	1	Archived named entities are no longer visible in the UI.
SYSTEM_GENERATED	2	System generated entities that aren’t explicitly created or managed by a user.

Sort.Direction

Name	Number	Description
DESCENDING	0	By default, fields are sorted in descending order.
ASCENDING	1

flyteidl/cacheservice/cacheservice.proto

CachedOutput

Represents cached output, either as literals or an URI, with associated metadata.

Field	Type	Label	Description
output_literals	flyteidl.core.LiteralMap		Output literals
output_uri	string		URI to output data
metadata	Metadata		Associated metadata

DeleteCacheRequest

Request to delete cached data by key.

Field	Type	Label	Description
key	string		Cache key

DeleteCacheResponse

Response message of cache deletion operation.

Empty, success indicated by no errors

GetCacheRequest

Request to retrieve cached data by key.

Field	Type	Label	Description
key	string		Cache key

GetCacheResponse

Response with cached data for a given key.

Field	Type	Label	Description
output	CachedOutput		Cached output

GetOrExtendReservationRequest

Request to get or extend a reservation for a cache key

Field	Type	Label	Description
key	string		The unique ID for the reservation - same as the cache key
owner_id	string		The unique ID of the owner for the reservation
heartbeat_interval	google.protobuf.Duration		Requested reservation extension heartbeat interval

GetOrExtendReservationResponse

Request to get or extend a reservation for a cache key

Field	Type	Label	Description
reservation	Reservation		The reservation that was created or extended

KeyMapMetadata

Additional metadata as key-value pairs

Field	Type	Label	Description
values	KeyMapMetadata.ValuesEntry	repeated	Additional metadata as key-value pairs

KeyMapMetadata.ValuesEntry

Field	Type	Label	Description
key	string
value	string

Metadata

Metadata for cached outputs, including the source identifier and timestamps.

Field	Type	Label	Description
source_identifier	flyteidl.core.Identifier		Source task or workflow identifier
key_map	KeyMapMetadata		Additional metadata as key-value pairs
created_at	google.protobuf.Timestamp		Creation timestamp
last_updated_at	google.protobuf.Timestamp		Last update timestamp

PutCacheRequest

Request to store/update cached data by key.

Field	Type	Label	Description
key	string		Cache key
output	CachedOutput		Output to cache
overwrite	bool		Overwrite flag

PutCacheResponse

Response message of cache store/update operation.

Empty, success indicated by no errors

ReleaseReservationRequest

Request to release the reservation for a cache key

Field	Type	Label	Description
key	string		The unique ID for the reservation - same as the cache key
owner_id	string		The unique ID of the owner for the reservation

ReleaseReservationResponse

Response message of release reservation operation.

Empty, success indicated by no errors

Reservation

A reservation including owner, heartbeat interval, expiration timestamp, and various metadata.

Field	Type	Label	Description
key	string		The unique ID for the reservation - same as the cache key
owner_id	string		The unique ID of the owner for the reservation
heartbeat_interval	google.protobuf.Duration		Requested reservation extension heartbeat interval
expires_at	google.protobuf.Timestamp		Expiration timestamp of this reservation

CacheService

CacheService defines operations for cache management including retrieval, storage, and deletion of cached task/workflow outputs.

Method Name	Request Type	Response Type	Description
Get	GetCacheRequest	GetCacheResponse	Retrieves cached data by key.
Put	PutCacheRequest	PutCacheResponse	Stores or updates cached data by key.
Delete	DeleteCacheRequest	DeleteCacheResponse	Deletes cached data by key.
GetOrExtendReservation	GetOrExtendReservationRequest	GetOrExtendReservationResponse	Get or extend a reservation for a cache key
ReleaseReservation	ReleaseReservationRequest	ReleaseReservationResponse	Release the reservation for a cache key

flyteidl/service/signal.proto

SignalService

SignalService defines an RPC Service that may create, update, and retrieve signal(s).

Method Name	Request Type	Response Type	Description
GetOrCreateSignal	.flyteidl.admin.SignalGetOrCreateRequest	.flyteidl.admin.Signal	Fetches or creates a :ref:ref_flyteidl.admin.Signal.
ListSignals	.flyteidl.admin.SignalListRequest	.flyteidl.admin.SignalList	Fetch a list of :ref:ref_flyteidl.admin.Signal definitions.
SetSignal	.flyteidl.admin.SignalSetRequest	.flyteidl.admin.SignalSetResponse	Sets the value on a :ref:ref_flyteidl.admin.Signal definition

flyteidl/service/external_plugin_service.proto

TaskCreateRequest

Represents a request structure to create task.

Field	Type	Label	Description
inputs	flyteidl.core.LiteralMap		The inputs required to start the execution. All required inputs must be included in this map. If not required and not provided, defaults apply. +optional
template	flyteidl.core.TaskTemplate		Template of the task that encapsulates all the metadata of the task.
output_prefix	string		Prefix for where task output data will be written. (e.g. s3://my-bucket/randomstring)

TaskCreateResponse

Represents a create response structure.

Field	Type	Label	Description
job_id	string

TaskDeleteRequest

A message used to delete a task.

Field	Type	Label	Description
task_type	string		A predefined yet extensible Task type identifier.
job_id	string		The unique id identifying the job.

TaskDeleteResponse

Response to delete a task.

TaskGetRequest

A message used to fetch a job state from backend plugin server.

Field	Type	Label	Description
task_type	string		A predefined yet extensible Task type identifier.
job_id	string		The unique id identifying the job.

TaskGetResponse

Response to get an individual task state.

Field	Type	Label	Description
state	State		The state of the execution is used to control its visibility in the UI/CLI.
outputs	flyteidl.core.LiteralMap		The outputs of the execution. It’s typically used by sql task. Flyteplugins service will create a Structured dataset pointing to the query result table. +optional

State

The state of the execution is used to control its visibility in the UI/CLI.

Name	Number	Description
RETRYABLE_FAILURE	0
PERMANENT_FAILURE	1
PENDING	2
RUNNING	3
SUCCEEDED	4

ExternalPluginService

ExternalPluginService defines an RPC Service that allows propeller to send the request to the backend plugin server.

Method Name	Request Type	Response Type	Description
CreateTask	TaskCreateRequest	TaskCreateResponse	Send a task create request to the backend plugin server.
GetTask	TaskGetRequest	TaskGetResponse	Get job status.
DeleteTask	TaskDeleteRequest	TaskDeleteResponse	Delete the task resource.

flyteidl/service/dataproxy.proto

CreateDownloadLinkRequest

CreateDownloadLinkRequest defines the request parameters to create a download link (signed url)

Field	Type	Label	Description
artifact_type	ArtifactType		ArtifactType of the artifact requested.
expires_in	google.protobuf.Duration		ExpiresIn defines a requested expiration duration for the generated url. The request will be rejected if this exceeds the platform allowed max. +optional. The default value comes from a global config.
node_execution_id	flyteidl.core.NodeExecutionIdentifier		NodeId is the unique identifier for the node execution. For a task node, this will retrieve the output of the most recent attempt of the task.

CreateDownloadLinkResponse

CreateDownloadLinkResponse defines the response for the generated links

Field	Type	Label	Description
signed_url	string	repeated	Deprecated. SignedUrl specifies the url to use to download content from (e.g. https://my-bucket.s3.amazonaws.com/randomstring/suffix.tar?X-…)
expires_at	google.protobuf.Timestamp		Deprecated. ExpiresAt defines when will the signed URL expire.
pre_signed_urls	PreSignedURLs		New wrapper object containing the signed urls and expiration time

CreateDownloadLocationRequest

CreateDownloadLocationRequest specified request for the CreateDownloadLocation API.

Field	Type	Label	Description
native_url	string		NativeUrl specifies the url in the format of the configured storage provider (e.g. s3://my-bucket/randomstring/suffix.tar)
expires_in	google.protobuf.Duration		ExpiresIn defines a requested expiration duration for the generated url. The request will be rejected if this exceeds the platform allowed max. +optional. The default value comes from a global config.

CreateDownloadLocationResponse

Field	Type	Label	Description
signed_url	string		SignedUrl specifies the url to use to download content from (e.g. https://my-bucket.s3.amazonaws.com/randomstring/suffix.tar?X-…)
expires_at	google.protobuf.Timestamp		ExpiresAt defines when will the signed URL expires.

CreateUploadLocationRequest

CreateUploadLocationRequest specified request for the CreateUploadLocation API. The implementation in data proxy service will create the s3 location with some server side configured prefixes, and then:

• project/domain/(a deterministic str representation of the content_md5)/filename (if present); OR
• project/domain/filename_root (if present)/filename (if present).

Field	Type	Label	Description
project	string		Project to create the upload location for +required
domain	string		Domain to create the upload location for. +required
filename	string		Filename specifies a desired suffix for the generated location. E.g. file.py or pre/fix/file.zip. +optional. By default, the service will generate a consistent name based on the provided parameters.
expires_in	google.protobuf.Duration		ExpiresIn defines a requested expiration duration for the generated url. The request will be rejected if this exceeds the platform allowed max. +optional. The default value comes from a global config.
content_md5	bytes		ContentMD5 restricts the upload location to the specific MD5 provided. The ContentMD5 will also appear in the generated path. +required
filename_root	string		If present, data proxy will use this string in lieu of the md5 hash in the path. When the filename is also included this makes the upload location deterministic. The native url will still be prefixed by the upload location prefix in data proxy config. This option is useful when uploading multiple files. +optional
add_content_md5_metadata	bool		If true, the data proxy will add content_md5 to the metadata to the signed URL and it will force clients to add this metadata to the object. This make sure dataproxy is backward compatible with the old flytekit.
org	string		Optional, org key applied to the resource.

CreateUploadLocationResponse

Field	Type	Label	Description
signed_url	string		SignedUrl specifies the url to use to upload content to (e.g. https://my-bucket.s3.amazonaws.com/randomstring/suffix.tar?X-…)
native_url	string		NativeUrl specifies the url in the format of the configured storage provider (e.g. s3://my-bucket/randomstring/suffix.tar)
expires_at	google.protobuf.Timestamp		ExpiresAt defines when will the signed URL expires.
headers	CreateUploadLocationResponse.HeadersEntry	repeated	Data proxy generates these headers for client, and they have to add these headers to the request when uploading the file.

CreateUploadLocationResponse.HeadersEntry

Field	Type	Label	Description
key	string
value	string

GetDataRequest

General request artifact to retrieve data from a Flyte artifact url.

Field	Type	Label	Description
flyte_url	string		A unique identifier in the form of flyte://<lt;something>gt; that uniquely, for a given Flyte backend, identifies a Flyte artifact ([i]nput, [o]output, flyte [d]eck, etc.). e.g. flyte://v1/proj/development/execid/n2/0/i (for 0th task execution attempt input) flyte://v1/proj/development/execid/n2/i (for node execution input) flyte://v1/proj/development/execid/n2/o/o3 (the o3 output of the second node)

GetDataResponse

Field	Type	Label	Description
literal_map	flyteidl.core.LiteralMap		literal map data will be returned
pre_signed_urls	PreSignedURLs		Flyte deck html will be returned as a signed url users can download
literal	flyteidl.core.Literal		Single literal will be returned. This is returned when the user/url requests a specific output or input by name. See the o3 example above.

PreSignedURLs

Wrapper object since the message is shared across this and the GetDataResponse

Field	Type	Label	Description
signed_url	string	repeated	SignedUrl specifies the url to use to download content from (e.g. https://my-bucket.s3.amazonaws.com/randomstring/suffix.tar?X-…)
expires_at	google.protobuf.Timestamp		ExpiresAt defines when will the signed URL expire.

ArtifactType

ArtifactType

Name	Number	Description
ARTIFACT_TYPE_UNDEFINED	0	ARTIFACT_TYPE_UNDEFINED is the default, often invalid, value for the enum.
ARTIFACT_TYPE_DECK	1	ARTIFACT_TYPE_DECK refers to the deck html file optionally generated after a task, a workflow or a launch plan finishes executing.

DataProxyService

DataProxyService defines an RPC Service that allows access to user-data in a controlled manner.

Method Name	Request Type	Response Type	Description
CreateUploadLocation	CreateUploadLocationRequest	CreateUploadLocationResponse	CreateUploadLocation creates a signed url to upload artifacts to for a given project/domain.
CreateDownloadLocation	CreateDownloadLocationRequest	CreateDownloadLocationResponse	CreateDownloadLocation creates a signed url to download artifacts.
CreateDownloadLink	CreateDownloadLinkRequest	CreateDownloadLinkResponse	CreateDownloadLocation creates a signed url to download artifacts.
GetData	GetDataRequest	GetDataResponse

flyteidl/service/identity.proto

UserInfoRequest

UserInfoResponse

See the OpenID Connect spec at https://openid.net/specs/openid-connect-core-1_0.html#UserInfoResponse for more information.

Field	Type	Label	Description
subject	string		Locally unique and never reassigned identifier within the Issuer for the End-User, which is intended to be consumed by the Client.
name	string		Full name
preferred_username	string		Shorthand name by which the End-User wishes to be referred to
given_name	string		Given name(s) or first name(s)
family_name	string		Surname(s) or last name(s)
email	string		Preferred e-mail address
picture	string		Profile picture URL
additional_claims	google.protobuf.Struct		Additional claims

IdentityService

IdentityService defines an RPC Service that interacts with user/app identities.

Method Name	Request Type	Response Type	Description
UserInfo	UserInfoRequest	UserInfoResponse	Retrieves user information about the currently logged in user.

flyteidl/service/auth.proto

OAuth2MetadataRequest

OAuth2MetadataResponse

OAuth2MetadataResponse defines an RFC-Compliant response for /.well-known/oauth-authorization-server metadata as defined in https://tools.ietf.org/html/rfc8414

Field	Type	Label	Description
issuer	string		Defines the issuer string in all JWT tokens this server issues. The issuer can be admin itself or an external issuer.
authorization_endpoint	string		URL of the authorization server’s authorization endpoint [RFC6749]. This is REQUIRED unless no grant types are supported that use the authorization endpoint.
token_endpoint	string		URL of the authorization server’s token endpoint [RFC6749].
response_types_supported	string	repeated	Array containing a list of the OAuth 2.0 response_type values that this authorization server supports.
scopes_supported	string	repeated	JSON array containing a list of the OAuth 2.0 [RFC6749] scope values that this authorization server supports.
token_endpoint_auth_methods_supported	string	repeated	JSON array containing a list of client authentication methods supported by this token endpoint.
jwks_uri	string		URL of the authorization server’s JWK Set [JWK] document. The referenced document contains the signing key(s) the client uses to validate signatures from the authorization server.
code_challenge_methods_supported	string	repeated	JSON array containing a list of Proof Key for Code Exchange (PKCE) [RFC7636] code challenge methods supported by this authorization server.
grant_types_supported	string	repeated	JSON array containing a list of the OAuth 2.0 grant type values that this authorization server supports.
device_authorization_endpoint	string		URL of the authorization server’s device authorization endpoint, as defined in Section 3.1 of [RFC8628]

PublicClientAuthConfigRequest

PublicClientAuthConfigResponse

FlyteClientResponse encapsulates public information that flyte clients (CLIs… etc.) can use to authenticate users.

Field	Type	Label	Description
client_id	string		client_id to use when initiating OAuth2 authorization requests.
redirect_uri	string		redirect uri to use when initiating OAuth2 authorization requests.
scopes	string	repeated	scopes to request when initiating OAuth2 authorization requests.
authorization_metadata_key	string		Authorization Header to use when passing Access Tokens to the server. If not provided, the client should use the default http Authorization header.
service_http_endpoint	string		ServiceHttpEndpoint points to the http endpoint for the backend. If empty, clients can assume the endpoint used to configure the gRPC connection can be used for the http one respecting the insecure flag to choose between SSL or no SSL connections.
audience	string		audience to use when initiating OAuth2 authorization requests.

AuthMetadataService

The following defines an RPC service that is also served over HTTP via grpc-gateway. Standard response codes for both are defined here: https://github.com/grpc-ecosystem/grpc-gateway/blob/master/runtime/errors.go RPCs defined in this service must be anonymously accessible.

Method Name	Request Type	Response Type	Description
GetOAuth2Metadata	OAuth2MetadataRequest	OAuth2MetadataResponse	Anonymously accessible. Retrieves local or external oauth authorization server metadata.
GetPublicClientConfig	PublicClientAuthConfigRequest	PublicClientAuthConfigResponse	Anonymously accessible. Retrieves the client information clients should use when initiating OAuth2 authorization requests.

flyteidl/service/agent.proto

AgentMetadataService

AgentMetadataService defines an RPC service that is also served over HTTP via grpc-gateway. This service allows propeller or users to get the metadata of agents.

Method Name	Request Type	Response Type	Description
GetAgent	.flyteidl.admin.GetAgentRequest	.flyteidl.admin.GetAgentResponse	Fetch a :ref:ref_flyteidl.admin.Agent definition.
ListAgents	.flyteidl.admin.ListAgentsRequest	.flyteidl.admin.ListAgentsResponse	Fetch a list of :ref:ref_flyteidl.admin.Agent definitions.

AsyncAgentService

AsyncAgentService defines an RPC Service that allows propeller to send the request to the agent server asynchronously.

Method Name	Request Type	Response Type	Description
CreateTask	.flyteidl.admin.CreateTaskRequest	.flyteidl.admin.CreateTaskResponse	CreateTask sends a task create request to the agent service.
GetTask	.flyteidl.admin.GetTaskRequest	.flyteidl.admin.GetTaskResponse	Get job status.
DeleteTask	.flyteidl.admin.DeleteTaskRequest	.flyteidl.admin.DeleteTaskResponse	Delete the task resource.
GetTaskMetrics	.flyteidl.admin.GetTaskMetricsRequest	.flyteidl.admin.GetTaskMetricsResponse	GetTaskMetrics returns one or more task execution metrics, if available.

Errors include * OutOfRange if metrics are not available for the specified task time range * various other errors | | GetTaskLogs | .flyteidl.admin.GetTaskLogsRequest | .flyteidl.admin.GetTaskLogsResponse stream | GetTaskLogs returns task execution logs, if available. |

SyncAgentService

SyncAgentService defines an RPC Service that allows propeller to send the request to the agent server synchronously.

Method Name	Request Type	Response Type	Description
ExecuteTaskSync	.flyteidl.admin.ExecuteTaskSyncRequest stream	.flyteidl.admin.ExecuteTaskSyncResponse stream	ExecuteTaskSync streams the create request and inputs to the agent service and streams the outputs back.

flyteidl/service/admin.proto

AdminService

The following defines an RPC service that is also served over HTTP via grpc-gateway. Standard response codes for both are defined here: https://github.com/grpc-ecosystem/grpc-gateway/blob/master/runtime/errors.go

Method Name	Request Type	Response Type	Description
CreateTask	.flyteidl.admin.TaskCreateRequest	.flyteidl.admin.TaskCreateResponse	Create and upload a :ref:ref_flyteidl.admin.Task definition
GetTask	.flyteidl.admin.ObjectGetRequest	.flyteidl.admin.Task	Fetch a :ref:ref_flyteidl.admin.Task definition.
ListTaskIds	.flyteidl.admin.NamedEntityIdentifierListRequest	.flyteidl.admin.NamedEntityIdentifierList	Fetch a list of :ref:ref_flyteidl.admin.NamedEntityIdentifier of task objects.
ListTasks	.flyteidl.admin.ResourceListRequest	.flyteidl.admin.TaskList	Fetch a list of :ref:ref_flyteidl.admin.Task definitions.
CreateWorkflow	.flyteidl.admin.WorkflowCreateRequest	.flyteidl.admin.WorkflowCreateResponse	Create and upload a :ref:ref_flyteidl.admin.Workflow definition
GetWorkflow	.flyteidl.admin.ObjectGetRequest	.flyteidl.admin.Workflow	Fetch a :ref:ref_flyteidl.admin.Workflow definition.
ListWorkflowIds	.flyteidl.admin.NamedEntityIdentifierListRequest	.flyteidl.admin.NamedEntityIdentifierList	Fetch a list of :ref:ref_flyteidl.admin.NamedEntityIdentifier of workflow objects.
ListWorkflows	.flyteidl.admin.ResourceListRequest	.flyteidl.admin.WorkflowList	Fetch a list of :ref:ref_flyteidl.admin.Workflow definitions.
CreateLaunchPlan	.flyteidl.admin.LaunchPlanCreateRequest	.flyteidl.admin.LaunchPlanCreateResponse	Create and upload a :ref:ref_flyteidl.admin.LaunchPlan definition
GetLaunchPlan	.flyteidl.admin.ObjectGetRequest	.flyteidl.admin.LaunchPlan	Fetch a :ref:ref_flyteidl.admin.LaunchPlan definition.
GetActiveLaunchPlan	.flyteidl.admin.ActiveLaunchPlanRequest	.flyteidl.admin.LaunchPlan	Fetch the active version of a :ref:ref_flyteidl.admin.LaunchPlan.
ListActiveLaunchPlans	.flyteidl.admin.ActiveLaunchPlanListRequest	.flyteidl.admin.LaunchPlanList	List active versions of :ref:ref_flyteidl.admin.LaunchPlan.
ListLaunchPlanIds	.flyteidl.admin.NamedEntityIdentifierListRequest	.flyteidl.admin.NamedEntityIdentifierList	Fetch a list of :ref:ref_flyteidl.admin.NamedEntityIdentifier of launch plan objects.
ListLaunchPlans	.flyteidl.admin.ResourceListRequest	.flyteidl.admin.LaunchPlanList	Fetch a list of :ref:ref_flyteidl.admin.LaunchPlan definitions.
UpdateLaunchPlan	.flyteidl.admin.LaunchPlanUpdateRequest	.flyteidl.admin.LaunchPlanUpdateResponse	Updates the status of a registered :ref:ref_flyteidl.admin.LaunchPlan.
CreateExecution	.flyteidl.admin.ExecutionCreateRequest	.flyteidl.admin.ExecutionCreateResponse	Triggers the creation of a :ref:ref_flyteidl.admin.Execution
RelaunchExecution	.flyteidl.admin.ExecutionRelaunchRequest	.flyteidl.admin.ExecutionCreateResponse	Triggers the creation of an identical :ref:ref_flyteidl.admin.Execution
RecoverExecution	.flyteidl.admin.ExecutionRecoverRequest	.flyteidl.admin.ExecutionCreateResponse	Recreates a previously-run workflow execution that will only start executing from the last known failure point. In Recover mode, users cannot change any input parameters or update the version of the execution. This is extremely useful to recover from system errors and byzantine faults like - Loss of K8s cluster, bugs in platform or instability, machine failures, downstream system failures (downstream services), or simply to recover executions that failed because of retry exhaustion and should complete if tried again. See :ref:ref_flyteidl.admin.ExecutionRecoverRequest for more details.
GetExecution	.flyteidl.admin.WorkflowExecutionGetRequest	.flyteidl.admin.Execution	Fetches a :ref:ref_flyteidl.admin.Execution.
UpdateExecution	.flyteidl.admin.ExecutionUpdateRequest	.flyteidl.admin.ExecutionUpdateResponse	Update execution belonging to project domain :ref:ref_flyteidl.admin.Execution.
GetExecutionData	.flyteidl.admin.WorkflowExecutionGetDataRequest	.flyteidl.admin.WorkflowExecutionGetDataResponse	Fetches input and output data for a :ref:ref_flyteidl.admin.Execution.
ListExecutions	.flyteidl.admin.ResourceListRequest	.flyteidl.admin.ExecutionList	Fetch a list of :ref:ref_flyteidl.admin.Execution.
TerminateExecution	.flyteidl.admin.ExecutionTerminateRequest	.flyteidl.admin.ExecutionTerminateResponse	Terminates an in-progress :ref:ref_flyteidl.admin.Execution.
GetNodeExecution	.flyteidl.admin.NodeExecutionGetRequest	.flyteidl.admin.NodeExecution	Fetches a :ref:ref_flyteidl.admin.NodeExecution.
GetDynamicNodeWorkflow	.flyteidl.admin.GetDynamicNodeWorkflowRequest	.flyteidl.admin.DynamicNodeWorkflowResponse	Fetches a :ref:ref_flyteidl.admin.DynamicNodeWorkflowResponse.
ListNodeExecutions	.flyteidl.admin.NodeExecutionListRequest	.flyteidl.admin.NodeExecutionList	Fetch a list of :ref:ref_flyteidl.admin.NodeExecution.
ListNodeExecutionsForTask	.flyteidl.admin.NodeExecutionForTaskListRequest	.flyteidl.admin.NodeExecutionList	Fetch a list of :ref:ref_flyteidl.admin.NodeExecution launched by the reference :ref:ref_flyteidl.admin.TaskExecution.
GetNodeExecutionData	.flyteidl.admin.NodeExecutionGetDataRequest	.flyteidl.admin.NodeExecutionGetDataResponse	Fetches input and output data for a :ref:ref_flyteidl.admin.NodeExecution.
RegisterProject	.flyteidl.admin.ProjectRegisterRequest	.flyteidl.admin.ProjectRegisterResponse	Registers a :ref:ref_flyteidl.admin.Project with the Flyte deployment.
UpdateProject	.flyteidl.admin.Project	.flyteidl.admin.ProjectUpdateResponse	Updates an existing :ref:ref_flyteidl.admin.Project flyteidl.admin.Project should be passed but the domains property should be empty; it will be ignored in the handler as domains cannot be updated via this API.
GetProject	.flyteidl.admin.ProjectGetRequest	.flyteidl.admin.Project	Fetches a :ref:ref_flyteidl.admin.Project
ListProjects	.flyteidl.admin.ProjectListRequest	.flyteidl.admin.Projects	Fetches a list of :ref:ref_flyteidl.admin.Project
GetDomains	.flyteidl.admin.GetDomainRequest	.flyteidl.admin.GetDomainsResponse
CreateWorkflowEvent	.flyteidl.admin.WorkflowExecutionEventRequest	.flyteidl.admin.WorkflowExecutionEventResponse	Indicates a :ref:ref_flyteidl.event.WorkflowExecutionEvent has occurred.
CreateNodeEvent	.flyteidl.admin.NodeExecutionEventRequest	.flyteidl.admin.NodeExecutionEventResponse	Indicates a :ref:ref_flyteidl.event.NodeExecutionEvent has occurred.
CreateTaskEvent	.flyteidl.admin.TaskExecutionEventRequest	.flyteidl.admin.TaskExecutionEventResponse	Indicates a :ref:ref_flyteidl.event.TaskExecutionEvent has occurred.
GetTaskExecution	.flyteidl.admin.TaskExecutionGetRequest	.flyteidl.admin.TaskExecution	Fetches a :ref:ref_flyteidl.admin.TaskExecution.
ListTaskExecutions	.flyteidl.admin.TaskExecutionListRequest	.flyteidl.admin.TaskExecutionList	Fetches a list of :ref:ref_flyteidl.admin.TaskExecution.
GetTaskExecutionData	.flyteidl.admin.TaskExecutionGetDataRequest	.flyteidl.admin.TaskExecutionGetDataResponse	Fetches input and output data for a :ref:ref_flyteidl.admin.TaskExecution.
UpdateProjectDomainAttributes	.flyteidl.admin.ProjectDomainAttributesUpdateRequest	.flyteidl.admin.ProjectDomainAttributesUpdateResponse	Creates or updates custom :ref:ref_flyteidl.admin.MatchableAttributesConfiguration for a project and domain.
GetProjectDomainAttributes	.flyteidl.admin.ProjectDomainAttributesGetRequest	.flyteidl.admin.ProjectDomainAttributesGetResponse	Fetches custom :ref:ref_flyteidl.admin.MatchableAttributesConfiguration for a project and domain.
DeleteProjectDomainAttributes	.flyteidl.admin.ProjectDomainAttributesDeleteRequest	.flyteidl.admin.ProjectDomainAttributesDeleteResponse	Deletes custom :ref:ref_flyteidl.admin.MatchableAttributesConfiguration for a project and domain.
UpdateProjectAttributes	.flyteidl.admin.ProjectAttributesUpdateRequest	.flyteidl.admin.ProjectAttributesUpdateResponse	Creates or updates custom :ref:ref_flyteidl.admin.MatchableAttributesConfiguration at the project level
GetProjectAttributes	.flyteidl.admin.ProjectAttributesGetRequest	.flyteidl.admin.ProjectAttributesGetResponse	Fetches custom :ref:ref_flyteidl.admin.MatchableAttributesConfiguration for a project and domain.
DeleteProjectAttributes	.flyteidl.admin.ProjectAttributesDeleteRequest	.flyteidl.admin.ProjectAttributesDeleteResponse	Deletes custom :ref:ref_flyteidl.admin.MatchableAttributesConfiguration for a project and domain.
UpdateWorkflowAttributes	.flyteidl.admin.WorkflowAttributesUpdateRequest	.flyteidl.admin.WorkflowAttributesUpdateResponse	Creates or updates custom :ref:ref_flyteidl.admin.MatchableAttributesConfiguration for a project, domain and workflow.
GetWorkflowAttributes	.flyteidl.admin.WorkflowAttributesGetRequest	.flyteidl.admin.WorkflowAttributesGetResponse	Fetches custom :ref:ref_flyteidl.admin.MatchableAttributesConfiguration for a project, domain and workflow.
DeleteWorkflowAttributes	.flyteidl.admin.WorkflowAttributesDeleteRequest	.flyteidl.admin.WorkflowAttributesDeleteResponse	Deletes custom :ref:ref_flyteidl.admin.MatchableAttributesConfiguration for a project, domain and workflow.
ListMatchableAttributes	.flyteidl.admin.ListMatchableAttributesRequest	.flyteidl.admin.ListMatchableAttributesResponse	Lists custom :ref:ref_flyteidl.admin.MatchableAttributesConfiguration for a specific resource type.
ListNamedEntities	.flyteidl.admin.NamedEntityListRequest	.flyteidl.admin.NamedEntityList	Returns a list of :ref:ref_flyteidl.admin.NamedEntity objects.
GetNamedEntity	.flyteidl.admin.NamedEntityGetRequest	.flyteidl.admin.NamedEntity	Returns a :ref:ref_flyteidl.admin.NamedEntity object.
UpdateNamedEntity	.flyteidl.admin.NamedEntityUpdateRequest	.flyteidl.admin.NamedEntityUpdateResponse	Updates a :ref:ref_flyteidl.admin.NamedEntity object.
GetVersion	.flyteidl.admin.GetVersionRequest	.flyteidl.admin.GetVersionResponse
GetDescriptionEntity	.flyteidl.admin.ObjectGetRequest	.flyteidl.admin.DescriptionEntity	Fetch a :ref:ref_flyteidl.admin.DescriptionEntity object.
ListDescriptionEntities	.flyteidl.admin.DescriptionEntityListRequest	.flyteidl.admin.DescriptionEntityList	Fetch a list of :ref:ref_flyteidl.admin.DescriptionEntity definitions.
GetExecutionMetrics	.flyteidl.admin.WorkflowExecutionGetMetricsRequest	.flyteidl.admin.WorkflowExecutionGetMetricsResponse	Fetches runtime metrics for a :ref:ref_flyteidl.admin.Execution.

flyteidl/event/cloudevents.proto

CloudEventExecutionStart

This event is to be sent by Admin after it creates an execution.

Field	Type	Label	Description
execution_id	flyteidl.core.WorkflowExecutionIdentifier		The execution created.
launch_plan_id	flyteidl.core.Identifier		The launch plan used.
workflow_id	flyteidl.core.Identifier
artifact_ids	flyteidl.core.ArtifactID	repeated	Artifact inputs to the workflow execution for which we have the full Artifact ID. These are likely the result of artifact queries that are run.
artifact_trackers	string	repeated	Artifact inputs to the workflow execution for which we only have the tracking bit that’s installed into the Literal’s metadata by the Artifact service.
principal	string

CloudEventNodeExecution

Field	Type	Label	Description
raw_event	NodeExecutionEvent
task_exec_id	flyteidl.core.TaskExecutionIdentifier		The relevant task execution if applicable
output_interface	flyteidl.core.TypedInterface		The typed interface for the task that produced the event.
artifact_ids	flyteidl.core.ArtifactID	repeated	The following are ExecutionMetadata fields We can’t have the ExecutionMetadata object directly because of import cycle
principal	string
launch_plan_id	flyteidl.core.Identifier		The ID of the LP that generated the execution that generated the Artifact. Here for provenance information. Launch plan IDs are easier to get than workflow IDs so we’ll use these for now.
labels	CloudEventNodeExecution.LabelsEntry	repeated	We can’t have the ExecutionMetadata object directly because of import cycle

CloudEventNodeExecution.LabelsEntry

Field	Type	Label	Description
key	string
value	string

CloudEventTaskExecution

Field	Type	Label	Description
raw_event	TaskExecutionEvent
labels	CloudEventTaskExecution.LabelsEntry	repeated	We can’t have the ExecutionMetadata object directly because of import cycle

CloudEventTaskExecution.LabelsEntry

Field	Type	Label	Description
key	string
value	string

CloudEventWorkflowExecution

This is the cloud event parallel to the raw WorkflowExecutionEvent message. It’s filled in with additional information that downstream consumers may find useful.

Field	Type	Label	Description
raw_event	WorkflowExecutionEvent
output_interface	flyteidl.core.TypedInterface
artifact_ids	flyteidl.core.ArtifactID	repeated	The following are ExecutionMetadata fields We can’t have the ExecutionMetadata object directly because of import cycle
reference_execution	flyteidl.core.WorkflowExecutionIdentifier
principal	string
launch_plan_id	flyteidl.core.Identifier		The ID of the LP that generated the execution that generated the Artifact. Here for provenance information. Launch plan IDs are easier to get than workflow IDs so we’ll use these for now.
labels	CloudEventWorkflowExecution.LabelsEntry	repeated	We can’t have the ExecutionMetadata object directly because of import cycle

CloudEventWorkflowExecution.LabelsEntry

Field	Type	Label	Description
key	string
value	string

flyteidl/event/event.proto

DynamicWorkflowNodeMetadata

For dynamic workflow nodes we send information about the dynamic workflow definition that gets generated.

Field	Type	Label	Description
id	flyteidl.core.Identifier		id represents the unique identifier of the workflow.
compiled_workflow	flyteidl.core.CompiledWorkflowClosure		Represents the compiled representation of the embedded dynamic workflow.
dynamic_job_spec_uri	string		dynamic_job_spec_uri is the location of the DynamicJobSpec proto message for this DynamicWorkflow. This is required to correctly recover partially completed executions where the workflow has already been compiled.

EventReason

Field	Type	Label	Description
reason	string		An explanation for this event
occurred_at	google.protobuf.Timestamp		The time this reason occurred

ExternalResourceInfo

This message contains metadata about external resources produced or used by a specific task execution.

Field	Type	Label	Description
external_id	string		Identifier for an external resource created by this task execution, for example Qubole query ID or presto query ids.
index	uint32		A unique index for the external resource with respect to all external resources for this task. Although the identifier may change between task reporting events or retries, this will remain the same to enable aggregating information from multiple reports.
retry_attempt	uint32		Retry attempt number for this external resource, ie., 2 for the second attempt
phase	flyteidl.core.TaskExecution.Phase		Phase associated with the external resource
cache_status	flyteidl.core.CatalogCacheStatus		Captures the status of caching for this external resource execution.
logs	flyteidl.core.TaskLog	repeated	log information for the external resource execution
workflow_node_metadata	WorkflowNodeMetadata
custom_info	google.protobuf.Struct		Extensible field for custom, plugin-specific info

NodeExecutionEvent

Field	Type	Label	Description
id	flyteidl.core.NodeExecutionIdentifier		Unique identifier for this node execution
producer_id	string		the id of the originator (Propeller) of the event
phase	flyteidl.core.NodeExecution.Phase
occurred_at	google.protobuf.Timestamp		This timestamp represents when the original event occurred, it is generated by the executor of the node.
input_uri	string
input_data	flyteidl.core.LiteralMap		Raw input data consumed by this node execution.
output_uri	string		URL to the output of the execution, it encodes all the information including Cloud source provider. ie., s3://…
error	flyteidl.core.ExecutionError		Error information for the execution
output_data	flyteidl.core.LiteralMap		Raw output data produced by this node execution.
workflow_node_metadata	WorkflowNodeMetadata
task_node_metadata	TaskNodeMetadata
parent_task_metadata	ParentTaskExecutionMetadata		[To be deprecated] Specifies which task (if any) launched this node.
parent_node_metadata	ParentNodeExecutionMetadata		Specifies the parent node of the current node execution. Node executions at level zero will not have a parent node.
retry_group	string		Retry group to indicate grouping of nodes by retries
spec_node_id	string		Identifier of the node in the original workflow/graph This maps to value of WorkflowTemplate.nodes[X].id
node_name	string		Friendly readable name for the node
event_version	int32
is_parent	bool		Whether this node launched a subworkflow.
is_dynamic	bool		Whether this node yielded a dynamic workflow.
deck_uri	string		String location uniquely identifying where the deck HTML file is NativeUrl specifies the url in the format of the configured storage provider (e.g. s3://my-bucket/randomstring/suffix.tar)
reported_at	google.protobuf.Timestamp		This timestamp represents the instant when the event was reported by the executing framework. For example, when first processing a node the occurred_at timestamp should be the instant propeller makes progress, so when literal inputs are initially copied. The event however will not be sent until after the copy completes. Extracting both of these timestamps facilitates a more accurate portrayal of the evaluation time-series.
is_array	bool		Indicates if this node is an ArrayNode.
target_entity	flyteidl.core.Identifier		So that Admin doesn’t have to rebuild the node execution graph to find the target entity, propeller will fill this in optionally - currently this is only filled in for subworkflows. This is the ID of the subworkflow corresponding to this node execution. It is difficult to find because Admin only sees one node at a time. A subworkflow could be nested multiple layers deep, and you’d need to access the correct workflow template to know the target subworkflow.
is_in_dynamic_chain	bool		Tasks and subworkflows (but not launch plans) that are run within a dynamic task are effectively independent of the tasks that are registered in Admin’s db. Confusingly, they are often identical, but sometimes they are not even registered at all. Similar to the target_entity field, at the time Admin receives this event, it has no idea if the relevant execution entity is was registered, or dynamic. This field indicates that the target_entity ID, as well as task IDs in any corresponding Task Executions, should not be used to looked up the task in Admin’s db.
is_eager	bool		Whether this node launched an eager task.

ParentNodeExecutionMetadata

Field	Type	Label	Description
node_id	string		Unique identifier of the parent node id within the execution This is value of core.NodeExecutionIdentifier.node_id of the parent node

ParentTaskExecutionMetadata

Field	Type	Label	Description
id	flyteidl.core.TaskExecutionIdentifier

ResourcePoolInfo

This message holds task execution metadata specific to resource allocation used to manage concurrent executions for a project namespace.

Field	Type	Label	Description
allocation_token	string		Unique resource ID used to identify this execution when allocating a token.
namespace	string		Namespace under which this task execution requested an allocation token.

TaskExecutionEvent

Plugin specific execution event information. For tasks like Python, Hive, Spark, DynamicJob.

Field	Type	Label	Description
task_id	flyteidl.core.Identifier		ID of the task. In combination with the retryAttempt this will indicate the task execution uniquely for a given parent node execution.
parent_node_execution_id	flyteidl.core.NodeExecutionIdentifier		A task execution is always kicked off by a node execution, the event consumer will use the parent_id to relate the task to it’s parent node execution
retry_attempt	uint32		retry attempt number for this task, ie., 2 for the second attempt
phase	flyteidl.core.TaskExecution.Phase		Phase associated with the event
producer_id	string		id of the process that sent this event, mainly for trace debugging
logs	flyteidl.core.TaskLog	repeated	log information for the task execution
occurred_at	google.protobuf.Timestamp		This timestamp represents when the original event occurred, it is generated by the executor of the task.
input_uri	string		URI of the input file, it encodes all the information including Cloud source provider. ie., s3://…
input_data	flyteidl.core.LiteralMap		Raw input data consumed by this task execution.
output_uri	string		URI to the output of the execution, it will be in a format that encodes all the information including Cloud source provider. ie., s3://…
error	flyteidl.core.ExecutionError		Error information for the execution
output_data	flyteidl.core.LiteralMap		Raw output data produced by this task execution.
custom_info	google.protobuf.Struct		Custom data that the task plugin sends back. This is extensible to allow various plugins in the system.
phase_version	uint32		Some phases, like RUNNING, can send multiple events with changed metadata (new logs, additional custom_info, etc) that should be recorded regardless of the lack of phase change. The version field should be incremented when metadata changes across the duration of an individual phase.
reason	string		Deprecated. An optional explanation for the phase transition. Deprecated: Use reasons instead.
reasons	EventReason	repeated	An optional list of explanations for the phase transition.
task_type	string		A predefined yet extensible Task type identifier. If the task definition is already registered in flyte admin this type will be identical, but not all task executions necessarily use pre-registered definitions and this type is useful to render the task in the UI, filter task executions, etc.
metadata	TaskExecutionMetadata		Metadata around how a task was executed.
event_version	int32		The event version is used to indicate versioned changes in how data is reported using this proto message. For example, event_verison >gt; 0 means that maps tasks report logs using the TaskExecutionMetadata ExternalResourceInfo fields for each subtask rather than the TaskLog in this message.
reported_at	google.protobuf.Timestamp		This timestamp represents the instant when the event was reported by the executing framework. For example, a k8s pod task may be marked completed at (ie. occurred_at) the instant the container running user code completes, but this event will not be reported until the pod is marked as completed. Extracting both of these timestamps facilitates a more accurate portrayal of the evaluation time-series.

TaskExecutionMetadata

Holds metadata around how a task was executed. As a task transitions across event phases during execution some attributes, such its generated name, generated external resources, and more may grow in size but not change necessarily based on the phase transition that sparked the event update. Metadata is a container for these attributes across the task execution lifecycle.

Field	Type	Label	Description
generated_name	string		Unique, generated name for this task execution used by the backend.
external_resources	ExternalResourceInfo	repeated	Additional data on external resources on other back-ends or platforms (e.g. Hive, Qubole, etc) launched by this task execution.
resource_pool_info	ResourcePoolInfo	repeated	Includes additional data on concurrent resource management used during execution.. This is a repeated field because a plugin can request multiple resource allocations during execution.
plugin_identifier	string		The identifier of the plugin used to execute this task.
instance_class	TaskExecutionMetadata.InstanceClass

TaskNodeMetadata

Field	Type	Label	Description
cache_status	flyteidl.core.CatalogCacheStatus		Captures the status of caching for this execution.
catalog_key	flyteidl.core.CatalogMetadata		This structure carries the catalog artifact information
reservation_status	flyteidl.core.CatalogReservation.Status		Captures the status of cache reservations for this execution.
checkpoint_uri	string		The latest checkpoint location
dynamic_workflow	DynamicWorkflowNodeMetadata		In the case this task launched a dynamic workflow we capture its structure here.

WorkflowExecutionEvent

Field	Type	Label	Description
execution_id	flyteidl.core.WorkflowExecutionIdentifier		Workflow execution id
producer_id	string		the id of the originator (Propeller) of the event
phase	flyteidl.core.WorkflowExecution.Phase
occurred_at	google.protobuf.Timestamp		This timestamp represents when the original event occurred, it is generated by the executor of the workflow.
output_uri	string		URL to the output of the execution, it encodes all the information including Cloud source provider. ie., s3://…
error	flyteidl.core.ExecutionError		Error information for the execution
output_data	flyteidl.core.LiteralMap		Raw output data produced by this workflow execution.

WorkflowNodeMetadata

For Workflow Nodes we need to send information about the workflow that’s launched

Field	Type	Label	Description
execution_id	flyteidl.core.WorkflowExecutionIdentifier

TaskExecutionMetadata.InstanceClass

Includes the broad category of machine used for this specific task execution.

Name	Number	Description
DEFAULT	0	The default instance class configured for the flyte application platform.
INTERRUPTIBLE	1	The instance class configured for interruptible tasks.

flyteidl/datacatalog/datacatalog.proto

AddTagRequest

Request message for tagging an Artifact.

Field	Type	Label	Description
tag	Tag

AddTagResponse

Response message for tagging an Artifact.

Artifact

Artifact message. It is composed of several string fields.

Field	Type	Label	Description
id	string		The unique ID of the artifact
dataset	DatasetID		The Dataset that the artifact belongs to
data	ArtifactData	repeated	A list of data that is associated with the artifact
metadata	Metadata		Free-form metadata associated with the artifact
partitions	Partition	repeated
tags	Tag	repeated
created_at	google.protobuf.Timestamp		creation timestamp of artifact, autogenerated by service

ArtifactData

ArtifactData that belongs to an artifact

Field	Type	Label	Description
name	string
value	flyteidl.core.Literal

ArtifactPropertyFilter

Artifact properties we can filter by

Field	Type	Label	Description
artifact_id	string

CreateArtifactRequest

Request message for creating an Artifact and its associated artifact Data.

Field	Type	Label	Description
artifact	Artifact

CreateArtifactResponse

Response message for creating an Artifact.

CreateDatasetRequest

Request message for creating a Dataset.

Field	Type	Label	Description
dataset	Dataset

CreateDatasetResponse

Response message for creating a Dataset

Dataset

Dataset message. It is uniquely identified by DatasetID.

Field	Type	Label	Description
id	DatasetID
metadata	Metadata
partitionKeys	string	repeated

DatasetID

DatasetID message that is composed of several string fields.

Field	Type	Label	Description
project	string		The name of the project
name	string		The name of the dataset
domain	string		The domain (eg. environment)
version	string		Version of the data schema
UUID	string		UUID for the dataset (if set the above fields are optional)
org	string		Optional, org key applied to the resource.

DatasetPropertyFilter

Dataset properties we can filter by

Field	Type	Label	Description
project	string
name	string
domain	string
version	string
org	string		Optional, org key applied to the dataset.

FilterExpression

Filter expression that is composed of a combination of single filters

Field	Type	Label	Description
filters	SinglePropertyFilter	repeated

GetArtifactRequest

Request message for retrieving an Artifact. Retrieve an artifact based on a query handle that can be one of artifact_id or tag. The result returned will include the artifact data and metadata associated with the artifact.

Field	Type	Label	Description
dataset	DatasetID
artifact_id	string
tag_name	string

GetArtifactResponse

Response message for retrieving an Artifact. The result returned will include the artifact data and metadata associated with the artifact.

Field	Type	Label	Description
artifact	Artifact

GetDatasetRequest

Request message for retrieving a Dataset. The Dataset is retrieved by it’s unique identifier which is a combination of several fields.

Field	Type	Label	Description
dataset	DatasetID

GetDatasetResponse

Response message for retrieving a Dataset. The response will include the metadata for the Dataset.

Field	Type	Label	Description
dataset	Dataset

GetOrExtendReservationRequest

Try to acquire or extend an artifact reservation. If an active reservation exists, retrieve that instance.

Field	Type	Label	Description
reservation_id	ReservationID		The unique ID for the reservation
owner_id	string		The unique ID of the owner for the reservation
heartbeat_interval	google.protobuf.Duration		Requested reservation extension heartbeat interval

GetOrExtendReservationResponse

Response including either a newly minted reservation or the existing reservation

Field	Type	Label	Description
reservation	Reservation		The reservation to be acquired or extended

KeyValuePair

Field	Type	Label	Description
key	string
value	string

ListArtifactsRequest

List the artifacts that belong to the Dataset, optionally filtered using filtered expression.

Field	Type	Label	Description
dataset	DatasetID		Use a datasetID for which you want to retrieve the artifacts
filter	FilterExpression		Apply the filter expression to this query
pagination	PaginationOptions		Pagination options to get a page of artifacts

ListArtifactsResponse

Response to list artifacts

Field	Type	Label	Description
artifacts	Artifact	repeated	The list of artifacts
next_token	string		Token to use to request the next page, pass this into the next requests PaginationOptions

ListDatasetsRequest

List the datasets for the given query

Field	Type	Label	Description
filter	FilterExpression		Apply the filter expression to this query
pagination	PaginationOptions		Pagination options to get a page of datasets

ListDatasetsResponse

List the datasets response with token for next pagination

Field	Type	Label	Description
datasets	Dataset	repeated	The list of datasets
next_token	string		Token to use to request the next page, pass this into the next requests PaginationOptions

Metadata

Metadata representation for artifacts and datasets

Field	Type	Label	Description
key_map	Metadata.KeyMapEntry	repeated	key map is a dictionary of key/val strings that represent metadata

Metadata.KeyMapEntry

Field	Type	Label	Description
key	string
value	string

PaginationOptions

Pagination options for making list requests

Field	Type	Label	Description
limit	uint32		the max number of results to return
token	string		the token to pass to fetch the next page
sortKey	PaginationOptions.SortKey		the property that we want to sort the results by
sortOrder	PaginationOptions.SortOrder		the sort order of the results

Partition

An artifact could have multiple partitions and each partition can have an arbitrary string key/value pair

Field	Type	Label	Description
key	string
value	string

PartitionPropertyFilter

Partition properties we can filter by

Field	Type	Label	Description
key_val	KeyValuePair

ReleaseReservationRequest

Request to release reservation

Field	Type	Label	Description
reservation_id	ReservationID		The unique ID for the reservation
owner_id	string		The unique ID of the owner for the reservation

ReleaseReservationResponse

Response to release reservation

Reservation

A reservation including owner, heartbeat interval, expiration timestamp, and various metadata.

Field	Type	Label	Description
reservation_id	ReservationID		The unique ID for the reservation
owner_id	string		The unique ID of the owner for the reservation
heartbeat_interval	google.protobuf.Duration		Recommended heartbeat interval to extend reservation
expires_at	google.protobuf.Timestamp		Expiration timestamp of this reservation
metadata	Metadata		Free-form metadata associated with the artifact

ReservationID

ReservationID message that is composed of several string fields.

Field	Type	Label	Description
dataset_id	DatasetID		The unique ID for the reserved dataset
tag_name	string		The specific artifact tag for the reservation

SinglePropertyFilter

A single property to filter on.

Field	Type	Label	Description
tag_filter	TagPropertyFilter
partition_filter	PartitionPropertyFilter
artifact_filter	ArtifactPropertyFilter
dataset_filter	DatasetPropertyFilter
operator	SinglePropertyFilter.ComparisonOperator		field 10 in case we add more entities to query

Tag

Tag message that is unique to a Dataset. It is associated to a single artifact and can be retrieved by name later.

Field	Type	Label	Description
name	string		Name of tag
artifact_id	string		The tagged artifact
dataset	DatasetID		The Dataset that this tag belongs to

TagPropertyFilter

Tag properties we can filter by

Field	Type	Label	Description
tag_name	string

UpdateArtifactRequest

Request message for updating an Artifact and overwriting its associated ArtifactData.

Field	Type	Label	Description
dataset	DatasetID		ID of dataset the artifact is associated with
artifact_id	string
tag_name	string
data	ArtifactData	repeated	List of data to overwrite stored artifact data with. Must contain ALL data for updated Artifact as any missing ArtifactData entries will be removed from the underlying blob storage and database.
metadata	Metadata		Update execution metadata(including execution domain, name, node, project data) when overwriting cache

UpdateArtifactResponse

Response message for updating an Artifact.

Field	Type	Label	Description
artifact_id	string		The unique ID of the artifact updated

PaginationOptions.SortKey

Name	Number	Description
CREATION_TIME	0

PaginationOptions.SortOrder

Name	Number	Description
DESCENDING	0
ASCENDING	1

SinglePropertyFilter.ComparisonOperator

as use-cases come up we can add more operators, ex: gte, like, not eq etc.

Name	Number	Description
EQUALS	0

DataCatalog

Data Catalog service definition Data Catalog is a service for indexing parameterized, strongly-typed data artifacts across revisions. Artifacts are associated with a Dataset, and can be tagged for retrieval.

Method Name	Request Type	Response Type	Description
CreateDataset	CreateDatasetRequest	CreateDatasetResponse	Create a new Dataset. Datasets are unique based on the DatasetID. Datasets are logical groupings of artifacts. Each dataset can have one or more artifacts
GetDataset	GetDatasetRequest	GetDatasetResponse	Get a Dataset by the DatasetID. This returns the Dataset with the associated metadata.
CreateArtifact	CreateArtifactRequest	CreateArtifactResponse	Create an artifact and the artifact data associated with it. An artifact can be a hive partition or arbitrary files or data values
GetArtifact	GetArtifactRequest	GetArtifactResponse	Retrieve an artifact by an identifying handle. This returns an artifact along with the artifact data.
AddTag	AddTagRequest	AddTagResponse	Associate a tag with an artifact. Tags are unique within a Dataset.
ListArtifacts	ListArtifactsRequest	ListArtifactsResponse	Return a paginated list of artifacts
ListDatasets	ListDatasetsRequest	ListDatasetsResponse	Return a paginated list of datasets
UpdateArtifact	UpdateArtifactRequest	UpdateArtifactResponse	Updates an existing artifact, overwriting the stored artifact data in the underlying blob storage.
GetOrExtendReservation	GetOrExtendReservationRequest	GetOrExtendReservationResponse	Attempts to get or extend a reservation for the corresponding artifact. If one already exists (ie. another entity owns the reservation) then that reservation is retrieved. Once you acquire a reservation, you need to periodically extend the reservation with an identical call. If the reservation is not extended before the defined expiration, it may be acquired by another task. Note: We may have multiple concurrent tasks with the same signature and the same input that try to populate the same artifact at the same time. Thus with reservation, only one task can run at a time, until the reservation expires. Note: If task A does not extend the reservation in time and the reservation expires, another task B may take over the reservation, resulting in two tasks A and B running in parallel. So a third task C may get the Artifact from A or B, whichever writes last.
ReleaseReservation	ReleaseReservationRequest	ReleaseReservationResponse	Release the reservation when the task holding the spot fails so that the other tasks can grab the spot.

Scalar Value Types

.proto Type	Notes	C++	Java	Python	Go	C#	PHP	Ruby
double		double	double	float	float64	double	float	Float
float		float	float	float	float32	float	float	Float
int32	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
int64	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.	int64	long	int/long	int64	long	integer/string	Bignum
uint32	Uses variable-length encoding.	uint32	int	int/long	uint32	uint	integer	Bignum or Fixnum (as required)
uint64	Uses variable-length encoding.	uint64	long	int/long	uint64	ulong	integer/string	Bignum or Fixnum (as required)
sint32	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
sint64	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.	int64	long	int/long	int64	long	integer/string	Bignum
fixed32	Always four bytes. More efficient than uint32 if values are often greater than 2^28.	uint32	int	int	uint32	uint	integer	Bignum or Fixnum (as required)
fixed64	Always eight bytes. More efficient than uint64 if values are often greater than 2^56.	uint64	long	int/long	uint64	ulong	integer/string	Bignum
sfixed32	Always four bytes.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
sfixed64	Always eight bytes.	int64	long	int/long	int64	long	integer/string	Bignum
bool		bool	boolean	boolean	bool	bool	boolean	TrueClass/FalseClass
string	A string must always contain UTF-8 encoded or 7-bit ASCII text.	string	String	str/unicode	string	string	string	String (UTF-8)
bytes	May contain any arbitrary sequence of bytes.	string	ByteString	str	[]byte	ByteString	string	String (ASCII-8BIT)