Introducing Flyte Agents

Experience the unparalleled convenience of seamless integration with Union.
Agents empower you to effortlessly connect any hosted service into your workflows.

Featured agents

AWS Sagemaker

Train ML models on Sagemaker from within Flyte™.

pip install flytekitplugins-awssagemaker

Airflow

Run an Airflow operator as a Flyte task.

pip install flytekitplugins-airflow

BigQuery

Query a BigQuery table.

pip install flytekitplugins-bigquery

ChatGPT

Interact with ChatGPT.

pip install flytekitplugins-chatgpt

Databricks

Submit Spark jobs to Databricks.

pip install flytekitplugins-spark

File Sensor

Build event driven workflows.

pyflyte run --remote file_sensor_example.py wf

Memory Machine Cloud

Run a task on MMCloud.

pip install flytekitplugins-mmcloud

Snowflake

Query a Snowflake service.

pip install flytekitplugins-snowflake

Bridge the gap between scalability and ease of use

Execute locally

You can run a query on BigQuery, Snowflake, or even submit a Spark job to a Databricks cluster locally without running the backend server, making local development easier.

Copied to clipboard!
>>>	from flytekit import StructuredDataset, kwtypes
>>>	from flytekitplugins.bigquery import BigQueryConfig, BigQueryTask
>>>	query = BigQueryTask(
...		name="bigquery",
...		inputs=kwtypes(limit=int),
...		output_structured_dataset_type=StructuredDataset, 
...		task_config=BigQueryConfig(ProjectID="dogfood-gcp-dataplane"),
...		query_template="SELECT * from dataset.people LIMIT @limit;", # type: ignore
... )
>>> query(limit=5).open(pd.DataFrame).all()
		name		age
0		Alice		15
1 		bob		20
2 		charlie		31
3 		david		43
4 		edward		35
>>>
Infinite scalability

Infinite scalability

Agents are long-running, stateless services that can easily be scaled up or down based on the workload.

Language agnostic

Agents are flexible and can be coded in your preferred language, with the agent interface defined in protobuf.

Language agnostic
Enhanced efficiency

Enhanced efficiency

Agents reduce execution times and optimize resource usage by eliminating the need to create individual pods for tasks. They are capable of handling thousands of task requests simultaneously.

Three powerful agent types for your needs

Async Agent

Facilitate long-running jobs on external clouds like MMCloud or Nvidia DGX. Submit requests and monitor job status until completion.

Sync Agent

Swiftly send small gRPC/REST requests to external services and get instant results. Perfect for ChatGPT prompts!

Sensor

Monitor specific events like bucket changes or real-time workflow status updates.

Join our community to discuss and share ideas for agents