Release Notes

With the Ollama plugin, serving fine-tuned models becomes even more efficient. Whether you’re fine-tuning a model for further evaluation or deploying an LLM for downstream tasks, the plugin makes it easy to integrate Ollama into Flyte tasks without the usual complexities of orchestrating serving infrastructure. Simply instantiate the plugin, and you’ll have direct access to your model right within your Flyte task.

Learn more: Serve Fine-tuned LLMs with Ollama

With Union and NIM, you can rapidly promote your workflows from AI development to production, reducing both operational costs and time to market. This integration enables you to self-host and serve optimized AI models on your own infrastructure powered by Union, ensuring full control over costs and data security. By eliminating dependence on third-party APIs for AI model access, you gain not only enhanced control but also potentially lower expenses compared to traditional API services.  

Union and NGC combined will take AI development and deployment up a notch. If you find yourself struggling with serving AI models and orchestrating AI pipelines, give this integration a try. 

Learn more: Union Powers Faster End-to-End AI Application Deployment using NVIDIA NIM

With Flyte’s latest plugin for Weights & Biases, you can now effectively run Machine Learning or AI workflows on Union and integrate with Weights & Biases capabilities. Union provides scalability, declarative infrastructure, and data lineage allowing you to quickly iterate and productionize AI or ML workflows. Weights & Biases helps customers build models faster, fine-tune LLMs, and develop GenAI applications with confidence, all in one system of record. 

With flytekit's Weights & Biases plugin, you can easily track our experiments, visualize results, and debug our models. Use the plugin by installing it with pip install flytekitplugins-wandb.

Learn more: Flyte and Weights & Biases Integration

The new Comet Flyte plugin enables you to use Comet’s machine-learning platform to manage, track, and visualize models during training.

Union's declarative infrastructure and scalable orchestration platform makes it simple to scale up your machine learning or AI workflows and put them in production. With flytekit's Comet plugin, you can easily track experiments, visualize results, and debug models. To use the plugin, install it with pip install flytekitplugins-comet-ml.

Union and Comet offer powerful features independently. This integration significantly enhances their combined capabilities, reducing manual effort, improving efficiency, and ensuring more comprehensive tracking and visualization of AI workflows. 

Learn More: Comet Integration with Union & Flyte

Union Serverless users can now harness the power of NVIDIA A100 Tensor Core GPUs and NVIDIA L4 Tensor Core GPUs, in addition to our existing support for NVIDIA T4 GPUs, with NVIDIA H100 Tensor Core GPUs coming soon.

Modern machine learning development requires access to production-grade GPU hardware optimized for parallelized computation and high throughput. Union Serverless makes it easier than ever to develop and scale your ML applications with expanded GPU access that now includes NVIDIA A100 GPUs. Use the Union Serverless platform to first develop locally in Python and then seamlessly orchestrate complex jobs and efficiently run distributed training in the cloud.

For specific examples on how to leverage NVIDIA A100 GPUs to build out rich ML applications, see:

• Credit Default Prediction with XGBoost & NVIDIA RAPIDS
• Forecasting with GluonTS & PyTorch on GPUs
• HDBSCAN Soft Clustering With Headline Embeddings on GPUs

Learn more: Introducing A100s

Until now, users needed to juggle a combination of command-line tools, namely `flytekit` and `uctl`, in order to develop workflows and manage user-facing parts of the platform. That’s why we created the `union` SDK, so you can simplify getting started and workflow development on Union’s Serverless and bring-your-own compute (BYOC) platforms. The `union` SDK removes complexities while adding functionality.

A Seamless, Integrated Development Kit

The `union` SDK is now the go-to resource for most users starting with Union. By innovating on the flytekit plugin system, we’ve allowed for customized behaviors specific to Union’s requirements. The introduction of a FlytekitPlugin means behaviors can be extended and adapted by Union, enhancing the `pyflyte` CLI to incorporate additional capabilities not found in open-source Flyte. These enhancements ensure a smooth integration and extend the functionality of existing tools, making the development process more intuitive and powerful. Our documentation walks through how to get started with `union`.

Empowering Users with Cutting-Edge Features

`Union` has already proven itself to be the platform of choice for developers tasked with managing a range of workflows in production. Now, the `union` SDK simplifies new user onboarding: users can simply visit Union.ai, sign up, install the `union` SDK, and run their workflows remotely. Key new functionalities include a serverless image builder that simplifies image management and a streamlined secrets management experience that removes the need to configure secrets at your cloud provider. Additionally, the SDK extends `FlyteRemote` to `UnionRemote`, facilitating seamless operations in both serverless environments and BYOC setups with minimal user adjustment. With these enhancements, the `union` SDK makes advanced computing more accessible to a broader audience.

Managing projects in a multi-tenant environment just got easier. You can now filter by archived projects and create new projects directly from the user interface.

Additionally, we have introduced memory and resource utilization so you can better manage resource consumption for each execution. 

Learn how to set up a project in our docs: docs.union.ai/byoc/development-cycle/setting-up-a-project#create-a-union-project

See your visibility story with ease. You can now create and execute workflows directly in the web interface. This visual workflow interface allows any team in your organization to maintain a common utility project and collaborate on composable workflows on the fly.

Learn how to create workflows in our docs: 

• ‍docs.union.ai/byoc/first-workflow
• ‍docs.union.ai/byoc/core-concepts/workflows/viewing-workflow-executions#graph

Artifacts create a first-class abstraction over the inputs and outputs of tasks and workflows. Introduce soft dependencies between workflows with artifact queries, allowing you to break out your evaluation steps and batch prediction step with your modeling step. 

Reactive workflows enable cross-workflow automation by reactively kicking off downstream workflows in response to the completion of upstream workflows. Reactive workflows leverage artifacts as the medium of exchange between workflows, such that when an upstream workflow emits an artifact, an artifact-driven trigger in a downstream workflow passes the artifact to a new downstream workflow execution. 

Artifacts and reactive workflows can be used together to modularize individual teams’ workflows while automating and providing reproducibility across the overall AI development pipeline.

Read our blog post “Move Fast and Don’t Break Things: Introducing Artifacts Lineage and Reactive Workflows” 

Learn more about Artifacts in our docs: docs.union.ai/byoc/core-concepts/artifacts/#artifacts

Learn more about Reactive Workflows in our docs: docs.union.ai/byoc/core-concepts/launch-plans/reactive-workflows#triggers

Pre-load static, read-only datasets into compute nodes and reuse them to reduce a major part of the overhead incurred by using ephemeral compute resources. 

Accelerated datasets bring together the competing requirements to unlock the cost savings that ephemeral compute provides, with close to native disk performance. The on node caching mechanism is great for tasks requiring access to large and sometimes static datasets. Accelerated datasets are ideal for long running, high volume, homogenous workflows.

Read our blog post “Reduce the Runtime & Memory Requirements of your Workloads by more than 50% with Accelerated Datasets” 

Learn more in our docs: docs.union.ai/byoc/data-input-output/accelerated-datasets#accelerated-datasets