/ Union for Machine Learning

Streamline ML,
Not Tool Collections

Building ML pipelines takes more than just training, testing and deploying models. You need accuracy, scalability, reliability and efficiency throughout the process. Union, a managed version of the Flyte™ workflow orchestrator, ensures flawless, efficient modeling and MLOps workflows.

“One of the biggest reasons we picked Flyte™ was because it is ideologically aligned with what we think all MLOps systems should have: strong lineage guarantees.”

— Jake Neyer, Software Engineer at Striveworks

How ML orchestration works

Machine Learning Orchestration is a game-changer for scalable machine-learning workflows and pipelines. These systems need to manage computing resources far more efficiently than data orchestrators and rely on features like check-pointing and visualization. ML orchestrators support the dynamic demands of ML development, enabling seamless implementation, management and deployment of production-grade ML pipelines, ultimately simplifying the creation of ML-powered products.

1. Explore

Conduct Exploratory Data Analysis (EDA) to gain dataset insights, identify patterns and relationships, and determine the most suitable techniques and tools for your machine learning project, such as natural language processing, computer vision and reinforcement learning.

6. Retrain

Retraining ML models is crucial to maintain their accuracy and relevance. It entails updating models with new data, insights or improved algorithms; adapting to changing conditions or user needs; and addressing concept drift.

5. Monitor

This component tracks model performance, health, and data and concept drift over time to identify the ideal moment to retrain or update a model. Tools like Grafana, Prometheus and Arize are instrumental in monitoring ML models.

ExploreRetrainMonitorCreateTestDeploy
Modeling
2. Create

ML models are trained on features, a process that demands significant computing power and can be repeated multiple times to develop an optimal model. To train models, you need frameworks like PyTorch and TensorFlow.

3. Test

Deployment ensures ML models are regularly updated or re-trained to manage dependency, optimize performance and scale horizontally. Specialized tools address specific aspects of deployment, such as Docker for dependency management and BentoML for serving models.

4. Deploy

Deployment ensures ML models are regularly updated or re-trained to manage dependency, optimize performance and scale horizontally. Specialized tools address specific aspects of deployment, such as Docker for dependency management and BentoML for serving models.

ML orchestration coordinates all these pipeline components end to end, providing critical features such as scheduling, versioning, resource management, dynamic workflows and caching. Orchestrators like Flyte™ or Kubeflow Pipeline offer ML pipeline orchestration, so teams can focus on building ML-powered products.

Learn More

Limitless: Union powered by Flyte™

Flyte™ is a machine-learning orchestrator that enforces an architectural design pattern for data and ML workflows. Built on top of Flyte™, Union lets ML engineers and data scientists focus on their work instead of managing infrastructure.

Union with Flyte™ makes it easy for data scientists and ML engineers to focus on their ML and model pipelines.

Infinitely scalable

Union lets you effortlessly expand your ML workflows, capitalizing on its virtually unlimited capabilities for growth and adaptability.

Built for scale

Union employs the Kubernetes-native Flyte™ engine — designed to handle extensive ML operations — to optimize performance even of large-scale deployments.

Language agnostic

Union provides data scientists and ML engineers the flexibility to compose workflows in their preferred programming languages, fostering seamless collaboration and innovation.

Versioned workflows

Union’s versioned workflows preserve the history of your runs, empowering you to track progress and easily revert to previous iterations when necessary.

Multi-tenancy

Union’s multi-tenant support lets multiple teams collaborate on a unified platform, promoting efficient knowledge sharing and streamlined teamwork.

Data lineage

Union enhances end-to-end data observability by tracking data lineage, ensuring that you have a clear understanding of data transformations and dependencies throughout the ML pipeline.

Strong typing

Union incorporates robust compile-time checks to bulletproof your workflows, reducing the likelihood of errors and increasing overall reliability.

Parallelism

Union’s inherent parallelism reduces wait times for your workflows, equipping you to execute tasks concurrently and complete projects more quickly.

Maximal efficiency

Union  ensures that you don't have to rerun workflows if assumptions remain unchanged, optimizing resource usage and promoting maximal efficiency.

Infrastructure managed

Union is responsible for provisioning and managing your Kubernetes cluster and resources. This allows users to scale workflows and data planes with ease.

“What sets machine learning apart from traditional software is that ML products must adapt rapidly in response to new demands.”

— Ketan Umare, Co-Creator of Flyte™ and CEO of Union.ai

Union.ai

A platform for every orchestration use case

Union is a versatile solution that can help you address any custom orchestration challenge, not just those related to machine learning.

Data

Build scalable and robust data workflows faster with strongly typed interfaces and data provenance.

Learn more

Analytics

Analyze and visualize trends and patterns in your data with native data-crunchers and FlyteDecks.

Learn more

Bioinformatics

Generate biological insights faster by collaborating on a centralized platform.

Learn more

Your Use Case

Build Flyte™ workflows to effectively address your orchestration challenge.

Request a demo