Improve your Airflow DAGs with Flyte

“We get a lot of reusable workflows and it makes it fairly easy to share complex machine learning and different dependencies between teams without actually having to put all the dependencies into one container.”

“Gojek is experiencing rapid growth and incorporating machine learning into various products. To sustain this growth and guarantee success, a reliable and scalable pipeline solution is critical. Flyte plays a vital role as a key component of Gojek’s ML Platform by providing exactly that.”

“You can say, ‘Give me imputation’ and [Flyte™ will] launch 40 spot instances that are cheaper than your on-demand instance that you're using for your notebook and return the results back in memory.”

“We’re going to have 10,000-plus CPUs that we plan to use every day processing the raw data. There’ll be 30 different targets that we’re collecting data on every day, and it’s that’s about 200 gigs of raw data, and then our estimate is for final data, we’re probably like two terabytes or so on the output. So a lot of data to process, and we’re leaning heavily on Flyte™ to make that happen.”

“Flyte is really execution engine agnostic and so like if you wanted to run Spark workloads if you wanted to run pytorch workloads uh or whatever else like you know AWS batch or Google AI platform or something like that like we can basically write a plug-in uh that would work and you know some most of these plugins already exist.”

“One thing that I really like compared to my previous experience with some of these tools was the local dev experience with pyflyte and the sandbox reduce that friction between production and dev environments so that was really nice for me to see it.”