Setting up a project

In Union.ai, your work is organized in a hierarchy with the following structure:

  • Account: Your account on Union.ai, tied to your GitHub identity.
  • Domains: Within your account there are three domains, development, staging, and production, used to organize your code during the development process.
  • Projects: Orthogonal to domains, projects are used to organize your code into logical groups. You can create as many projects as you need.

A given workflow will reside in a specific project. For example, let’s say my_workflow is a workflow in my_project. When you start working on my_workflow you would typically register it in the project-domain my_project/development. As you work on successive iterations of the workflow you might promote my_workflow to my_project/staging and eventually my_project/production. Promotion is done simply by re-registering the workflow to the new project-domain.

Terminology

In everyday use, the term “project” is often used to refer to not just the Union.ai entity that holds a set of workflows, but also to the local directory in which you are developing those workflows, and to the GitHub (or other SCM) repository that you are using to store the same workflow code.

To avoid confusion, in this guide we will stick to the following naming conventions:

  • Union.ai project: The entity in your Union.ai instance that holds a set of workflows, as described above. Often referred to simply as a project.
  • Local project: The local directory in which you are developing workflows. This directory does not necessarily correspond one-to-one with a Union.ai project. Often you will have multiple workflow directories that are all registered to a single Union.ai project.
  • Project repository: The GitHub (or other SCM) repository that you are using to store and manage your workflow code. This repository does not necessarily correspond one-to-one with either workflow directory or a Union.ai project.

Create a Union.ai project

You can create a new project in the Union.ai UI by clicking on the project breadcrumb at the top left and selecting All projects:

Select all projects

This will take you to the Projects list:

Projects list

Click on the New Project button and fill in the details for your new project.

You now have a project on Union.ai into which you can register your workflows. The next step is to set up a local workflow directory.

Creating a local workflow directory using union init

Earlier, in the First workflow section of the guide, we started with a pre-existing example project, cloned from git. In this section we’ll start from scratch and create a new project using the Union CLI tool.

We will use the union init command to create a new workflow directory on your local machine pre-populated with a basic project structure defined by the basic-union-template found in the repository flyteorg/flytekit-python-template.

To create the workflow directory, run the following command:

$ union init --template basic-union-template basic-example

Directory structure

In the basic-example directory you’ll see the following file structure:

├── LICENSE
├── README.md
├── requirements.txt
└── workflows
    ├── __init__.py
    └── example.py

You can create your own conventions and file structure for your Union.ai projects. The union init command just provides a good starting point.

However, the separate workflows subdirectory and the contained __init__.py file are significant. We will discuss them when we cover the registration process.