curia.infra.databricks.deploy module

This script deploys container tasks defined in airml_analytics

curia.infra.databricks.deploy.deploy_databricks_tasks(tasks: List[DatabricksTaskDefinition], env: str, resource_naming_config: DatabricksTaskDeployResourceNamingConfig) → None

Deploys all tasks as databricks jobs :param tasks: List of tasks to deploy :param env: The environment to deploy to :param resource_naming_config: The resource naming config to use :return:

curia.infra.databricks.deploy.create_or_update_databricks_job(task: DatabricksTaskDefinition, env: str, db_client: WorkspaceClient, resource_naming_config: DatabricksTaskDeployResourceNamingConfig)

Creates or updates a databricks job in the Databricks workspace, configuring it to run the task with access to appropriate S3 resources and the correct cluster configuration.