Production jobs

Anyscale Jobs (API ref) allows you to execute discrete workloads in production such as batch inference, embeddings generation, or model fine-tuning.

• define and manage Jobs in many different ways, including with a CLI or Python SDK.

• set up queues and schedules.

• set up all the observability, alerting, etc. around your Jobs.

Wrap the batch embedding generation workload as an Anyscale Job by providing the main command to run, python doggos/embed.py, and the appropriate compute and dependencies required for it. Also set the working directory to the default directory so that the Job has access to all the files for the workload.

Note:

• this step uses a containerfile to define dependencies, but you could easily use a pre-built image as well.

• you can specify the compute as a compute config or inline in a job config file.

• when you don’t specify compute while launching from a workspace, the configuration defaults to the compute configuration of the workspace.

• and of course we can launch Jobs from anywhere (not just from within Workspaces) where we can specify the compute config and dependencies for the Job to use. Learn more on how to create and manage Jobs.

%%bash
# Production batch embedding generation job
anyscale job submit -f /home/ray/default/configs/generate_embeddings.yaml