Skip to main content

Ctrl+K

Course Notebooks

Ray Enablement Content

Anyscale For Admins

Anyscale Administrator Overview
1. What is an Anyscale Cloud?
2. Cloud Deployment Types
3. A Demonstrative Example of Resource Creation with AWS EC2
3.1 IAM Role Definition
4. Register Anyscale Cloud to Your Cloud Provider
Deployment Options: Virtual Machines vs. Kubernetes
2. Virtual Machines (VM) vs. Kubernetes (K8s)
3. (Optional) More Kubernetes Deployments Components
Introduction: Deploy Anyscale Ray on AWS EC2 Instances
1. Create Anyscale Resources with Terraform
2. Register the Anyscale Cloud
3. Test
4. Cleanup
5. Conclusion
Introduction: Deploy Anyscale Ray on GCP Compute Engine Instances (GCE)
Prerequisites
1. Installation
2. Create Anyscale Resources with Terraform
3. Register the Anyscale Cloud
4. Test
5. Cleanup
Introduction: Deploy Anyscale Ray on A New AWS EKS Cluster
1. Create Anyscale Resources with Terraform
2. Install Kubernetes Components
3. Register the Anyscale Cloud
4. Install the Anyscale Operator
5. Verify the Installation
6. Test
7. Clean up
8. Conclusion
Introduction: Deploy Anyscale Ray on An Existing AWS EKS Cluster
Prerequisites
1. Create Anyscale Resources with Terraform
2. Attach Required IAM Policies to Your existing EKS’s Node Role
3. Install Kubernetes Components
4. Register the Anyscale Cloud
5. Install the Anyscale Operator
6. Verify the Installation
7. Test
8. Troubleshooting
9. Clean up
10. Conclusion
Introduction: Deploy Anyscale Ray on A New Google Kubernetes Engine (GKE) Cluster
Prerequisites
1. Installation
2. Create Anyscale Resources with Terraform
3. Troubleshooting GPU Availability
4. kubectl Configuration
5. Install NGINX Ingress Controller
6. (Optional) Upgrade Anyscale Dependencies
7. Register the Anyscale Cloud
8. Install the Anyscale Operator
8. Test
9. Cleanup

Anyscale Getting Started

101 — Introduction to Anyscale Workspaces
101 – Developing Application with Anyscale
101 – Compute Configs and Execution Environments in Anyscale
101 – Storage Options in the Anyscale Platform
101 – Debug and Monitor Your Anyscale Application
101 – Introduction to Anyscale Jobs
101 – Introduction to Anyscale Services
101 – Collaboration on Anyscale
101 - Anyscale Organization and Cloud Setup
📌 Overview of Structure
🧠 Summary

Llm Serving

Introduction to Ray Serve LLM: Foundations of Large Language Model Serving
What is LLM Serving?
Key Concepts and Optimizations
Challenges in LLM Serving
Ray Serve LLM + Anyscale Architecture
Getting Started with Ray Serve LLM
Key Takeaways
Deploy a Medium-Sized LLM with Ray Serve LLM
Overview: Why Medium-Sized Models?
Setting up Ray Serve LLM
Local Deployment & Inference
Deploying to Anyscale Services
Advanced Topics: Monitoring & Optimization
Summary & Outlook
Advanced LLM Features with Ray Serve LLM
Overview: Advanced Features Preview
Example: Deploying LoRA Adapters
Example: Getting Structured JSON Output
Example: Setting up Tool Calling
How to Choose an LLM?
Conclusion: Next Steps

Multimodal Ai Workloads

Multi-modal AI pipeline
Batch inference
Data ingestion
Batch embeddings
Ray Data
Data storage
Monitoring and Debugging
Production jobs
Similar images
Distributed training
Preprocess
Model
Batching
Model registry
Training
Ray Train
Production Job
Evaluation
Online serving
Deployments
Application
Ray Serve
Observability
Production services
CI/CD

Observability

Observability Introduction
Observability Overview
Setting Up Local Ray Observability
Ray and Anyscale Observability Introduction
Ray Observability
Anyscale Observability
Example
Ray and Anyscale Observability in Detail
Data Pipeline Observability (Ray Data)
Web Application Observability (Ray Serve)

Ray Ai Libs

Introduction to Ray: Developer
Introduction to the Ray AI Libraries: An example of using Ray data, Ray Train, Ray Tune, Ray Serve to implement a XGBoost regression model
1. Overview of the Ray AI Libraries
2. Quick end-to-end example

Ray Core

Introduction to Ray Core: Getting Started
0. Overview
1. Creating Remote Functions
2. Executing Remote Functions
4. Putting It All Together
Introduction to Ray Core (Advancement): Object store, Tasks, Actors
1. Object store
2. Chaining Tasks and Passing Data
3. Task retries
4. Task Runtime Environments
5. Resource allocation and management
6. Nested Tasks
7. Pattern: Pipeline data processing and waiting for results
8. Ray Actors

Ray Data

Introduction to Ray Data: Industry Landscape
The Compute Layer
The Orchestration Layer
Distributed Computing Frameworks
Data Processing with Ray Data
Ray Serve
Introduction to Ray Data: Ray Data + Structured Data
0. What is Ray Data?
2. Loading Data
3. Transforming Data
4. Writing Data
5. Data Operations: Shuffling, Grouping and Aggregation
6. When to use Ray Data
Intro to Ray Data: Ray Data + Unstructured Data
1. When to Consider Ray Data
2. How to work with Ray Data
3. Loading data
3. Lazy execution mode
4. Transforming data
5. Stateful transformations with Ray Actors
6. Materializing data
7. Data Operations: grouping, aggregation, and shuffling
8. Persisting data
9. Ray Data in production

Ray Serve

Introduction to Ray Serve with PyTorch
1. When to Consider Ray Serve
2. Overview of Ray Serve
3. Implement an image classification service
4. Development workflow

Ray Train

📚 01 · Introduction to Ray Train
01 · Imports
04 · Define ResNet-18 Model for MNIST
05 · Define the Ray Train Loop (DDP per-worker)
06 · Define train_loop_config
07 · Configure Scaling with ScalingConfig
08 · Wrap the Model with prepare_model()
09 · Build the DataLoader with prepare_data_loader()
10 · Report Training Metrics
11 · Save Checkpoints and Report Metrics
14 · Create the TorchTrainer
16 · Inspect the Training Results
18 · Load a Checkpoint for Inference
🔄 02 · Integrating Ray Train with Ray Data
01 · Define Training Loop with Ray Data
02 · Build DataLoader from Ray Data
03 · Prepare Dataset for Ray Data
05 · Define Image Transformation
07 · Configure TorchTrainer with Ray Data
🛡️ 03 · Fault Tolerance in Ray Train
01 · Modify Training Loop to Enable Checkpoint Loading
02 · Save Full Checkpoint with Extra State
04 · Launch Fault-Tolerant Training
05 · Manual Restoration from Checkpoints
07 · Clean Up Cluster Storage
🎉 Wrapping Up & Next Steps

Ray Tune

Introduction to Ray Tune
1. Loading the data
2. Starting out with vanilla PyTorch
3. Hyperparameter tuning with Ray Tune
4. Ray Tune in Production

Pytorch Lightning

Introduction to Ray Train: Ray Train + PyTorch Lightning
1. When to use Ray Train
2. Single GPU Training with PyTorch Lightning
3. Distributed Training with Ray Train and PyTorch Lightning
4. Ray Train in Production

Ray Data Batch Inference

Batch Inference with Ray Data
Architecture
Load a dataset
Batch Inference Class
Create a batch data and call the model
Run inference on the entire dataset

Ray Data Processing

Data Processing with Ray Data
Library Imports
Convert to Ray Dataset
Filter Ray Dataset
Join Two Ray Datasets
Preprocessing with a Tokenizer

Ray Distributed Training

Distributed training with Ray Train, PyTorch and Hugging Face
1. Architecture
3. Metrics Setup
4. Training function per worker
5. Main Training Function
6. Start Training

Ray Serve Online Serving

Online Model Serving with Ray Serve
Architecture
FastAPI webservice and deploy a model
Simulate Client: Send test requests

Train Generative Cv

04-d1 Generative computer-vision pattern with Ray Train
1. Imports and setup
8. Pixel diffusion LightningModule
9. Ray Train train_loop (Lightning + Ray integration)
12. Resume from latest checkpoint
13. Reverse diffusion sampler

Train Policy Learning

04-d2 Diffusion-Policy Pattern with Ray Train
1. Imports and setup
4. DiffusionPolicy LightningModule
5. Distributed Train loop with checkpointing
8. Reverse diffusion helper

Train Rec Sys

04e Recommendation system pattern with Ray Train
1. Imports
7. Define matrix factorization model
8. Define Ray Train loop (with validation, checkpointing, and Ray-managed metrics)
11. Resume training from checkpoint
12. Inference: recommend top-N items for a user

Train Tabular

04b Tabular workload pattern with Ray Train
1. Imports
8. Define the Ray Train worker loop (Arrow-based, memory-efficient)
12. Confusion matrix visualization
15. Continue training from the latest checkpoint

Train Time Series

04c Time-Series workload pattern with Ray Train
1. Imports
9. PositionalEncoding and Transformer model
10. Ray Train training loop (with teacher forcing)
13. Resume training from checkpoint
14. Inference helper — Ray Data batch predictor on GPU

Train Vision Pattern

04a Computer-vision pattern with Ray Train
1. Imports
6. Custom Food101Dataset for Parquet
10. Helper: Ray-prepared DataLoaders
11. train_loop_per_worker
12. Launch distributed training with TorchTrainer
13. Plot training and validation loss curves
14. Demonstrate fault-tolerant resumption
15. Batch inference with Ray Data

Search

Ctrl+K

By The Jupyter Book community