Skip to content

Ray patterns

A gallery of small Ray programs, one notebook each. Every notebook pairs a picture of a graph drawn in Eugo's visual Ray builder with the Ray code the builder generated for it, then runs that code and checks the result. The generated code is plain Ray, so you do not need the builder to run or adapt it. The notebooks were run on Ray 2.55.1 with Ray capped at four CPUs and no GPU; the few that need a GPU or an external service are marked code only.

Each notebook has the same layout:

  1. what the pattern is for and when to use it;
  2. the graph in the visual builder;
  3. a setup cell that starts Ray quietly;
  4. the user functions the graph's nodes call;
  5. the generated code (where a notebook edited it by hand, the cell's first line says so);
  6. a check of the result;
  7. a walkthrough that maps each node to the lines it produced;
  8. other forms the builder can emit (a bind() DAG, a standalone script, a Ray Job submit script), where they exist;
  9. clean-up.

Running the notebooks

pip install "ray[default,data,serve,train,tune]==2.55.1" "protobuf<7" jupyterlab

Some notebooks need more packages; the Extra packages column below lists them. For example:

pip install torch lightgbm xgboost optuna

Then open a notebook in JupyterLab and run all cells. Each notebook starts and stops its own local Ray instance, so run them one at a time. The Serve notebooks listen on local ports 8000, 8123 or 9000.

The visual builder itself is not publicly available, and nothing here needs it.

Ray Core

Tasks, actors, objects and scheduling.

# Pattern What it shows Extra packages Graph
1 Map and reduce with remote tasks Two mapper tasks run in parallel and a reducer task combines their results. — graph
2 Stateful actor: a parameter server An actor keeps state between calls. — graph
3 Fan out one task per list item A single node launches one remote task per element of a list and gathers all results with one ray.get. — graph
4 Distributed queue between a producer and a consumer A ray.util.queue.Queue connects a remote producer task to a remote consumer task. experimental — graph
5 Task resources, retries and scheduling options Per-call .options(): CPU and memory requests, automatic retries on application errors, a task name for the dashboard, the SPREAD scheduling strategy and a timeout on ray.get. — graph
6 Multiple return values and streaming generators One task returns two separate ObjectRefs (num_returns=2); another is a generator whose values stream back one by one (num_returns="streaming"). — graph
7 Take the first results with ray.wait Three tasks with different run times feed a Wait node. — graph
8 Cancel a running task A long-running task is cancelled with ray.cancel. — graph
9 Share one object with ray.put A Put node stores a value in the object store once; two tasks receive the same ObjectRef instead of each getting their own copy. — graph
10 Named actor: create, look up, call and kill A detached actor is registered under a name, looked up again with ray.get_actor, called, and finally removed with ray.kill. — graph
11 Reserve resources with a placement group A placement group reserves two one-CPU bundles. — graph
12 Actor with concurrency groups The builder writes an actor class whose methods run in separate concurrency groups: up to four I/O calls at once, compute calls one at a time. — graph
13 Graph inputs as notebook parameters Input nodes become typed variables at the top of the code. — graph
14 Lazy DAG with bind() and InputNode The same graph style in bind mode: .bind() builds a lazy Ray DAG around an InputNode, and execute(x=5) runs it with a concrete input. — graph

Ray Data

Distributed datasets: read, transform, aggregate, write.

# Pattern What it shows Extra packages Graph
15 Ray Data: range, map_batches, count A three-step Ray Data pipeline: create a dataset, transform it in batches, and count the rows. — graph
16 Ray Data: write Parquet Writes a dataset to a Parquet directory with SaveMode.OVERWRITE, so re-running the cell replaces the output. — graph
17 Ray Data: groupby and aggregate Groups rows by a key and computes Count and Sum per group with Ray Data built-in aggregations. — graph
18 Ray Data: computed columns and expression filters with_column adds a computed column from col() and lit() expressions, and filter(expr=...) keeps the matching rows, with no Python UDF. experimental — graph
19 Ray Data: join two datasets An inner join of two datasets on an id column. — graph
20 Ray Data: train/test split train_test_split has two outputs; each one feeds its own count. — graph
21 Ray Data: batch inference with an actor pool map_batches with a callable class and ActorPoolStrategy: each actor runs the expensive setup once and then processes many batches. — graph
22 Ray Data: shuffle, repartition and sort A seeded random_shuffle, a repartition into four blocks and a descending sort, then take(5). — graph
23 Ray Data: from pandas and back Turns a pandas DataFrame into a Ray Dataset, adds a column row by row with map, and collects the result as a DataFrame again. — graph
24 Ray Data: fit a StandardScaler preprocessor A Ray Data preprocessor learns column statistics in one pass (fit) and applies them (transform). experimental — graph
25 Ray Data: read a catalog table A catalog node creates a DatabricksUnityCatalog client from an environment variable; the reader resolves the table through it. Needs Ray 2.58.0 or later: the catalog API is not in Ray 2.55.1 (the notebook shows the read_unity_catalog alternative for 2.55.1). code only, experimental — Mermaid diagram in the notebook
26 Ray Data: batch LLM inference The llm transform builds a ray.data.llm processor around a vLLM engine and runs prompts through it as a dataset stage. code only, experimental ray[llm] graph

Ray Serve

Model serving: deployments, composition, ingress, scaling.

# Pattern What it shows Extra packages Graph
27 Ray Serve: compose two deployments A Greeter deployment is passed into a Driver deployment with .bind(); serve.run starts both as one application. — graph
28 Ray Serve: replicas and per-replica resources A three-deployment pipeline where the classifier runs two replicas with half a CPU each and a cap on in-flight requests. — graph
29 Ray Serve: autoscaling a deployment autoscaling_config lets Serve add replicas (here between 1 and 3) as the number of ongoing requests per replica grows; max_queued_requests sheds load beyond a limit. — graph
30 Ray Serve: FastAPI ingress with a route prefix A FastAPI app, built on each replica by a builder function, becomes the HTTP ingress of a deployment served under the /api route prefix. — graph
31 Ray Serve: dynamic request batching The builder writes a deployment whose handler is wrapped in @serve.batch, so concurrent requests are grouped into one call. experimental — graph
32 Ray Serve: model multiplexing One deployment serves many models: @serve.multiplexed loads a model by id on demand and keeps up to two per replica. experimental — graph
33 Ray Serve: HTTP proxy settings and status An HTTP proxy config node starts Serve on 127.0.0.1:8123 before the app is deployed, and a status node reads serve.status() afterwards. experimental — graph
34 Ray Serve: gRPC ingress A gRPC ingress node adds a gRPC proxy on port 9000 with a servicer function; the deployment answers protobuf requests. — graph
35 Ray Serve: an OpenAI-compatible LLM endpoint An LLM node builds an OpenAI-compatible app with ray.serve.llm: model id, model files on Hugging Face, accelerator type and autoscaling bounds. code only openai, ray[llm] graph

Ray Train

Distributed training with Torch, LightGBM and XGBoost. The trainer type is a setting of the Trainer node that the canvas does not draw, so the graphs for 36, 37 and 38 look the same.

# Pattern What it shows Extra packages Graph
36 Ray Train: TorchTrainer over a dataset A TorchTrainer runs a training loop on one CPU worker and reads its shard of a Ray Dataset. torch graph
37 Ray Train: LightGBMTrainer A LightGBMTrainer runs a gradient-boosted tree training loop on a Ray Dataset shard. lightgbm graph
38 Ray Train: XGBoostTrainer An XGBoostTrainer runs a gradient-boosted tree training loop on a Ray Dataset shard. xgboost graph
39 Ray Train: validation data, run config and results Two workers train with a train and a validation dataset, a train_loop_config dict and a RunConfig; result nodes pull the final metrics and the run directory out of the Result. torch graph

Ray Tune

Hyperparameter search, schedulers and stoppers.

# Pattern What it shows Extra packages Graph
40 Ray Tune: grid search with ASHA early stopping A Tuner sweeps the learning rate of a TorchTrainer over a grid, and the ASHA scheduler stops weak trials early. torch graph
41 Ray Tune: Optuna search The same trainer tuned with OptunaSearch sampling the learning rate from a uniform range. optuna, torch graph
42 Ray Tune: median stopping, a stopper and the ResultGrid Two grid trials under the median stopping rule and a MaximumIteration stopper; ResultGrid nodes pull a DataFrame of trial results and the error count. experimental torch graph

Across libraries

Programs that combine several Ray libraries.

# Pattern What it shows Extra packages Graph
43 One program across Core, Data, Train and Tune A Core task, a Data pipeline, a standalone Train run and a Tune sweep in one graph; the builder emits them in that order as one program. torch graph
44 Export as a script and submit it as a Ray Job The builder exports a graph as a standalone Python script (functions inlined, an if name == "main" entry point) and writes a shell script that submits it with ray job submit. — graph

Notes

  • Experimental marks graphs that use a node the builder lists in its Experimental section, or a Ray API that Ray 2.55.1 marks as alpha or beta. The Ray code they generate ran on Ray 2.55.1 unless the notebook is also marked code only.
  • A grey alpha badge on a node in a picture means the Ray API behind that node is alpha in Ray 2.55.1 (for example Dataset.with_column in 18 and the StandardScaler preprocessor in 24). Those graphs are marked experimental.
  • Code only notebooks need a GPU, a model-serving engine or an external data catalog, and the catalog pattern (25) also needs a Ray release newer than 2.55.1 (2.58.0 or later). Their code is shown without outputs.
  • Ray Serve 2.55.1 does not work with protobuf 7: it uses FieldDescriptor.label, which protobuf 7 removed. Keep protobuf<7, as in the install line above.
  • On Ray 2.55.1, @serve.ingress(app) with a FastAPI app object fails to pickle with current FastAPI releases. Pass a builder function that creates the app instead, as notebook 30 does.
  • Some pictures cannot show everything: a node's caption is the builder's shorthand (for example ds.llm()), options such as num_returns are not drawn, and the layout may place a graph's inputs in a different order from the code. The caption under each picture and the walkthrough map every node to the real Ray calls.
  • Ray, Ray Core, Ray Data, Ray Serve, Ray Train and Ray Tune are part of the open-source Ray project.

Sections

  • Ray Core: 14 notebooks. Tasks, actors, objects and scheduling.
  • Ray Data: 12 notebooks. Distributed datasets: read, transform, aggregate, write.
  • Ray Serve: 9 notebooks. Model serving: deployments, composition, ingress, scaling.
  • Ray Train: 4 notebooks. Distributed training with Torch, LightGBM and XGBoost. The trainer type is a setting of the Trainer node that the canvas does not draw, so the graphs for 36, 37 and 38 look the same.
  • Ray Tune: 3 notebooks. Hyperparameter search, schedulers and stoppers.
  • Across libraries: 2 notebooks. Programs that combine several Ray libraries.