Ray Core¶
Tasks, actors, objects and scheduling.
This section holds 14 notebooks from ray_patterns/core/ in the ganymede repository. Each page shows the notebook as committed; use the buttons at the top right of a page to download the .ipynb or open it on GitHub.
Notebooks¶
- Map and reduce with remote tasks: Two mapper tasks run in parallel and a reducer task combines their results.
- Stateful actor: a parameter server: An actor keeps state between calls. One node creates the ParameterServer actor and a second node calls its set method.
- Fan out one task per list item: A single node launches one remote task per element of a list and gathers all results with one ray.get.
- Distributed queue between a producer and a consumer: A ray.util.queue.Queue connects a remote producer task to a remote consumer task. The builder writes both tasks for you.
- Task resources, retries and scheduling options: Per-call .options(): CPU and memory requests, automatic retries on application errors, a task name for the dashboard, the SPREAD scheduling strategy and a…
- Multiple return values and streaming generators: One task returns two separate ObjectRefs (num_returns=2); another is a generator whose values stream back one by one (num_returns=“streaming”).
- Take the first results with ray.wait: Three tasks with different run times feed a Wait node. ray.wait returns as soon as two are done, leaving the slow one running.
- Cancel a running task: A long-running task is cancelled with ray.cancel. Getting its result afterwards raises TaskCancelledError.
- Share one object with ray.put: A Put node stores a value in the object store once; two tasks receive the same ObjectRef instead of each getting their own copy.
- Named actor: create, look up, call and kill: A detached actor is registered under a name, looked up again with ray.get_actor, called, and finally removed with ray.kill.
- Reserve resources with a placement group: A placement group reserves two one-CPU bundles. Each task is pinned to a bundle, and the group is removed at the end.
- Actor with concurrency groups: The builder writes an actor class whose methods run in separate concurrency groups: up to four I/O calls at once, compute calls one at a time.
- Graph inputs as notebook parameters: Input nodes become typed variables at the top of the code. In the multi-cell layout they land in one cell tagged parameters, so papermill can override them.
- Lazy DAG with bind() and InputNode: The same graph style in bind mode: .bind() builds a lazy Ray DAG around an InputNode, and execute(x=5) runs it with a concrete input.