A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 03:30 UTC
[Docs] Add Anyscale to frameworks (#20590)
Signed-off-by: Ricardo Decal <rdecal@anyscale.com>
Ricardo Decal committed
Jul 7, 2025 at 20:09 UTC
3eb4ad53f3e4ed0d3224c37efd08b4a1a42076fc
1 file changed
+9
docs/deployment/frameworks/anyscale.md
new
+9
@@ -0,0 +1,9 @@
1
+---
2
+title: Anyscale
3
+---
4
+[](){ #deployment-anyscale }
5
+
6
+[Anyscale](https://www.anyscale.com) is a managed, multi-cloud platform developed by the creators of Ray.
7
+It hosts Ray clusters inside your own AWS, GCP, or Azure account, delivering the flexibility of open-source Ray
8
+without the operational overhead of maintaining Kubernetes control planes, configuring autoscalers, or managing observability stacks.
9
+When serving large language models with vLLM, Anyscale can rapidly provision [production-ready HTTPS endpoints](https://docs.anyscale.com/examples/deploy-ray-serve-llms) or [fault-tolerant batch inference jobs](https://docs.anyscale.com/examples/ray-data-llm).