Bound the Onde Cloud request timeouts
A lone 300s read_timeout with no connect or write timeout meant a slow or hung Onde Cloud could sit on a puma thread for five minutes. A handful at once emptied the pool and took the site down, which is what happened on 2026-06-24. Added a 5s connect timeout and a 15s write timeout, and cut the read timeout to 90s. read_timeout fires per read rather than over the whole request, so a streaming reply can still take as long as it needs while tokens keep arriving; it only trips when the upstream goes quiet. A stall now returns a 504 quickly instead of parking a worker. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
keypair34 committed
Jun 24, 2026 at 23:58 UTC
d8e590b83c464cb26648642d87fd41bfd5038652
1 file changed
+16
-1
app/services/onde_cloud_service.rb
+16
-1
index 6c06624..a57396d 100644
--- a/app/services/onde_cloud_service.rb
+++ b/app/services/onde_cloud_service.rb
@@ -34,7 +34,20 @@ class OndeCloudService
end
DEFAULT_BASE_URL = "https://cloud.ondeinference.com/v1"
- READ_TIMEOUT = 300 # seconds — long enough for a full agent turn
+
+ # Net::HTTP timeouts. These bound how long a single request can pin a puma
+ # thread — load-bearing for availability: a hung/slow Onde Cloud must never be
+ # able to exhaust the web pool and take the whole app down (it did, 2026-06-24,
+ # when an Onde app was pinned to a server-side GGUF that stalled on CPU
+ # inference).
+ #
+ # read_timeout is PER-READ, not total, so streaming agent turns of any length
+ # stay safe as long as tokens keep flowing; it only fires when the upstream
+ # goes silent. Keep it short enough that a stall fails fast as a 504 instead of
+ # blocking a worker.
+ OPEN_TIMEOUT = 5 # connect
+ WRITE_TIMEOUT = 15 # sending the request body
+ READ_TIMEOUT = 90 # silence between reads before we give up
# Non-streaming completion. Returns the parsed JSON response (a Hash) verbatim.
def self.create(payload)
@@ -77,6 +90,8 @@ class OndeCloudService
uri.hostname,
uri.port,
use_ssl: uri.scheme == "https",
+ open_timeout: OPEN_TIMEOUT,
+ write_timeout: WRITE_TIMEOUT,
read_timeout: READ_TIMEOUT,
&block
)