Route inference through Onde Cloud; retire direct-Anthropic path
siGit is an Onde Cloud customer. Add OndeCloudService + POST /api/v1/chat/completions that authenticates the user token and forwards OpenAI-compatible requests to Onde Cloud with siGit's Onde app credentials (ONDE_CLOUD_APP_ID/SECRET), streaming passthrough included. Remove the direct-Anthropic /api/v1/messages path and AnthropicService; .env.example now documents the Onde Cloud customer creds. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Seto Elkahfi committed
Jun 23, 2026 at 01:52 UTC
7c35b5ac5ddef8f9d7c7ad4d95404b3bbf88874d
6 files changed
+200
-216
.env.example
+9
-5
@@ -24,13 +24,17 @@ SMBCLOUD_ENVIRONMENT=production
24
25
26
# -----------------------------------------------------------------------------
27
-# Anthropic (required for the desktop coding agent)
28
-# sigit-si proxies the Messages API on behalf of the siGit Code & Deploy app so
29
-# the desktop binary never holds this key. Get one at https://console.anthropic.com
30
-# POST /api/v1/messages forwards native Messages API requests using this key.
27
+# Onde Cloud (inference for the siGit clients)
28
+# The siGit platform is an Onde Cloud customer. sigit-si authenticates the user,
29
+# then forwards OpenAI-compatible chat completions to Onde Cloud using these app
30
+# credentials, so the client never holds them. The model provider lives behind
31
+# Onde Cloud. POST /api/v1/chat/completions uses these.
32
# -----------------------------------------------------------------------------
33
33
-ANTHROPIC_API_KEY=your-anthropic-api-key-here
34
+ONDE_CLOUD_APP_ID=your-onde-app-id-here
35
+ONDE_CLOUD_APP_SECRET=your-onde-app-secret-here
36
+# Optional; defaults to https://cloud.ondeinference.com/v1
37
+# ONDE_CLOUD_BASE_URL=https://cloud.ondeinference.com/v1
38
39
40
# -----------------------------------------------------------------------------
app/controllers/api/v1/chat_completions_controller.rb
new
+72
@@ -0,0 +1,72 @@
1
+# frozen_string_literal: true
2
+
3
+module Api
4
+ module V1
5
+ # OpenAI-compatible chat completions for the siGit clients (siGit Code, siGit
6
+ # Code Cloud). The client sends an OpenAI chat-completion request with its
7
+ # smbCloud access token as the bearer; we authenticate it, then forward the
8
+ # request to Onde Cloud through OndeCloudService using siGit's Onde app
9
+ # credentials. The client never sees those credentials or the model provider.
10
+ #
11
+ # POST /api/v1/chat/completions
12
+ # Header: Authorization: Bearer <access_token>
13
+ # Body: an OpenAI chat-completions request (model, messages, tools,
14
+ # max_tokens, …). Set "stream": true for an SSE response.
15
+ #
16
+ # Responses are OpenAI-compatible, forwarded verbatim:
17
+ # - non-stream → the chat.completion JSON object
18
+ # - stream → text/event-stream of chat.completion.chunk events
19
+ class ChatCompletionsController < Api::BaseController
20
+ include ActionController::Live
21
+
22
+ before_action :authenticate_token!
23
+
24
+ def create
25
+ if streaming_requested?
26
+ stream_completion
27
+ else
28
+ render json: OndeCloudService.create(completion_payload), status: :ok
29
+ end
30
+ rescue OndeCloudService::UpstreamError => e
31
+ render json: error_body(e.message), status: e.status
32
+ end
33
+
34
+ private
35
+
36
+ # Pipe Onde Cloud's SSE response straight through to the client.
37
+ def stream_completion
38
+ response.headers["Content-Type"] = "text/event-stream"
39
+ response.headers["Cache-Control"] = "no-cache"
40
+ response.headers["X-Accel-Buffering"] = "no" # disable nginx proxy buffering
41
+
42
+ OndeCloudService.stream(completion_payload) do |chunk|
43
+ response.stream.write(chunk)
44
+ end
45
+ rescue ActionController::Live::ClientDisconnected
46
+ # Client hung up — stop writing.
47
+ rescue OndeCloudService::UpstreamError => e
48
+ # Status line is already sent; surface the error in the stream.
49
+ response.stream.write("data: #{error_body(e.message).to_json}\n\n")
50
+ ensure
51
+ response.stream.close
52
+ end
53
+
54
+ # The OpenAI request body, forwarded verbatim. Read from the raw post so we
55
+ # send exactly what the client sent (no Rails parameter wrapping).
56
+ def completion_payload
57
+ @completion_payload ||= JSON.parse(request.raw_post)
58
+ rescue JSON::ParserError
59
+ {}
60
+ end
61
+
62
+ def streaming_requested?
63
+ ActiveModel::Type::Boolean.new.cast(completion_payload["stream"])
64
+ end
65
+
66
+ # Neutral OpenAI-style error envelope — no upstream provider detail.
67
+ def error_body(message)
68
+ { error: { message: message, type: "server_error" } }
69
+ end
70
+ end
71
+ end
72
+end
app/controllers/api/v1/messages_controller.rb
deleted
-99
@@ -1,99 +0,0 @@
1
-# frozen_string_literal: true
2
-
3
-module Api
4
- module V1
5
- # Server-side proxy to the Anthropic Messages API for the desktop coding
6
- # agent. The desktop app sends a native Messages API request with its
7
- # smbCloud access token as the bearer; we authenticate it, then forward the
8
- # call through AnthropicService, which injects the confidential
9
- # `ANTHROPIC_API_KEY` server-side. The desktop never sees the key.
10
- #
11
- # POST /api/v1/messages
12
- # Header: Authorization: Bearer <access_token>
13
- # Body: a native Anthropic Messages API request (model, messages,
14
- # system, tools, thinking, max_tokens, …). Set "stream": true for
15
- # an SSE response.
16
- #
17
- # Responses use the native Anthropic shapes so the desktop can speak the
18
- # standard Messages protocol:
19
- # - non-stream → the Message JSON object
20
- # - stream → text/event-stream of Messages API events
21
- # (message_start, content_block_delta, message_stop, …)
22
- class MessagesController < Api::BaseController
23
- include ActionController::Live
24
-
25
- before_action :authenticate_token!
26
-
27
- def create
28
- if streaming_requested?
29
- stream_message
30
- else
31
- message = AnthropicService.create(message_payload)
32
- render json: message.to_h, status: :ok
33
- end
34
- rescue AnthropicService::InvalidRequestError => e
35
- render_anthropic_error("invalid_request_error", e.message, status: :bad_request)
36
- rescue Anthropic::Errors::APIStatusError => e
37
- render_upstream_error(e)
38
- rescue Anthropic::Errors::APIConnectionError => e
39
- Rails.logger.error("Anthropic connection error: #{e.message}")
40
- render_anthropic_error("api_error", "Could not reach the model provider.",
41
- status: :bad_gateway)
42
- end
43
-
44
- private
45
-
46
- # Streams the upstream response to the client as Server-Sent Events,
47
- # re-emitting each Messages API stream event with its native `type` as the
48
- # SSE event name and the event body as JSON `data`. Errors raised mid-stream
49
- # are surfaced as an `error` event because the status line is already sent.
50
- def stream_message
51
- response.headers["Content-Type"] = "text/event-stream"
52
- response.headers["Cache-Control"] = "no-cache"
53
- response.headers["X-Accel-Buffering"] = "no" # disable proxy buffering (nginx)
54
-
55
- sse = ActionController::Live::SSE.new(response.stream)
56
- AnthropicService.stream(message_payload) do |event|
57
- sse.write(event.to_h, event: event.type)
58
- end
59
- rescue AnthropicService::InvalidRequestError => e
60
- write_sse_error(sse, "invalid_request_error", e.message)
61
- rescue Anthropic::Errors::APIStatusError => e
62
- write_sse_error(sse, e.try(:type) || "api_error", e.message)
63
- rescue Anthropic::Errors::APIConnectionError => e
64
- Rails.logger.error("Anthropic stream connection error: #{e.message}")
65
- write_sse_error(sse, "api_error", "Could not reach the model provider.")
66
- rescue ActionController::Live::ClientDisconnected
67
- # Client hung up mid-stream — nothing to do but stop writing.
68
- ensure
69
- sse&.close
70
- end
71
-
72
- # The raw inbound request body (everything except Rails routing keys),
73
- # handed to AnthropicService for curation.
74
- def message_payload
75
- params.except(:controller, :action, :format, :message).to_unsafe_h
76
- end
77
-
78
- def streaming_requested?
79
- ActiveModel::Type::Boolean.new.cast(params[:stream])
80
- end
81
-
82
- # Forwards Anthropic's own status code and error shape so the desktop's
83
- # Messages client sees a native error rather than a translated one.
84
- def render_upstream_error(error)
85
- status = error.try(:status) || 502
86
- render_anthropic_error(error.try(:type) || "api_error", error.message, status: status)
87
- end
88
-
89
- # Native Anthropic error envelope: { "type": "error", "error": { type, message } }.
90
- def render_anthropic_error(type, message, status:)
91
- render json: { type: "error", error: { type: type, message: message } }, status: status
92
- end
93
-
94
- def write_sse_error(sse, type, message)
95
- sse&.write({ type: "error", error: { type: type, message: message } }, event: "error")
96
- end
97
- end
98
- end
99
-end
app/services/anthropic_service.rb
deleted
-111
@@ -1,111 +0,0 @@
1
-# frozen_string_literal: true
2
-
3
-require "anthropic"
4
-
5
-# Server-side wrapper around the Anthropic Messages API.
6
-#
7
-# sigit-si is the trust boundary for the siGit Code & Deploy desktop app's
8
-# coding agent: the desktop is a public client (a shipped binary), so it must
9
-# never hold the org's Anthropic API key. Instead the app sends a Messages API
10
-# request to sigit.si/api/v1/messages with its smbCloud access token as the
11
-# bearer; sigit-si verifies that token (Api::BaseController#authenticate_token!)
12
-# and then makes the upstream call here, injecting the confidential
13
-# `ANTHROPIC_API_KEY` that lives only on the server.
14
-#
15
-# The request/response shapes are the native Anthropic Messages API shapes — we
16
-# only curate the inbound parameters and forward the model's output verbatim, so
17
-# the desktop can speak the standard Messages protocol (including streaming SSE,
18
-# tool use, and adaptive thinking) to its own backend.
19
-#
20
-# Required env var:
21
-# ANTHROPIC_API_KEY — the org's Anthropic API key (confidential, server-only)
22
-class AnthropicService
23
- # Models the proxy is allowed to serve. The key is the org's, so this caps the
24
- # blast radius (and cost) of an arbitrary `model` value from a public client.
25
- # Defaults to Claude Opus 4.8 — the coding-agent default.
26
- ALLOWED_MODELS = %w[
27
- claude-opus-4-8
28
- claude-opus-4-7
29
- claude-sonnet-4-6
30
- claude-haiku-4-5
31
- ].freeze
32
-
33
- DEFAULT_MODEL = "claude-opus-4-8"
34
-
35
- # Required by the Messages API; a sane ceiling for a single agent turn when the
36
- # client doesn't specify one. Clients that need long outputs (and stream) can
37
- # override up to the model's limit.
38
- DEFAULT_MAX_TOKENS = 16_000
39
-
40
- # Inbound keys we forward to the Messages API. Anything else (including
41
- # `stream`, which the helpers set themselves) is dropped.
42
- ALLOWED_KEYS = %i[
43
- model messages system max_tokens tools tool_choice thinking
44
- stop_sequences metadata output_config service_tier
45
- ].freeze
46
-
47
- # Raised when the inbound request is malformed before it reaches Anthropic
48
- # (e.g. an unsupported model). Carries an HTTP-ish status for the controller.
49
- class InvalidRequestError < StandardError; end
50
-
51
- # A single, app-wide client. `Anthropic::Client` is threadsafe and keeps its
52
- # own connection pool, so the recommendation is one instance per process.
53
- def self.client
54
- @client ||= Anthropic::Client.new(api_key: api_key, max_retries: 2)
55
- end
56
-
57
- # Non-streaming completion. Returns the `Anthropic::Message` response object;
58
- # the controller serializes it with `#to_h` to forward the native shape.
59
- def self.create(payload)
60
- client.messages.create(**message_params(payload))
61
- end
62
-
63
- # Streaming completion. Yields each raw Messages-API stream event
64
- # (`message_start`, `content_block_delta`, `message_stop`, …) so the controller
65
- # can re-emit them verbatim as Server-Sent Events.
66
- #
67
- # Uses `stream_raw` (not the `stream` helper): the helper interleaves
68
- # SDK-specific accumulation events (`text`, `thinking`, …) that aren't part of
69
- # the wire protocol, whereas `stream_raw` yields only the native events — so a
70
- # client speaking the standard Messages SSE protocol sees an unpolluted stream.
71
- def self.stream(payload, &block)
72
- client.messages.stream_raw(**message_params(payload)).each(&block)
73
- end
74
-
75
- # Curates an inbound request hash into Messages API parameters: symbolizes
76
- # keys, slices to the allowed set, applies defaults, and validates the model.
77
- def self.message_params(payload)
78
- raw = (payload || {}).deep_symbolize_keys.slice(*ALLOWED_KEYS)
79
-
80
- raw[:model] = resolve_model(raw[:model])
81
- raw[:max_tokens] = (raw[:max_tokens] || DEFAULT_MAX_TOKENS).to_i
82
-
83
- if raw[:messages].blank?
84
- raise InvalidRequestError, "`messages` is required and must be a non-empty array."
85
- end
86
-
87
- raw
88
- end
89
-
90
- # Validates an inbound model string against the allowlist, defaulting when
91
- # absent. Rejects unknown models rather than silently swapping them, so a
92
- # client typo surfaces instead of quietly billing a different model.
93
- def self.resolve_model(model)
94
- return DEFAULT_MODEL if model.blank?
95
-
96
- model = model.to_s
97
- unless ALLOWED_MODELS.include?(model)
98
- raise InvalidRequestError,
99
- "Unsupported model #{model.inspect}. Allowed: #{ALLOWED_MODELS.join(', ')}."
100
- end
101
- model
102
- end
103
-
104
- def self.api_key
105
- ENV.fetch("ANTHROPIC_API_KEY") do
106
- raise KeyError, "Missing required env var: ANTHROPIC_API_KEY"
107
- end
108
- end
109
-
110
- private_class_method :client, :message_params, :resolve_model, :api_key
111
-end
app/services/onde_cloud_service.rb
new
+118
@@ -0,0 +1,118 @@
1
+# frozen_string_literal: true
2
+
3
+require "net/http"
4
+require "json"
5
+
6
+# Server-side client for Onde Cloud, the OpenAI-compatible inference API.
7
+#
8
+# The siGit platform is an Onde Cloud customer: it holds Onde app credentials
9
+# (`app_id:app_secret`) and uses them to run inference on Onde Cloud. sigit-si is
10
+# the trust boundary — it authenticates the end user (Api::BaseController#
11
+# authenticate_token!), then forwards the request here with the customer creds,
12
+# which the public client never sees. The model provider behind Onde Cloud is
13
+# Onde Cloud's concern, not ours.
14
+#
15
+# Requests and responses are OpenAI-compatible chat completions, forwarded
16
+# verbatim (no translation). Streaming responses pass through as Server-Sent
17
+# Events.
18
+#
19
+# Required env vars:
20
+# ONDE_CLOUD_APP_ID — the siGit Onde app id
21
+# ONDE_CLOUD_APP_SECRET — the siGit Onde app secret
22
+# Optional:
23
+# ONDE_CLOUD_BASE_URL — API root (default https://cloud.ondeinference.com/v1)
24
+class OndeCloudService
25
+ # Raised when Onde Cloud returns a non-success status. `status` is forwarded to
26
+ # the client; the message is neutral (no upstream provider detail).
27
+ class UpstreamError < StandardError
28
+ attr_reader :status
29
+
30
+ def initialize(message, status: 502)
31
+ super(message)
32
+ @status = status
33
+ end
34
+ end
35
+
36
+ DEFAULT_BASE_URL = "https://cloud.ondeinference.com/v1"
37
+ READ_TIMEOUT = 300 # seconds — long enough for a full agent turn
38
+
39
+ # Non-streaming completion. Returns the parsed JSON response (a Hash) verbatim.
40
+ def self.create(payload)
41
+ JSON.parse(post(payload.merge("stream" => false)))
42
+ end
43
+
44
+ # Streaming completion. Yields raw SSE byte chunks as Onde Cloud emits them, so
45
+ # the controller can write them straight to the client. Onde Cloud already
46
+ # speaks the OpenAI SSE protocol (`data: {…}` … `data: [DONE]`).
47
+ def self.stream(payload)
48
+ uri = endpoint
49
+ request = build_request(uri, payload.merge("stream" => true), accept: "text/event-stream")
50
+ http_start(uri) do |http|
51
+ http.request(request) do |response|
52
+ ensure_success!(response)
53
+ response.read_body { |chunk| yield chunk }
54
+ end
55
+ end
56
+ end
57
+
58
+ # ── internals ────────────────────────────────────────────────────────────────
59
+
60
+ def self.post(payload)
61
+ uri = endpoint
62
+ request = build_request(uri, payload)
63
+ response = http_start(uri) { |http| http.request(request) }
64
+ ensure_success!(response)
65
+ response.body
66
+ end
67
+
68
+ def self.ensure_success!(response)
69
+ return if response.is_a?(Net::HTTPSuccess)
70
+
71
+ Rails.logger.error("Onde Cloud upstream #{response.code}: #{response.body.to_s.byteslice(0, 500)}")
72
+ raise UpstreamError.new("Onde Cloud upstream error (#{response.code})", status: response.code.to_i)
73
+ end
74
+
75
+ def self.http_start(uri, &block)
76
+ Net::HTTP.start(
77
+ uri.hostname,
78
+ uri.port,
79
+ use_ssl: uri.scheme == "https",
80
+ read_timeout: READ_TIMEOUT,
81
+ &block
82
+ )
83
+ rescue StandardError => e
84
+ Rails.logger.error("Onde Cloud connection error: #{e.message}")
85
+ raise UpstreamError.new("Onde Cloud is temporarily unavailable", status: 502)
86
+ end
87
+
88
+ def self.build_request(uri, payload, accept: "application/json")
89
+ request = Net::HTTP::Post.new(uri)
90
+ request["Authorization"] = auth_header
91
+ request["Content-Type"] = "application/json"
92
+ request["Accept"] = accept
93
+ request.body = payload.to_json
94
+ request
95
+ end
96
+
97
+ def self.endpoint
98
+ URI("#{base_url}/chat/completions")
99
+ end
100
+
101
+ def self.base_url
102
+ ENV.fetch("ONDE_CLOUD_BASE_URL", DEFAULT_BASE_URL).chomp("/")
103
+ end
104
+
105
+ # `Bearer app_id:app_secret`, the Onde Cloud customer credential.
106
+ def self.auth_header
107
+ app_id = ENV.fetch("ONDE_CLOUD_APP_ID") do
108
+ raise KeyError, "Missing required env var: ONDE_CLOUD_APP_ID"
109
+ end
110
+ app_secret = ENV.fetch("ONDE_CLOUD_APP_SECRET") do
111
+ raise KeyError, "Missing required env var: ONDE_CLOUD_APP_SECRET"
112
+ end
113
+ "Bearer #{app_id}:#{app_secret}"
114
+ end
115
+
116
+ private_class_method :post, :ensure_success!, :http_start, :build_request,
117
+ :endpoint, :base_url, :auth_header
118
+end
config/routes.rb
+1
-1
@@ -71,7 +71,7 @@ Rails.application.routes.draw do
71
get "repos", to: "repos#index"
72
post "repos", to: "repos#create"
73
post "git_credentials", to: "git_credentials#create"
74
- post "messages", to: "messages#create"
74
+ post "chat/completions", to: "chat_completions#create"
75
end
76
end
77