expert: stream the expert on the non-streaming escalation path too
ober
5e5805667af6358300dbeb0a65c07add3bd34688
--- a/src/jcode/core/expert.ss +++ b/src/jcode/core/expert.ss @@ -182,9 +182,19 @@ (strip-expert-sentinel content) (message-tool-calls response)) response)]) - (provider-chat expert - (with-expert-handoff messages reason-text) - tools)))) + ;; Stream the expert even on this non-streaming wrapper: chunked + ;; reads return as tokens arrive (~50ms apart) instead of one read + ;; that blocks in-kernel for the whole max_tokens-bounded, possibly + ;; multi-minute response — the operation that (with a mis-set + ;; O_NONBLOCK) previously pinned Chez's GC and froze the UI. No + ;; token sink is reachable from here, so collect silently and + ;; rebuild the assistant message from the streamed values. + (let-values (((xcontent xtcs _xusage) + (provider-stream-chat expert + (with-expert-handoff messages reason-text) + tools + (lambda (_tok) (void))))) + (make-assistant-message xcontent xtcs))))) (sentinel? (log-warn logger "expert-requested-but-not-configured" '()) (make-assistant-message