Streaming answers

By default Ask an assistant waits for the whole answer and returns JSON. Set stream to true to receive the answer as it is written, as server-sent events.

curl -N https://api.lookio.app/v1/assistants/ASSISTANT_ID/query \
  -H "Authorization: Bearer $LOOKIO_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -d '{ "question": "How long do I have to return an item?", "stream": true }'

The response has Content-Type: text/event-stream; charset=utf-8, Cache-Control: no-store and an X-Request-Id header. Each event has a name and one line of JSON:

event: status
data: {"phase":"searching"}

event: delta
data: {"text":"You can return any item "}

event: delta
data: {"text":"within 30 days [1]."}

event: done
data: {"answer":"You can return any item within 30 days [1].","citations":[],"conversation_id":"cnv_p2x8w5r9m3hk","message_id":"msg_k7m3x9p2w8qr","credit_cost":30,"credits_remaining":970,"simulated":false}

The citations array is shortened here. It has the same shape as in the JSON answer.

Events

done is always last on success. Use its answer as the final text: it is the complete answer with its citations numbered.

While nothing else is sent, for example when an Eco answer takes a while to start, the stream sends a comment line, : keep-alive, every 15 seconds so that proxies do not close it. SSE parsers ignore it.

A development deployment without the live engine sends the whole answer in one delta, then done, with no status event.

Failures

Two kinds of failure behave differently.

• Before the first event. A refusal such as insufficient_credits, insufficient_scope, resource_not_found or service_unavailable comes back as a normal JSON error with its HTTP status. See Errors.
• After the stream has started. The HTTP status is already 200, so a failure arrives as an error event that carries the standard error envelope, and the stream ends. Branch on error.code as you would for any failed request. internal_error means the answer could not be generated. Treat any error event as a failed answer.

Read until you receive done or error. A stream that closes without either is a failed answer: treat it as an error and do not use the partial text.

Conversations and costs

• Lookio stores the question and the answer together when done is sent, in the person’s chat or in the API key’s conversation, unless the key sent store: false. done carries the conversation_id to continue it and the answer’s message_id. See Conversations and end users.
• If your client disconnects mid-answer, Lookio finishes the answer anyway, charges it and stores it. Leaving does not cancel the cost.
• The cost is the same as for a JSON answer, and done reports credit_cost and credits_remaining.
• A stream is not replayed by the idempotency key, so do not retry one blindly: the retry would be a new answer. Check your conversations or your usage before you ask again. See Conventions.

Read a stream in code

const response = await fetch(
  "https://api.lookio.app/v1/assistants/ASSISTANT_ID/query",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.LOOKIO_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ question: "How long do I have to return an item?", stream: true }),
  },
);

if (!response.ok) throw new Error(await response.text()); // a JSON error, before any event

const decoder = new TextDecoder();
let buffer = "";
for await (const chunk of response.body) {
  buffer += decoder.decode(chunk, { stream: true });
  let end;
  while ((end = buffer.indexOf("\n\n")) >= 0) {
    const block = buffer.slice(0, end);
    buffer = buffer.slice(end + 2);
    const event = /^event: (.+)$/m.exec(block)?.[1];
    const data = JSON.parse(/^data: (.+)$/m.exec(block)?.[1] ?? "null");
    if (event === "delta") process.stdout.write(data.text);
    if (event === "done") console.log("\nCost:", data.credit_cost);
    if (event === "error") throw new Error(`${data.code}: ${data.message}`);
  }
}

In the CLI

lookio ask ... --stream prints the text as it arrives in a terminal. When stdout is not a terminal, it prints one JSON object per line: {"event":"status","data":{...}}. See Commands.