Streaming answers

By default, Ask an assistant returns the whole answer as JSON. Set stream to true to get the answer while it is written, as server-sent events.

curl -N https://api.lookio.app/v1/assistants/ASSISTANT_ID/query \
  -H "Authorization: Bearer $LOOKIO_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -d '{ "question": "How long do I have to return an item?", "stream": true }'

The response has Content-Type: text/event-stream; charset=utf-8, Cache-Control: no-store and an X-Request-Id header. Each event has a name and one line of JSON:

event: status
data: {"phase":"searching"}

event: delta
data: {"text":"You can return any item "}

event: delta
data: {"text":"within 30 days [1]."}

event: done
data: {"answer":"You can return any item within 30 days [1].","citations":[],"conversation_id":"cnv_p2x8w5r9m3hk","message_id":"msg_k7m3x9p2w8qr","credit_cost":30,"credits_remaining":970,"simulated":false}

citations is shortened here. It has the same shape as in the JSON answer.

Events

On success, done is always last. Use its answer as the final text, with its numbered citations.

When there is no other data, the stream sends the comment : keep-alive every 15 seconds, for example while an Eco answer starts. It keeps proxies from closing the stream. SSE parsers ignore it.

Failures

• Before the first event. A refusal such as insufficient_credits, insufficient_scope, resource_not_found or service_unavailable comes as a normal JSON error with its HTTP status. See Errors.
• After the first event. The HTTP status is already 200. The failure comes as an error event, then the stream ends. Treat any error event as a failed answer.

Read until done or error. If the stream closes without one of them, the answer failed. Do not use the partial text.

Conversations and costs

• With done, Lookio stores the question and the answer in the person’s chat or the API key’s conversation, except with store: false. done has the conversation_id and the answer’s message_id. See Conversations and end users.
• If your client disconnects, Lookio still finishes, charges and stores the answer.
• The cost is the same as for a JSON answer.
• The idempotency key does not replay a stream. A retry is a new answer, with a new charge. Before you ask again, check your conversations or your usage. See Conventions.

Read a stream in code

const response = await fetch(
  "https://api.lookio.app/v1/assistants/ASSISTANT_ID/query",
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.LOOKIO_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ question: "How long do I have to return an item?", stream: true }),
  },
);

if (!response.ok) throw new Error(await response.text()); // a JSON error, before any event

const decoder = new TextDecoder();
let buffer = "";
for await (const chunk of response.body) {
  buffer += decoder.decode(chunk, { stream: true });
  let end;
  while ((end = buffer.indexOf("\n\n")) >= 0) {
    const block = buffer.slice(0, end);
    buffer = buffer.slice(end + 2);
    const event = /^event: (.+)$/m.exec(block)?.[1];
    const data = JSON.parse(/^data: (.+)$/m.exec(block)?.[1] ?? "null");
    if (event === "delta") process.stdout.write(data.text);
    if (event === "done") console.log("\nCost:", data.credit_cost);
    if (event === "error") throw new Error(`${data.code}: ${data.message}`);
  }
}

In the CLI

In a terminal, lookio ask ... --stream prints the text as it comes. When stdout is not a terminal, it prints one JSON object per line: {"event":"status","data":{...}}. See Commands.