SemiLayerDocs

Streaming with Joins

include works on the chunked streaming ops — stream.search and stream.query. It does not work on subscribe or observe (those stream single-record events; relations are fetched differently).

The shape

for await (const row of beam.recipes.stream.query({
  where:   { cuisine: 'italian' },
  include: {
    reviews: { limit: 3, orderBy: { field: 'rating', dir: 'desc' } },
  },
})) {
  row.reviews   // always present — populated before the row yields
}

Same IncludeSpec as the non-streaming ops. The relations are fetched per-batch, attached to each row, and the iterator yields the row with relations already in place.

Per-batch enrichment

The planner runs once per streamed batch, not per-row. So a stream pulling 10,000 rows in 50-row batches runs 200 batches × 1 batchRead per relation = 200 enrichment calls — not 10,000.

primary stream:    [batch 1] [batch 2] [batch 3] ...
                       ↓         ↓         ↓
enrichment:        batchRead  batchRead  batchRead   (per include, per batch)
                       ↓         ↓         ↓
                    yield     yield     yield         (rows with relations attached)

Latency note: each batch pays one round-trip for each include it carries. A stream.query with two relations is ~2x the round-trip count of the same stream without relations.

stream.search

Works the same:

for await (const hit of beam.recipes.stream.search({
  query:   'comforting pasta',
  include: { reviews: true },
})) {
  hit.metadata.reviews   // on metadata, just like non-streaming search
}

Same rule for where the relations land: .metadata.<relation> on search/similar/feed; directly on the row for query.

Partial failures mid-stream

If a relation's bridge goes down mid-stream, subsequent batches return empty for that relation. The error is added to meta.includeErrors on the final done frame, not per-row. Primary rows keep streaming.

let errors: any
for await (const row of beam.recipes.stream.query({
  include: { reviews: true, ingredientRows: true },
}) as any) {
  // rows arrive; reviews[] may be [] on later batches if that bridge failed
}

// After the loop, check the stream's done meta if your client surfaces it
// (the typed Beam wrapper exposes this as a trailing meta event)

For critical reads where partial results are unacceptable, use the non-streaming query() — it returns the full response in one shot and fails cleanly.

What doesn't support include

  • subscribe — streams row-level events (insert/update/delete). Relations aren't expanded because the event payload is a single row. If you need joined data on events, re-fetch via a follow-up query({ where: { id: event.recordId }, include }).
  • observe — single-record live view. Same rationale as subscribe.

Backpressure

The streaming transport is back-pressured at the WebSocket layer. If the consumer is slow, the server pauses the primary read — the batchRead enrichment only runs when the next batch is actually being emitted. Memory pressure on the service stays bounded by the concurrent batch size, not the full stream.

Choosing

NeedUse
Full table walk with joinsstream.query + include
Progressive results as they rankstream.search + include
Row-level events with joined datasubscribe then follow-up query
One-shot snapshotquery + include (cleanest error semantics)