Component Model Aggregation

The aggregate endpoint allows you to run aggregation pipelines against component model data. This is useful for computing summaries, grouping data, and performing complex queries beyond simple CRUD operations.

Endpoint

POST /{version}/system/component/{ref}/aggregate

Response Modes

The aggregate endpoint supports two response modes: buffered (default) and streaming. The mode is selected via the Accept request header.

Buffered Mode (default)

Returns the full result set as a single JSON array. Best for small-to-medium result sets.

Parameter	Value
Default limit (no $limit in pipeline)	100
Maximum allowed limit	1,000

Exceeding the limit

If your pipeline includes a $limit stage greater than 1,000, the request will be rejected with an error: limit N exceeds maximum allowed limit of 1000.

Implicit limit

When no $limit stage is provided, the server automatically appends a limit of 100 to the pipeline.

Streaming Mode

Returns results as newline-delimited JSON (NDJSON), one document per line. Best for large result sets that would otherwise hit the buffered limit.

To enable streaming, set the Accept header to one of:

• application/x-ndjson
• application/stream+json

Parameter	Value
Default limit (no $limit in pipeline)	25,000
Maximum allowed limit	25,000
Timeout	5 minutes (fixed)

Exceeding the limit

If your pipeline includes a $limit stage greater than 25,000, the request will be rejected with an error: limit N exceeds maximum allowed limit of 25000 for streaming.

Implicit limit

When no $limit stage is provided, the server automatically appends a limit of 25,000 to the pipeline.

Pipeline Validation

All pipelines are validated before execution. The following rules apply:

Allowed Stages

Only these aggregation stages are permitted:

Stage	Description
$match	Filter documents
$limit	Limit result count
$skip	Skip documents
$unset	Remove fields
$addFields	Add computed fields
$set	Set field values
$sort	Sort documents
$project	Shape output documents
$group	Group and aggregate
$facet	Multi-faceted aggregation
$unwind	Deconstruct arrays
$count	Count documents

Pipeline Complexity

• Maximum 10 stages per pipeline. Exceeding this returns an error: pipeline too complex: has N stages, maximum allowed is 10.

Automatic Behaviour

• A { $match: { deleted_at: null } } stage is always prepended to exclude soft-deleted documents.

---

Usage Examples

Using the TypeScript SDK

Buffered Mode

The simplest way to use aggregate — just await the call. Results are returned as a single response (up to 1,000 documents).

import Protokol from '@protokol/sdk';

const client = new Protokol({
  baseURL: 'https://api.example.com',
  apiKey: 'your-api-key',
  project: 'your-project-uuid',
});

const component = client.component('orders');

// Simple aggregate — returns all results at once
const result = await component.aggregate([
  { $match: { status: 'completed' } },
  { $group: { _id: '$customer_id', total: { $sum: '$amount' } } },
  { $sort: { total: -1 } },
  { $limit: 50 },
]);

console.log(result.data); // Array of aggregated results

Streaming Mode

For large result sets (up to 25,000 documents), chain .onData() to enable streaming. Each document is delivered individually as it arrives.

const component = client.component('events');

await component.aggregate([
  { $match: { type: 'page_view' } },
  { $group: { _id: '$page', views: { $sum: 1 } } },
  { $sort: { views: -1 } },
  { $limit: 10000 },
])
  .onData((doc) => {
    // Called once per document as it streams in
    console.log('Received:', doc);
  })
  .onError((err) => {
    console.error('Stream error:', err);
  })
  .onEnd(() => {
    console.log('Stream complete');
  });

When to use streaming

Use streaming when you expect more than 1,000 results or want to process documents incrementally without buffering the entire result set in memory.

Without the SDK (raw HTTP)

Buffered Mode

curl -X POST \
  'https://api.example.com/v4/system/component/orders/aggregate' \
  -H 'Authorization: Bearer <token>' \
  -H 'Content-Type: application/json' \
  -H 'x-protokol-project: <project-uuid>' \
  -d '[
    { "$match": { "status": "completed" } },
    { "$group": { "_id": "$customer_id", "total": { "$sum": "$amount" } } },
    { "$sort": { "total": -1 } },
    { "$limit": 50 }
  ]'

Response:

[
  { "customer_id": "cust_001", "total": 15200 },
  { "customer_id": "cust_042", "total": 9800 }
]

Streaming Mode

Add the Accept: application/x-ndjson header to receive newline-delimited JSON:

curl -X POST \
  'https://api.example.com/v4/system/component/orders/aggregate' \
  -H 'Authorization: Bearer <token>' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/x-ndjson' \
  -H 'x-protokol-project: <project-uuid>' \
  -d '[
    { "$match": { "status": "completed" } },
    { "$group": { "_id": "$customer_id", "total": { "$sum": "$amount" } } },
    { "$sort": { "total": -1 } },
    { "$limit": 10000 }
  ]'

Response (each line is a separate JSON object):

{"customer_id":"cust_001","total":15200}
{"customer_id":"cust_042","total":9800}
{"customer_id":"cust_103","total":7450}
...

Parsing NDJSON in JavaScript (without SDK)

const response = await fetch(
  'https://api.example.com/v4/system/component/orders/aggregate',
  {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <token>',
      'Content-Type': 'application/json',
      'Accept': 'application/x-ndjson',
      'x-protokol-project': '<project-uuid>',
    },
    body: JSON.stringify([
      { $match: { status: 'completed' } },
      { $group: { _id: '$customer_id', total: { $sum: '$amount' } } },
      { $limit: 5000 },
    ]),
  }
);

const reader = response.body!.getReader();
const decoder = new TextDecoder();
let buffer = '';

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split('\n');
  buffer = lines.pop()!; // keep incomplete line in buffer

  for (const line of lines) {
    if (line.trim()) {
      const doc = JSON.parse(line);
      console.log('Document:', doc);
    }
  }
}