Pagination and querying

Open in the interactive reference

In practice: request pages with page[size], send each response's pagination.nextCursor back as page[cursor], and stop when a response carries no nextCursor. The rest of this section is the detail behind that loop, and the query grammar every collection shares.

Paging through a collection

Collections return their records a page at a time: 25 by default, and at most 100. A larger size is served as 100; zero, a negative number or anything other than a whole number is refused with 400.

Each response reports pagination.hasMore and, when there is a next page to fetch, pagination.nextCursor.

The cursor is opaque: do not parse or construct one. It belongs to the filters and sort of the request that returned it, and is refused with 400 if they change. page[size] and fields can change from page to page.

A walk through a collection ends after about 100,000 records: from there on, a response carries no nextCursor even though hasMore is still true. To read further, narrow the query with a filter.

A page is a position in the result. When records are created or deleted during a walk, later pages shift, and a record can be skipped or returned twice. Without sort, the order itself is not guaranteed, so always sort on a field that orders records uniquely, such as id, and make your processing tolerate a record it has already seen.

Paging through a search

Search endpoints page the same way, in the request body: send page.size and page.cursor, and read data.pagination.hasMore and data.pagination.nextCursor. Each search endpoint states its default and largest page size, and what happens to a larger one.

Filtering, sorting and selecting fields

A collection takes filters, a sort order and a field selection in the query string. Its endpoint lists which fields support which filters, and which can be sorted. A filter or sort on any other field is refused with 400, never ignored.

Filters combine with AND: a record is returned only if it matches every filter. Each parameter can appear once; repeating one, such as sort or the same filter, is refused with 400.

A like pattern works on text fields only and is case-sensitive: % matches any sequence of characters and _ matches exactly one. In a URL, % is written %25, so %25part%25 finds values containing “part” and part%25 values starting with it. A pattern with neither must match the whole field.

The values of in, notIn and between are comma-separated in one parameter, so a value cannot itself contain a comma. in and notIn take up to 100 values, and between exactly two.

URL-encode every value you put in the query string yourself: a literal + as %2B, or it arrives as a space, and a % in a pattern as %25. Where your client encodes parameters for you, pass the characters themselves and let it encode them; encoding twice sends %2525, which matches nothing. The brackets in parameter names can be sent as they are; with curl, pass -g so that it does not treat them as a pattern.

On a collection's endpoint, the chips beside each field show what it supports; hover over one to see its parameter names.

Examples

Reading every page, in Python

params = {"page[size]": 100}
while True:
    body = requests.get(url, params=params, headers=headers).json()
    handle(body["data"])
    cursor = body["pagination"].get("nextCursor")
    if cursor is None:
        break
    params["page[cursor]"] = cursor

A page of a collection

{
  "data": [
    "…"
  ],
  "pagination": {
    "hasMore": true,
    "nextCursor": "…"
  }
}