# Pagination and querying

[Open in the interactive reference](https://docs.api.lensing.ai/?op=guide%3Apagination)

In practice: request pages with `page[size]`, send each response's `pagination.nextCursor` back as `page[cursor]`, and stop when a response carries no `nextCursor`. The rest of this section is the detail behind that loop, and the query grammar every collection shares.

### Paging through a collection

Collections return their records a page at a time: 25 by default, and at most 100. A larger size is served as 100; zero, a negative number or anything other than a whole number is refused with `400`.

Each response reports `pagination.hasMore` and, when there is a next page to fetch, `pagination.nextCursor`.

The cursor is opaque: do not parse or construct one. It belongs to the filters and sort of the request that returned it, and is refused with `400` if they change. `page[size]` and `fields` can change from page to page.

A walk through a collection ends after about 100,000 records: from there on, a response carries no `nextCursor` even though `hasMore` is still true. To read further, narrow the query with a filter.

A page is a position in the result. When records are created or deleted during a walk, later pages shift, and a record can be skipped or returned twice. Without `sort`, the order itself is not guaranteed, so always sort on a field that orders records uniquely, such as `id`, and make your processing tolerate a record it has already seen.

### Paging through a search

Search endpoints page the same way, in the request body: send `page.size` and `page.cursor`, and read `data.pagination.hasMore` and `data.pagination.nextCursor`. Each search endpoint states its default and largest page size, and what happens to a larger one.

### Filtering, sorting and selecting fields

A collection takes filters, a sort order and a field selection in the query string. Its endpoint lists which fields support which filters, and which can be sorted. A filter or sort on any other field is refused with `400`, never ignored.

Filters combine with AND: a record is returned only if it matches every filter. Each parameter can appear once; repeating one, such as `sort` or the same filter, is refused with `400`.

|  |  |  |
| --- | --- | --- |
| `Exact` | Match a value, or everything except it. | `filter[field]=value filter[field][ne]=value` |
| `Range` | Bound it from above, below, or both. gte and lte include the bound, gt and lt exclude it; between takes exactly two values and includes both. | `filter[field][gte]=value filter[field][lte]=value filter[field][gt]=value filter[field][lt]=value filter[field][between]=a,b` |
| `Any of` | Match any value in a list, or none of them. Up to 100 values. | `filter[field][in]=a,b,c filter[field][notIn]=a,b,c` |
| `Pattern` | Match text against a case-sensitive pattern: % stands for any characters, _ for exactly one. %25 is the URL spelling of %; where a client encodes parameters for you, pass % itself. | `filter[field][like]=%25part%25 filter[field][like]=part%25` |
| `Sort` | Applied in the order given; a leading - sorts descending. | `sort=field,-otherField` |
| `Trim` | Return only these fields. The rest are omitted, not null. | `fields=a,b,c` |
| `Page` | Up to 100 records per page, 25 unless asked. Pass back pagination.nextCursor for as long as a response carries one. | `page[size]=25&page[cursor]=…` |

A `like` pattern works on text fields only and is case-sensitive: `%` matches any sequence of characters and `_` matches exactly one. In a URL, `%` is written `%25`, so `%25part%25` finds values containing “part” and `part%25` values starting with it. A pattern with neither must match the whole field.

The values of `in`, `notIn` and `between` are comma-separated in one parameter, so a value cannot itself contain a comma. `in` and `notIn` take up to 100 values, and `between` exactly two.

URL-encode every value you put in the query string yourself: a literal `+` as `%2B`, or it arrives as a space, and a `%` in a pattern as `%25`. Where your client encodes parameters for you, pass the characters themselves and let it encode them; encoding twice sends `%2525`, which matches nothing. The brackets in parameter names can be sent as they are; with curl, pass `-g` so that it does not treat them as a pattern.

On a collection's endpoint, the chips beside each field show what it supports; hover over one to see its parameter names.

## Examples

**Reading every page, in Python**

```
params = {"page[size]": 100}
while True:
    body = requests.get(url, params=params, headers=headers).json()
    handle(body["data"])
    cursor = body["pagination"].get("nextCursor")
    if cursor is None:
        break
    params["page[cursor]"] = cursor
```

**A page of a collection**

```json
{
  "data": [
    "…"
  ],
  "pagination": {
    "hasMore": true,
    "nextCursor": "…"
  }
}
```