Skip to content

API operations and incremental reads

Use these interfaces to follow changed records and inspect operations data with an authorized token. Historical additions are recorded in the changelog.

Only what changed

A caller that asks for the same data again and again (a page that refreshes, a routine that follows a collection) can ask for what was written since it last asked.

Collections: since

GET /api/collection/<database>/<period>/<source>/<name>?since=2026-10-07T12:00:00Z
  • Only the documents written after that time: their mtime, which every write sets and which is indexed. It adds to the query the caller gives.
  • They come in the order they changed, per_page at a time (page is not used). Two response headers: Latest-Mtime, the time to ask from next, and More-Changes, 1 while more wait. 204 when nothing changed.
  • A page never ends among documents written in the same millisecond (asked again from that time, the rest of them would be skipped): it stops before such a group, or holds all of it.
  • With a sort of the caller's the documents are only filtered, and paged as always.
  • Every collection read now carries Latest-Mtime (the newest of what it returned), so a caller can start following after a full read. To be safe, follow from the time the full read began.
  • From Python: documents, latest = ClientAPI.get_collection_changes(database, period, source, name, since, query=None) reads all the pages; pass latest as since next time. ClientAPI.get_collection(..., since=...) passes it through.

Tables: last and since

GET /api/table/<database>/<period>/<source>/<name>?last=100
GET /api/table/<database>/<period>/<source>/<name>?since=2026-10-07T12:00:00Z

A table filters fast by its date index, by the tail and by mtime; any other column is compared row by row (query, equality only). So these two, and no range filters on other columns:

  • last=N: the last N rows of what the other filters leave (rows are kept in the order they were written, so these are the newest). Without other filters the tail is read directly, whatever the size of the table.
  • since=<time>: only the rows written after it, by each row's own mtime. When the table was not written since (its header says so), 204 is answered without reading a row. Otherwise one column is compared.
  • Both combine with startdate, enddate, symbols, portfolios, tags, query; last is applied after them.
  • Latest-Mtime response header: the newest mtime among all the rows that matched (with since) or among the rows of the page. With since, it is emitted only on the final page. Keep the original since and filters while following the Next-Page header; advance the watermark only when Since-Complete: 1 is returned. Earlier pages return Since-Complete: 0. This is page-based traversal, not a snapshot across concurrent writes.
  • From Python: ClientAPI.get_table(..., last=100), ClientAPI.get_table(..., since=time).

The same limit as collections: a row written back with the mtime it had is not seen as changed.

Operations data with a token

What the WebUI shows a signed-in master, for a script. Master tokens only, read-only, arguments always masked.

Address What
GET /api/schedules/<name> Today's routines of a schedule in run order: status, level, last message and its age, start and end, what each waits for. Filters: status=A,B level=error,starting realtime=true|false computer= q=. counts is of the whole schedule.
GET /api/workers Every worker enrolled here or heard in the last 15 minutes, as its heartbeat says. Filters: alive=true|false q=.
GET /api/jobs Batch jobs, newest first: the active ones, or month=YYYYMM once the jobs are in the job store. Filters: status= q= group=<key>; limit, offset.
GET /api/jobs/groups Their groups (by=time|routine|status|computer) with how many jobs of each status.
GET /api/logs?offset=<bytes> To follow a log: the records appended after that offset, oldest first, with the offset to ask from next. (Without offset, as before: newest first, limit and skip.)

GET /api/logs and GET /api/schedules (who sends each schedule's commands) existed already.

Editing scheduled routines in the WebUI

SCHED's Realtime and Batch tabs show the full user@computer target and each has an Add / edit routine card. Select an existing definition to edit or delete it, or add a routine using the Python-script suggestions from installed repositories, SharedData modules and saved schedules. Remote-only script paths can be typed manually.

The master-only /api/web/sched/definitions endpoint reads a schedule with GET (?schedule=MASTER) and creates, edits or deletes a routine with POST, PUT or DELETE. Writes require the page CSRF token and the revision returned by GET; concurrent changes return 409. Arguments omitted from an edit retain their saved value, including masked secrets. Invalid dependencies, cycles, duplicate keys and invalid run times are rejected. GET /api/web/sched/scripts supplies autocomplete suggestions.

An updated Scheduler applies definition edits on its next tick while keeping unchanged run instances' status and dispatch markers. Due new/rescheduled routines may start then; editing arguments does not restart an existing process. Removing a definition does not kill its process. A new schedule must still be assigned to a Scheduler. Deploy the WebUI and Scheduler changes together; an older Scheduler only reloads at startup or day change.

Tests

tests/test_api_collection_since.py (a real MongoDB), tests/test_api_table_last_since.py (a real table, and ClientAPI over HTTP), tests/test_api_operations.py.