Cloud Firestore

Fireemu serves Cloud Firestore over gRPC, REST and the WebChannel transport the Web SDK uses, with Firestore Security Rules. Connect to it at 127.0.0.1:8080 (see Connect your app).

Editions and API modes

Select the database edition and API mode in fireemu.toml. fireemu init writes the Standard edition with the Native API:

TOML
JSON

schemaVersion = 1

[firestore]
edition = "standard"
apiMode = "native"

{
  "schemaVersion": 1,
  "firestore": {
    "edition": "standard",
    "apiMode": "native"
  }
}

edition accepts standard and enterprise. The Enterprise Pipeline API supports a limited subset of read operations; see Enterprise pipelines and Limitations.

Enterprise pipelines

On an Enterprise database, fireemu runs pipelines from the Admin SDK (gRPC) and the Web SDK’s firebase/firestore/lite/pipelines (REST :executePipeline). The supported subset is a collection input followed by, in this order, an optional where, an optional select, an optional offset and an optional limit.

Expression	Supported
Fields	Nested paths (`a.b`) and `__name__` (the document reference)
Literals	Values of every type, including arrays and maps of literals and vectors of doubles
Comparisons	`equal`, `not_equal`, `less_than`, `less_than_or_equal`, `greater_than` and `greater_than_or_equal`, anywhere an expression stands
Construction	`array(...)` and `map(...)`
Document functions	`documentId`, `collectionId` and `parent`

A where keeps a row only when its condition (a comparison, a field or another supported function) is true. A select sets at most 64 fields, as in production. Results without a select carry the document’s name, create time and update time, and every answer carries the execution time. A stored -0.0 reads back as -0.0, as in production; the official emulator stores it as 0.0.

The answers follow production as recorded by fireemu’s production observations (comparisons across value types, vectors and missing fields, integer and double values compared exactly, conditions of every kind, literal values and map keys, select alias rules and the alias length limit, document functions, map(...), limit and offset arguments, and the refusal texts and orders production returned). Under strict, a refusal is answered only when the request has the shape of a request production refused, with that request’s text, and is UNIMPLEMENTED otherwise. A shape keeps the stages, function names, value kinds and every map’s entries in order, and replaces only plain field names and values of the same kind, so select({total: documentId(field)}) is answered like the recorded select({v: documentId(field)}). A REST body keeps its key order; over gRPC a map of two or more entries has no order, so strict answers no refusal of such a request there. Where production has not been observed, the profiles differ:

Input	`strict`	`emulator`
A document function of a collection reference with a non-ASCII name	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, the index counted in UTF-16 units
A literal holding a field or function under a map key that is not a plain name	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, naming a reserved key first
A document function of a double, timestamp, geo point, bytes, array or vector	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, naming the type
A `map(...)` key that is a double, null or another type than an integer or boolean	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, naming the type
A vector literal with integer or NaN elements, no element, without `value` or with another key	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, the official text
A vector literal with an infinite element	`UNIMPLEMENTED`	The vector
A `__type__` key that is not a string, or a reference literal that names no document	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, the official text
Two refusals in one pipeline	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, one of them
A refusal production recorded only in another stage, such as a literal holding a field inside a `where` condition	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, the official text
A vector literal with a negative-zero element	`UNIMPLEMENTED`	The vector
A literal with more than one empty or reserved key, or the key `__name__`	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, naming one key
A `limit` of another type than an integer or string; an `offset` that is negative or of another type than an integer or string	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, the official text
A `limit` of 0, or an `offset` of 0 up to 2147483647 (production answered only larger offsets)	`UNIMPLEMENTED`	The rows
A refusal after a stage `strict` does not run past, such as an empty `select` after an `aggregate`	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, the official text
A `select` alias inside another in another order than production’s `{m, m.r}`, such as `{a.b, a}`	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, naming the later alias over REST (as the official emulator does) and the child over gRPC
Two `select` fields that both fail when evaluated, such as `{a: documentId("s"), b: parent(true)}`	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, the first in key order
A vector inside an array, or a `__type__` key beside other keys in a literal	`UNIMPLEMENTED`	The official answer
The wrong number of arguments for a supported function	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, fireemu’s text
Function options	`UNIMPLEMENTED`	Ignored, as the official emulator ignores them
An empty `select` map or a later stage’s check after a function or stage value fireemu does not check (production names an unknown function first)	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, the empty map or the later check
An empty map or another refusal beside a function call that fails on its constants, such as `map(1.5, i)`, or beside any other part `strict` does not answer	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, the empty map or the other refusal
A stage’s own argument count or options, such as `limit()` or `where(a, b)`	`UNIMPLEMENTED`	`INVALID_ARGUMENT`, fireemu’s text, for `collection`, `where`, `select`, `limit`, `offset`, `add_fields`, `distinct` and `aggregate`; `UNIMPLEMENTED` for every other stage
An empty `array()` or `map()`	`UNIMPLEMENTED`	An empty array or map

| An invalid, empty or 1500-byte alias in add_fields, distinct or aggregate, or an empty alias map of select, add_fields, distinct or aggregate | UNIMPLEMENTED | INVALID_ARGUMENT with the official emulator’s text, in its order (every alias before an empty map), before the stage is refused as unsupported | | An aggregate without its groups argument | INVALID_ARGUMENT with production’s arity text for the recorded shape (directly after the collection, a non-empty map of plain aliases to count), else UNIMPLEMENTED | INVALID_ARGUMENT with the same text |

Every other function and stage is refused with UNIMPLEMENTED, after the argument checks in the table above, never ignored.

Composite indexes

Production refuses a query whose composite index is not defined. The official emulator assumes every index exists, so a missing index is usually found only after deploying. Fireemu checks each query against the indexes declared in firestore.indexes.json (referenced from firebase.json), as production does.

`firestore.missingIndexes`	Behavior
`refuse` (default)	The query is refused with `FAILED_PRECONDITION` and the console link, as production refuses it.
`report`	The query runs. Each missing index is reported once per database on standard error, with production’s refusal text and the `firestore.indexes.json` entry to add.

The emulator profile always runs the query, as the official emulator does, while reporting missing-index suggestions on standard error.

An OR query can require a separate composite index for each branch. Since v0.26.0, supplementary diagnostics list all distinct missing branch indexes in one attempt. Under strict with refuse, the SDK still receives its existing FAILED_PRECONDITION response and first creation link; the additional branch definitions are on standard error. Under report, the control API report also collects the distinct missing definitions.

Fail a CI job on a missing index

The report on standard error is easy to miss in a CI log. Under strict with "report", the control API also lists every index that was missing, so a job can fail on a non-empty list. Turn the report on:

TOML
JSON

schemaVersion = 1
profile = "strict"

[firestore]
missingIndexes = "report"

{
  "schemaVersion": 1,
  "profile": "strict",
  "firestore": {
    "missingIndexes": "report"
  }
}

fireemu exec gives its command FIREEMU_CONTROL_URL (ending in /v1/) and FIREEMU_CONTROL_TOKEN, so the script that runs your tests can check the list when they are done:

npx fireemu exec -- sh -c '
  npm test
  status=$?
  curl -fsS -H "Authorization: Bearer $FIREEMU_CONTROL_TOKEN" \
    "${FIREEMU_CONTROL_URL}firestore/missingIndexes" > missing-indexes.json || exit 1
  [ "$(jq .count missing-indexes.json)" = 0 ] || { jq .indexesJson missing-indexes.json; exit 1; }
  exit $status
'

GET /v1/firestore/missingIndexes answers:

{
  "count": 1,
  "truncated": false,
  "dropped": 0,
  "missingIndexes": [
    {
      "project": "demo-app",
      "database": "(default)",
      "name": "projects/demo-app/databases/(default)",
      "index": {
        "collectionGroup": "notes",
        "queryScope": "COLLECTION",
        "fields": [
          { "fieldPath": "owner", "order": "ASCENDING" },
          { "fieldPath": "created", "order": "ASCENDING" }
        ]
      },
      "message": "The query requires an index. You can create it here: https://console.firebase.google.com/..."
    }
  ],
  "indexesJson": {
    "projects/demo-app/databases/(default)": {
      "indexes": [{ "collectionGroup": "notes", "queryScope": "COLLECTION", "fields": ["..."] }]
    }
  }
}

Each entry is one index per database, in the order it was first needed, with production’s refusal text. indexesJson groups the entries per database in the shape firestore.indexes.json holds, so you can merge the indexes array of a database into that file.
GET /v1/sessions/{session}/firestore/missingIndexes lists only the session’s project; GET /v1/firestore/missingIndexes lists every project. A session reset does not clear the list, so a suite that resets between cases still reports every index it missed, and deleting a session clears its project’s entries.
The list holds at most 1,000 entries. Past that, truncated is true and dropped counts the queries served whose index is not listed. A project whose every index was dropped is answered with 409 and a message that the list is full, never with an empty list, so a check on count cannot pass on an incomplete list. Clear the list and run again.
DELETE on either route clears the list and the dropped count and answers {"cleared": <count>}. The session route clears that session’s project only; the route without a session clears every session’s entries, so a test world that shares a daemon with others should use the session route. An index that is needed again afterwards is reported again.
Where nothing is reported (the emulator profile, strict with "refuse", or Firestore not selected), GET answers 409 with FAILED_PRECONDITION, so a check that depends on the list cannot pass on an empty answer. DELETE still answers {"cleared": 0}.
A page in a browser needs Authorization: Bearer <control token> for both methods, as for the resource report, because the entries name the fields your queries filter and order on.
With fireemu/testing, call world.control('GET', '/v1/sessions/default/firestore/missingIndexes') and world.control('DELETE', ...) on the same path, so the read and the clear stay within your world’s project.

Checks on `firestore.indexes.json`

Under strict, Fireemu refuses at start-up (and keeps the previous indexes when the file changes while it runs) what firebase deploy refuses, and the message names the entry:

A composite index with a single field, such as indexes[3] with only {"fieldPath": "n", "order": "ASCENDING"}. Production answers “this index is not necessary, configure using single field index controls”. Declare it under fieldOverrides instead, which is also how a collection-group query on one field is enabled. (field, __name__ DESC), (__name__ DESC) and a single vector field are accepted, as production accepts them.
An entry without a fields array, which is what a misspelled key such as feilds looks like.
A fieldOverrides entry without an indexes array. A TTL-only override needs "indexes": [].
A fieldOverrides index that has only a vectorConfig: each override index needs exactly one of order and arrayConfig. Declare a single vector field as a composite index instead, as production does.

A missing or empty queryScope is read as COLLECTION, and the older collectionId and field mode keys are read as collectionGroup and order / arrayConfig, as firebase deploy reads them. The limits of at most 200 composite indexes, 100 fields per index and one array field per index apply under both profiles. The Admin API’s indexes.create applies the same checks, and counts the indexes of the file and the ones created at run time together against the 200.

The emulator profile loads the file as before, because the official emulator never reads it.

A search index (an index field with searchConfig, for Enterprise full-text or geo search) loads under both profiles. It is checked as firebase deploy checks it and then left out of query planning, so it never serves or blocks an ordinary query; each time the file loads, a note: line says how many search indexes were left out. Local search execution is not supported. Under both profiles, a field with searchConfig beside another mode, a malformed searchConfig, and searchConfig in a fieldOverrides index are refused with a message that names the field. Under strict, an unknown key inside searchConfig is refused as well, as the Firestore Admin API refuses it.

Query limits

Under the strict profile, the Standard edition query limits are enforced as production enforces them. Setting firestore.enforceLimits to false lets queries pass that production refuses, so leave it on unless you have a specific reason. Under strict the key switches only these query limits (at most 30 disjunctions, 10 in values, and so on): the limits on writes, transactions and indexes, and the wire shapes of the REST and WebChannel surfaces, stay production’s. Under the emulator profile, true also asks for those production limits.

Listen streams

Production ends an end user’s native gRPC Listen stream with INTERNAL an hour after it opened. By default Fireemu keeps the stream open however long it is held (firestore.listenStreamLifetime: "unlimited"). To test a client’s reconnect, either set it to "oneHour" (the default with productionExact), or call POST /v1/firestore/listen:expire on the control API to end every held end-user stream at once with production’s status.

Transactions and history

Like production, Fireemu keeps every version of a document written in the last hour, so a read_time read, a read-only transaction or a resumed listener can see it. Transactions lock what they read, as production does. Starting with v0.27.0, contended gRPC and WebChannel writes wait without occupying runtime workers. Canceling a waiting Commit ends that request’s deadlock wait; its transaction keeps read locks until it commits, rolls back or expires.

A test that updates the same documents many times therefore holds many versions in memory. These local limits, which production does not have, bound it:

Each database retains at most 1 GiB of logical history bytes and 1,000,000 versions, each session 1 GiB and 1,000,000 versions, and the daemon 4 GiB and 4,000,000 versions.
On a pinned clock (daemon.clockStart), a database keeps at most 1,024 versions of any one document.
Starting with v0.27.0, strict Listen retains the latest 4,096 boundary records per live database. A resumed stream echoes the original boundary’s read_time, then sends the new snapshot time with CURRENT. Token bytes are opaque and may change without a write. An expired record or expired document history triggers RESET and a full replay; reset, restore and daemon restart discard the records. This bound is local to Fireemu and is not a production limit.
A commit that would cross a limit is refused whole with RESOURCE_EXHAUSTED and the message Firestore retained history capacity is exhausted.

To use less memory, lower the limits in firestore.history:

TOML
JSON

schemaVersion = 1

[firestore]
edition = "standard"
apiMode = "native"

[firestore.history]
maxVersionsPerPath = 16
maxBytes = 268435456

{
  "schemaVersion": 1,
  "firestore": {
    "edition": "standard",
    "apiMode": "native",
    "history": {
      "maxVersionsPerPath": 16,
      "maxBytes": 268435456
    }
  }
}

maxVersionsPerPath (1 to 1,000,000) sets the most versions of one document a database keeps.
maxBytes (1 MiB to 1 GiB) lowers the logical history bytes one database retains.

Once any document has more versions than the cap, the database keeps no snapshot older than that document’s oldest kept version. An older read_time read is then refused with FAILED_PRECONDITION, a transaction open on an older snapshot fails with ABORTED, and a listener resuming from an older point is sent the full result again. Client SDKs retry ABORTED transactions, so avoid very small caps: choose a cap above the number of times any document is updated while your longest transaction is open.

fireemu doctor --connect <control URL> reports the retained versions and bytes of a running daemon.

Security Rules

Rules are read from the firestore.rules file named in firebase.json, or from rules.source in fireemu.toml. Under strict, a project without rules refuses every client request, as a production project without a rules release does. See Security Rules.

Named databases

Declare each named database in the firestore array of firebase.json, with the rules and indexes files it uses, or create it through the Admin API. Under strict a request to a database that is neither declared nor created gets NOT_FOUND, as in production, and a declared database id must be one production could create (4 to 63 characters of [a-z0-9-], starting with a letter), or Fireemu does not start.

Production governs a database only by its own rules release. So a database created at run time has no Security Rules, and refuses every client request, until you set some for it; the (default) database’s rules do not apply to it. Declare the database in firebase.json with its rules file, or load them at run time:

PUT /emulator/v1/projects/{project}/databases/{database}:securityRules
{"rules": {"files": [{"name": "firestore.rules", "content": "..."}]}}

This is the route PUT /emulator/v1/projects/{project}:securityRules uses for the (default) database, with the same body and answers. The rules set this way last until the database is deleted; they are not part of a snapshot. The emulator profile keeps the official emulator’s behavior: a database that has no rules of its own is served by the (default) database’s rules (rules loaded for it with the route above apply to it).

Production verification

Writes and limits, queries and indexes, transactions, real-time listeners, Security Rules and database administration are verified against recordings from real Firestore. See the compatibility tables.