How to Fix MongoDB Error 91331 (RetriableRemoteCommandFailure)

Published
Length
3 min read

Luis Felipe Dias

All posts

You get mongodb error 91331 when an internal command to a remote node fails. You fix it by stabilizing node links and letting the client retry the run. MongoDB names this failure code RetriableRemoteCommandFailure in its official error catalog. The engine groups this code under both the Interruption and RetriableError categories. Because it is a retriable error, a brief network drop often causes it. Your team does not need to rebuild the cluster to fix this problem. Instead, you can trace the n

Your coding agents run with your access.

Claude Code reaches prod with a developer's credentials, and approval prompts get clicked through. hoop.dev checks every query, masks sensitive data, and holds risky changes for a human.

You get mongodb error 91331 when an internal command to a remote node fails. You fix it by stabilizing node links and letting the client retry the run.

MongoDB names this failure code RetriableRemoteCommandFailure in its official error catalog. The engine groups this code under both the Interruption and RetriableError categories. Because it is a retriable error, a brief network drop often causes it. Your team does not need to rebuild the cluster to fix this problem. Instead, you can trace the node drop and verify your app retry logic.

What causes mongodb error 91331 in a cluster

In a distributed database, nodes send remote commands to other nodes all the time. A router node might ask a shard node to read or write records. Replica set members also talk to each other to sync state and elections. When one of these remote calls hits a temporary glitch, the command stops. The receiving node or the network path marks the failed call as retriable. That event triggers mongodb error 91331 and returns the RetriableRemoteCommandFailure signal.

Several common cluster events can trigger this interruption during daily runs. A primary node might step down while a remote command is in flight. A brief network blip between two data centers can drop a socket. High CPU load or disk stalls can also delay a remote node reply. When the wait takes too long, the calling node cancels the remote request. Because the engine labels this an interruption, it tells the caller to try again.

How to diagnose and fix the failure safely

Start your check by reading the server logs on the calling and target nodes. Look for network drops, primary stepdowns, or slow disk alerts at that exact time. Next, check the health metrics of the remote node that failed the call. If one shard or replica member has high load, balance your query traffic. Fixing the root network or resource bottleneck stops the remote command from failing.

Next, check how your app and database driver handle the RetriableError category. Modern drivers can retry failed reads and writes on their own when set up. Make sure your app connection setup turns on retry logic for transient faults. Also check that your app timeouts leave enough room for a second try. If a timeout is too short, the app gives up before the retry works.

MongoDB Authentication & Authorization + End-to-End Encryption: architecture patterns and best practices

Continue reading? Get the full guide.

Free. No spam. Unsubscribe anytime.

Keeping production checks safe during an incident

When engineers inspect a live cluster during an outage, they need safe access. Do not hand out full admin keys just to check node stats. MongoDB grants access through role-based authorization on a specific database. A role can grant access down to a collection level of granularity. For most incident checks, the built-in read role is enough for the team.

The read role lets users run find, listCollections, listIndexes, dbStats, and collStats. It does not allow insert, update, remove, or dropCollection actions on your data. This keeps engineers from changing records by mistake while they debug error 91331. Reserve the readWrite role for tasks that must create indexes or modify data. If sensitive fields like emails, phone numbers, or card numbers exist, use data masking. Masking rewrites matched fields in query results during live checks.

Inspect MongoDB wire traffic in flight

Built-in MongoDB roles govern collection access at setup time. They cannot score command risk per call or mask sensitive document values on the fly. By design, hoop.dev sits inline in the MongoDB protocol next to your server as a sidecar for your MongoDB. On every production command from mongosh, drivers, or AI agents, hoop.dev decides whether to pass, mask, block, or hold it for human review.

First, hoop.dev guardrails block bad deletes, drops, and write pipelines or hold them for approval. Next, hoop.dev data masking rewrites sensitive values in returned documents in memory and logs each decision. Explore the repo on GitHub.

Frequently asked questions

What does RetriableRemoteCommandFailure mean in MongoDB?

It means a command sent to a remote cluster node failed due to an interruption. Because it is in the RetriableError group, the caller can safely try the command again.

Does mongodb error 91331 mean my database lost data?

No, this error code only shows that a remote call stopped before finishing. Your app or the internal cluster router can retry the operation to complete the task.

Which MongoDB role should engineers use to check collection stats?

Give engineers the built-in read role on the target database during an incident. This role allows dbStats, collStats, listIndexes, and find actions without allowing data changes.

The query runs. The emails stay hidden.