It is rarely a good moment when GET _cluster/health comes back with "status": "red". Somewhere in the cluster, a primary shard has no home, searches against that index start failing, and the clock is already running. This guide walks through exactly how to find out why shards are unassigned and how to get the cluster back to green — without guessing.
Quick answer: run GET _cluster/allocation/explain to see exactly why Elasticsearch left the shard unassigned, fix that root cause (disk space, allocation rules, or a missing node), then run POST _cluster/reroute?retry_failed=true. Treat allocate_stale_primary and allocate_empty_primary strictly as last resorts — both can lose data.
What "Red" and "Unassigned Shards" Actually Mean
Cluster health is a summary of shard state, not a single metric. Green means every primary and replica shard is allocated. Yellow means every primary is allocated but at least one replica is not. Red means at least one primary shard itself is unassigned — which also means the data behind it is temporarily unreadable and unwritable for that shard.
Yellow vs. Red: the difference that matters
A yellow cluster is annoying but usually not urgent: your data is intact, you simply have reduced redundancy while a replica catches up or waits for a node. A red cluster is different. If the affected index is actively serving traffic, requests touching that shard will fail outright. Treat yellow as "monitor it" and red as "stop and diagnose now."
Step 1: Find Out Why With the Allocation Explain API
Before changing anything, ask Elasticsearch why a shard is unassigned. The cluster allocation explain API is built exactly for this:
GET _cluster/allocation/explainCalled with no body, Elasticsearch will explain an arbitrary unassigned shard, which is fine for a first look but not reliable when several shards are unassigned for different reasons. Once you know which index and shard number you care about (from GET _cat/shards?h=index,shard,prirep,state,unassigned.reason&s=state), pass it explicitly:
GET _cluster/allocation/explain
{
"index": "my-index",
"shard": 0,
"primary": true
}Reading the "decisions" array
The response includes a node_allocation_decisions array listing every node Elasticsearch considered, and for each one a deciders breakdown showing exactly which allocation decider said "no" and why — disk space, filtering rules, allocation awareness, shard limits, and so on. This is the single most useful piece of output when a shard is stuck; skipping straight to a reroute command without reading it usually means fixing the wrong thing.
The Causes You Will See Most Often
1. Disk watermark exceeded
By default Elasticsearch stops allocating new shards to a node once it crosses the 85% disk used low watermark, and can even relocate shards away at the higher watermark. If deciders mentions disk_threshold, the fix is disk space, not a reroute command — free up space, add capacity, or adjust ILM to delete/rollover more aggressively.
2. Shard allocation filtering or awareness rules
Index-level index.routing.allocation.* settings or cluster-wide allocation awareness (rack/zone awareness) can legitimately leave a shard unassigned if no node currently satisfies the rule — commonly after a node was decommissioned or relabeled without updating the setting.
3. Not enough eligible nodes for the replica count
If number_of_replicas is higher than the number of nodes that can legally hold a copy (for example, in a single-node development cluster with number_of_replicas: 1), the replica will sit unassigned forever. This one is easy to miss because the cluster otherwise looks healthy.
4. A node left mid-recovery
If a data node drops out of the cluster while shards are relocating or recovering, those shards go unassigned until the node returns or Elasticsearch finishes reallocating them elsewhere. Check GET _cat/nodes and cluster logs for recent departures before assuming the worst.
5. A corrupted or unrecoverable primary
The least common but most serious case: the primary's data is gone or corrupted on every copy Elasticsearch can find. The allocation explain output will typically show NO_VALID_SHARD_COPY or similar. This is the scenario where data loss becomes a real possibility, discussed below.
Fixing It: From Safe to Last Resort
Retry failed allocation first
Elasticsearch gives up retrying a shard after five consecutive allocation failures, to avoid an infinite retry loop. If the underlying cause is already resolved (disk freed, node rejoined), a simple retry often clears it without touching cluster settings:
POST _cluster/reroute?retry_failed=trueManual reroute (with care)
If a shard still will not move, POST _cluster/reroute with an explicit allocate_replica or move command lets you override the automatic allocator for one shard. Always run it with ?dry_run=true first so you can see what would happen before committing.
Only as a last resort: allocating a stale or empty primary
When the allocation explain output confirms no valid shard copy exists, Elasticsearch offers allocate_stale_primary (accept an older copy, losing recent writes) or allocate_empty_primary (accept total data loss for that shard, unblocking the index). These commands exist for a reason, but they are explicitly destructive — reach for them only after confirming there is truly no better copy, and only on the specific shard that needs it.
Preventing the Next Red Cluster
Most red-cluster incidents trace back to a small set of preventable gaps: disk watermarks left at defaults on undersized volumes, allocation awareness rules that were never updated after infrastructure changes, and no alerting on unassigned_shards > 0 before it becomes a page. Watching cluster state at scale is exactly where things get hard to eyeball manually — our Elasticsearch monitoring extension surfaces unassigned shards and their allocation reason directly, and includes a guided fix-allocation flow for the safe retry path above, so you are not memorizing these API calls at 3 a.m.
For a look at how this plays out at real scale — hundreds of terabytes and recurring RED events under heavy indexing — see our DataDome case study on stabilizing a petabyte-scale cluster after repeated RED incidents.
Frequently Asked Questions
Does a yellow cluster mean I lost data?
No. Yellow means every primary shard is allocated and your data is fully readable and writable — only replica redundancy is reduced. Fix it calmly: check why the replica cannot be placed (usually node count or disk) rather than restarting nodes.
Why did Elasticsearch stop retrying a failed allocation?
After five consecutive allocation failures for the same shard, Elasticsearch stops retrying automatically to avoid a hot retry loop. Once you have fixed the underlying cause, POST _cluster/reroute?retry_failed=true tells it to try again.
Can I just delete an unassigned shard to make the cluster green?
You cannot delete an individual shard. Your realistic options are: restore the affected index from a snapshot, delete and rebuild the whole index if the data is reproducible, or — as a genuine last resort — accept data loss for that shard with allocate_empty_primary.
How do I get alerted before the cluster turns red?
Alert on unassigned_shards > 0 and on disk usage approaching the 85% low watermark, not just on cluster status. Both signals appear well before a red event, which gives you time to act while it is still a yellow-level problem.