A Java pod can be OOMKilled while dashboards show 80% of its memory limit because a garbage collection copies live objects faster than any scrape can see. On EKS with SerialGC we watched memory jump from 80% to 96% between two samples 0.12 seconds apart, and then the kernel killed the JVM. The fix is to keep the Java heap inside the container limit. The 80% figure itself is specific to that setup: in local Docker runs SerialGC died higher, and G1 and ParallelGC went to 100%.
What does it look like?
This JVM ran with -Xmx1g in a 384 MiB container and a cache that never evicts, climbing about 5.5 MiB/s. On EKS on 2026-11-03, with lastwords removed from the node, a loop read memory.current from inside the container about every 0.12 s:
16:47:21.556 79%
16:47:21.670 79%
16:47:21.784 80%
16:47:21.904 96%
command terminated with exit code 137
A second data point from 2026-11-04, sampled from the node every 500 ms by lastwords on EKS, shows an earlier jump and the same ending:
2026-11-04T19:45:41.93740477Z,206807040,206839808,402653184,51.4
2026-11-04T19:45:42.437404528Z,268132352,268165120,402653184,66.6
...
2026-11-04T19:45:57.937408836Z,313864192,313896960,402653184,77.9
2026-11-04T19:45:58.437405254Z,317288448,317321216,402653184,78.8
2026-11-04T19:45:58.937407821Z,320647168,320679936,402653184,79.6
(Two rows and the last 3 of timeline.csv: time, working set, usage, limit, percent.) The first jump is 15 points in one sample, with the JVM surviving it. The termination record says OOMKilled at 19:45:59.
Why does the JVM die below the limit?
Because the JVM needs a second copy of its live data for a moment, and the limit does not leave room for it.
With 2 CPUs and 384 MiB, the JVM chooses SerialGC by itself (UseSerialGC = true {ergonomic} in -XX:+PrintFlagsFinal). Serial splits the heap into a young and an old generation. In a leak, nearly everything in eden is still alive at the next young collection, so it gets copied into the old generation. Eden's pages stay resident, and the old generation grows by the same amount.
GC.heap_info from 17 seconds before the kill shows the numbers:
def new generation total 133440K, used 11752K [0x00000000c0000000, 0x00000000c90c0000, 0x00000000d5550000)
eden space 118656K, 9% used [0x00000000c0000000, 0x00000000c0b7a258, 0x00000000c73e0000)
tenured generation total 296432K, used 177857K [0x00000000d5550000, 0x00000000e76cc000, 0x0000000100000000)
(Trimmed to 3 lines, from the same EKS run, right after the 15 point jump: eden is nearly empty.) Three things stand out:
| Number | Value | Why it matters |
|---|---|---|
| Young plus old capacity so far | 133,440K + 296,432K = 419.8 MiB | Above the 384 MiB limit, though not all of it was touched yet. -Xmx1g tells the JVM it has room |
| Eden | 118,656K = 115.9 MiB | 30% of the limit, all of which can need copying at once |
| Free eden at this moment | 118,656K minus 11,752K = 104 MiB | This service allocates about 6.25 MiB/s of payloads, so eden fills in about 17 s |
That last row is the useful one. This snapshot was taken 17 seconds before the kill, and eden was about 17 seconds from full. The working set by then was 305.8 MiB, or 79.6%. Copying 78 MiB crosses 384 MiB, and a full eden holds 116 MiB, nearly all of it live in this leak. This is our arithmetic from one snapshot, and we had no GC log for that cycle.
So the kill point is not a percentage. A young collection is dangerous whenever working set + live data in eden is above the limit.
Is it always 80%?
No, and we had that wrong at first. We ran the source of that 2026-11-04 checkout service 3 times per collector on a local Docker host (Temurin 21.0.13, 2 CPUs, 384 MiB, kernel 7.0 under OrbStack), forcing each collector, with the heap limit above the container limit, and reading memory.current about every 0.09 s. These are local results, not EKS:
| Collector | Last sample before the kill (3 runs) | Largest jump between samples, after the first 3 s | Seconds from first 90% sample to the kill |
|---|---|---|---|
| SerialGC | 92%, 100%, 88% | 15, 16, 15 points | 0.9, 2.6, never reached 90% |
| G1 | 100%, 100%, 100% | 8, 5, 6 points | 7.8, 9.7, 7.0 |
| ParallelGC | 100%, 100%, 100% | 2, 3, 2 points | 17.4, 12.0, 15.9 |
Only Serial made large single jumps. We left out the first 3 seconds because JVM startup alone moved memory.current by 9 to 20 points in every run. Serial was also the one that died short of the limit, in 2 of 3 runs, and it died higher here than the 80% on EKS. The JVM image, kernel and warm-up differed, and we did not isolate which of them moved the number. memory.current also includes reclaimable page cache, so 100% here does not mean the heap filled the limit. Only Serial has been measured on EKS. Treat 80% as what one small Serial heap did there, and measure your own.
The practical reading: small containers get SerialGC by default, because HotSpot needs at least 2 CPUs and about 1.75 GiB to treat a machine as server class and pick G1. Those are the containers where a 90% alert failed in our runs. G1 and Parallel gave 7 to 17 seconds locally. That is enough for an alert to fire, and it is rarely enough for a person to act.
How do you check your own pod?
-
See which collector you have.
kubectl exec <pod> -- java -XX:+PrintFlagsFinal -version | grep -E 'UseSerialGC|UseG1GC|UseParallelGC'. A JRE image hasjavabut nojcmd. This starts a second JVM inside the container, so run it soon after startup and not near the limit. -
Compare the heap limit to the container limit. Look for
-XmxinJAVA_TOOL_OPTIONS. If it is aboveresources.limits.memory, a leak reaches the kernel's limit first. -
Log collections. Add
-Xlog:gc:stdoutfor one run. The collection that kills the JVM never prints, because the line is written when it ends. What you get is the one before it, with the heap size in brackets. A local Docker run of the checkout source showed a 423M heap in a 384 MiB container:[31.753s][info][gc] GC(15) Pause Full (Allocation Failure) 181M->181M(423M) 4.686ms -
Be careful with the histogram. On EKS
jcmd 1 GC.class_histogramtook a container from 55% to a 75 to 76% peak in 4 of 4 runs, because it forces a full GC.GC.class_histogram -allpeaked at 61 to 62%. Near the limit, use-all, which skips that GC and can also count garbage.
What fixes it?
- Drop
-Xmxabove the limit. Use-XX:MaxRAMPercentage, so the JVM throwsOutOfMemoryErrorinside its own heap. We tested which percentages work in the heap dump post. - Fix the leak. A higher limit moves the kill later. See OOMKilled from a slow memory leak.
- Alert lower for small Serial heaps. Given the 80% on EKS, try around 70% of the limit, on
container_memory_working_set_bytes, for containers that run a JVM with a large-Xmx. - Do not trust "time to limit" alerts here. They extrapolate a straight line and miss the jump.
We hit this while building lastwords. Its capture first fired at 79% and the JVM died before answering, so its JVM tiers are now measured against 75% of the limit (capture.jvmGCHeadroom), which puts the captures at 67.5% and 72.75%. For the failure in general, see the OOMKilled runbook.
Verified 2026-11-03 and 2026-11-04 on EKS 1.33 (Amazon Linux 2023, kernel 6.12, t3.large) and local Docker 29.4, Eclipse Temurin 21.0.13. lastwords is built by the team behind Aurora SRE, an open-source AI SRE.