gce/WARN/2021_005

Serial logs don’t contain out-of-memory messages

Product: Compute Engine
Rule class: WARN - Something that is possibly wrong

Description

The messages: “Out of memory: Kill process” / “sacrifice child” / “Killed process” / “Memory cgroup out of memory” in serial output usually indicate that a Linux instance is under memory pressure.

Remediation

Scale up the instance (add memory) by adjusting the instance’s machineType, check for possible memory leaks in applications.

You can use the following log query to find matching log lines:

resource.type="gce_instance"
log_id("serialconsole.googleapis.com/serial_port_1_output")
textPayload:(
  "Out of memory: Kill process"
  OR
  "sacrifice child"
  OR
  "Killed process"
  OR
  "Memory cgroup out of memory"
  )

Further information