[{"id":"IL-0171","title":{"en":"Karpenter node auto-repair replaced EKS GPU nodes about 10 minutes after they became Ready, in a loop, because the Bottlerocket NVIDIA AMI lacked libdcgm.so and set AcceleratedHardwareReady to False","ko":"Bottlerocket AMI 에 libdcgm.so 가 없어 GPU 노드의 AcceleratedHardwareReady 가 False 가 됐고, Karpenter 노드 자동 복구가 Ready 뒤 10~11분마다 GPU 노드를 교체하는 일을 끝없이 되풀이했다."},"org":"Inflab","org_slug":"inflab","date":"2025-08-27","severity":"unknown","root_cause":"software-bug","url":"https://benchlet.com/incidentlib/app/cases/IL-0171","shared_components":["node-image"]},{"id":"IL-0078","title":{"en":"Enabling EKS secrets encryption with KMS through Terraform removed worker node IAM permissions and took down all ZenML Pro workspaces","ko":"EKS 업그레이드 중 Terraform 으로 KMS 시크릿 암호화를 켜자 워커 노드의 IAM 권한이 빠져 운영 ZenML Pro 워크스페이스에 모두 접속할 수 없게 됐다"},"org":"ZenML","org_slug":"zenml","date":"2025-05-11","severity":"unknown","root_cause":"software-bug","url":"https://benchlet.com/incidentlib/app/cases/IL-0078","shared_components":["node-pool"]},{"id":"IL-0161","title":{"en":"A faulty earlier API update made new LKE cluster provisioning in Toronto time out for three days, leaving clusters with no node pools or the wrong number of nodes","ko":"앞선 API 업데이트가 의도대로 동작하지 않아 토론토 데이터센터에서 새 LKE 클러스터 프로비저닝이 타임아웃으로 사흘 넘게 실패했다."},"org":"Linode","org_slug":"linode","date":"2024-04-22","severity":"unknown","root_cause":"software-bug","url":"https://benchlet.com/incidentlib/app/cases/IL-0161","shared_components":["node-pool"]},{"id":"IL-0112","title":{"en":"The daily node rollover put DFDS's EKS nodes on an AMI later recalled for a kernel issue, causing \"PLEG is not healthy\" NodeNotReady flaps and pod network timeouts","ko":"나중에 회수된 EKS 노드 AMI 로 노드를 교체한 뒤 노드가 PLEG is not healthy 를 보고하며 NotReady 를 오갔고 파드 네트워크 타임아웃이 났다"},"org":"DFDS","org_slug":"dfds","date":"2022-10-31","severity":"unknown","root_cause":"software-bug","url":"https://benchlet.com/incidentlib/app/cases/IL-0112","shared_components":["node-image"]},{"id":"IL-0146","title":{"en":"New AKS clusters with Azure CNI in Central India lost tunnel connectivity because VMs began getting MAC addresses with a 06 prefix, which exposed a dormant bug in bridge-mode Azure CNI","ko":"Central India 에서 새로 만든 Azure CNI 클러스터의 VM 이 높은 값의 MAC 주소를 받으면서 bridge 모드 Azure CNI 의 오래된 버그가 드러나 터널 연결이 끊기고 kubectl exec 와 파드 간 통신이 타임아웃됐다."},"org":"Microsoft Azure","org_slug":"microsoft-azure","date":"2020-12-04","severity":"unknown","root_cause":"software-bug","url":"https://benchlet.com/incidentlib/app/cases/IL-0146","shared_components":["node-image"]},{"id":"IL-0029","title":{"en":"Slow user-facing services from CPU throttling on containers that used far less CPU than their limit","ko":"CPU 한도를 건 컨테이너가 사용량이 한도에 한참 못 미치는데도 스로틀링되어 사용자 대면 서비스의 지연 시간이 늘었다."},"org":"Buffer","org_slug":"buffer","date":"2020-08-31","severity":"unknown","root_cause":"software-bug","url":"https://benchlet.com/incidentlib/app/cases/IL-0029","shared_components":["node-image"]},{"id":"IL-0190","title":{"en":"Cortex distributors kept a stuck connection to a dead etcd leader after a Kubernetes upgrade terminated its node, failing writes to Hosted Prometheus for 12 minutes","ko":"쿠버네티스 업그레이드로 etcd 리더가 있던 노드가 갑자기 종료된 뒤, Cortex distributor 두 개가 옛 리더와 맺은 TCP 연결이 멈춘 채 그 연결을 계속 써서 런던 리전 Hosted Prometheus 쓰기가 12분 동안 일부 실패했다."},"org":"Grafana Labs","org_slug":"grafana-labs","date":"2020-03-16","severity":"unknown","root_cause":"software-bug","url":"https://benchlet.com/incidentlib/app/cases/IL-0190","shared_components":["node-pool"]},{"id":"IL-0143","title":{"en":"GKE nodes upgraded to certain 1.13 and 1.14 versions had an elevated number of kernel panics, fixed by releases that downgraded the Container-Optimized OS image","ko":"GKE 1.13 과 1.14 의 특정 버전으로 업그레이드한 노드에서 커널 패닉이 늘었다."},"org":"Google Cloud","org_slug":"google-cloud","date":"2019-11-04","severity":"unknown","root_cause":"software-bug","url":"https://benchlet.com/incidentlib/app/cases/IL-0143","shared_components":["node-image"]}]