[{"id":"IL-0191","title":{"en":"A label selector accidentally added to the Loki distributor Service matched only the new pods of a rollout, which took all write traffic, were OOM-killed and stalled the rollout","ko":"Loki distributor 서비스에 레이블 셀렉터가 하나 더 들어가 롤아웃 중 새 파드 몇 개에만 쓰기 트래픽이 몰렸고, 그 파드들이 OOM으로 죽으면서 로그 쓰기 경로가 약 25분간 멈췄다."},"org":"Grafana Labs","org_slug":"grafana-labs","date":"2022-09-01","severity":"unknown","root_cause":"config-change","url":"https://benchlet.com/incidentlib/app/cases/IL-0191","shared_components":["memory-limit"]},{"id":"IL-0009","title":{"en":"A duplicated generated name made Argo CD replace the InfluxDB Cloud production Application and delete the whole production environment","ko":"ArgoCD Application 이름이 겹쳐 InfluxDB Cloud 운영 환경 하나가 통째로 삭제됐다"},"org":"InfluxData","org_slug":"influxdata","date":"2022-06-24","severity":"unknown","root_cause":"config-change","url":"https://benchlet.com/incidentlib/app/cases/IL-0009","shared_components":["persistent-volume"]},{"id":"IL-0035","title":{"en":"Worker nodes went NotReady after a new ElastAlert rule without memory limits used up node memory","ko":"ElastAlert 에 새로 넣은 알림 규칙 때문에 ElastAlert 파드가 메모리를 끝없이 써서 노드가 잇달아 NotReady 가 되었고, 같은 클러스터의 도구와 Grafana, Prometheus 가 함께 흔들렸다."},"org":"NU.nl","org_slug":"nu-nl","date":"2019-02-01","severity":"unknown","root_cause":"config-change","url":"https://benchlet.com/incidentlib/app/cases/IL-0035","shared_components":["memory-limit"]}]