[{"id":"IL-0133","title":{"en":"A storm shut down cooling at Nebius's us-central1 data center, and a disk hot-plug bug kept Managed Kubernetes control planes down for hours after power returned","ko":"폭풍으로 데이터센터 냉각이 멈춰 us-central1 리전 전체가 내려갔고, 디스크 핫플러그 버그로 Managed Kubernetes 컨트롤 플레인 약 140개의 복구가 길어졌다."},"org":"Nebius","org_slug":"nebius","date":"2026-08-19","severity":"unknown","root_cause":"dependency-failure","url":"https://benchlet.com/incidentlib/app/cases/IL-0133","shared_components":["infiniband"]},{"id":"IL-0023","title":{"en":"A training run crashed with an NCCL timeout after one InfiniBand port went down once and recovered","ko":"InfiniBand 포트 하나가 한 번 내려갔다 복구되면서 7시간째 돌던 학습 작업이 NCCL 타임아웃으로 죽었다"},"org":"Databricks","org_slug":"databricks","date":"2026-07-01","severity":"unknown","root_cause":"dependency-failure","url":"https://benchlet.com/incidentlib/app/cases/IL-0023","shared_components":["infiniband"]},{"id":"IL-0134","title":{"en":"A power event at Lambda's AUS01 data center rebooted 246 GPU nodes, all spine switches and the VAST storage, failing customer workloads and leaving stale NFS mounts on VMs","ko":"AUS01 데이터센터에 전력 문제가 생겨 GPU 노드 246대와 스파인 스위치, VAST 스토리지가 한꺼번에 재부팅됐고, 워크로드가 실패하고 NFS 마운트가 끊겼다."},"org":"Lambda","org_slug":"lambda","date":"2026-02-14","severity":"sev1","root_cause":"dependency-failure","url":"https://benchlet.com/incidentlib/app/cases/IL-0134","shared_components":["infiniband"]},{"id":"IL-0025","title":{"en":"Failed boots, InfiniBand link flapping, NCCL hangs and throughput sags while bringing up a 4,088-GPU H100 training cluster","ko":"H100 4,088개 클러스터를 구축하는 동안 InfiniBand 배선 오류, 링크 플래핑, GPU 와 PCIe 하드웨어 결함이 학습을 멈추거나 늦췄다"},"org":"Imbue","org_slug":"imbue","date":"2024-06-25","severity":"unknown","root_cause":"dependency-failure","url":"https://benchlet.com/incidentlib/app/cases/IL-0025","shared_components":["infiniband"]},{"id":"IL-0475","title":{"en":"Azure A100 InfiniBand all-reduce bandwidth dropped when over half of ToR switches' redundant uplinks were down","ko":"Azure A100 InfiniBand 클러스터에서 ToR 스위치의 이중화 업링크가 절반 넘게 끊겨 2노드 all-reduce 대역폭이 크게 떨어졌다"},"org":"Microsoft Azure","org_slug":"microsoft-azure","date":"2024-06-08","severity":"unknown","root_cause":"dependency-failure","url":"https://benchlet.com/incidentlib/app/cases/IL-0475","shared_components":["infiniband"]},{"id":"IL-0126","title":{"en":"The OPT-175B training run restarted more than 40 times in two weeks as cloud GPU nodes failed with ECC errors, replacement nodes came back faulty, and checkpoint downloads hung","ko":"OPT-175B 학습이 클라우드 노드의 ECC 메모리 오류, 같은 결함을 안고 돌아온 교체 노드, 멈추는 체크포인트 다운로드 때문에 2주 동안 40번 넘게 재시작했다."},"org":"Meta AI","org_slug":"meta-ai","date":"2021-12-03","severity":"unknown","root_cause":"dependency-failure","url":"https://benchlet.com/incidentlib/app/cases/IL-0126","shared_components":["infiniband"]},{"id":"IL-0397","title":{"en":"Orders on one exchange connection hung in pending states as its NSE leased line failed and the backup line kept flapping","ko":"거래소 NSE 로 가는 CTCL 하나의 주 전용 회선이 끊기고 예비 회선도 끊겼다 붙기를 되풀이해, 그 CTCL 을 지난 주문이 거래소에 들어가지 못하고 대기 상태에 머물렀다"},"org":"Zerodha","org_slug":"zerodha","date":"2018-04-12","severity":"unknown","root_cause":"dependency-failure","url":"https://benchlet.com/incidentlib/app/cases/IL-0397","shared_components":["network-link"]}]