A faulty DAC (Direct Attach Copper) cable on port enp65s0f0np0 of the dual-port Mellanox ConnectX NIC caused continuous link flapping. The mlx5_core driver repeatedly rebalanced the LACP lag map between the two ports every 3-4 seconds, disrupting established Ceph connections.

Each port flap caused the Ceph client on Node 6 to lose its monitor sessions and mark osd4 and osd6 as down. While the OSDs recovered within seconds each time, the repeated cycling caused rbd watch errors (-107 ENOTCONN) and MDS capability timeouts — resulting in elevated I/O latency for VMs whose disks were served by these OSDs.

The LACP bond across two separate switches prevented a full outage — traffic continued flowing through the healthy port — but the constant rebalancing between a good and bad port created the intermittent disruption pattern seen in the logs.

Disabled the faulty port with ifdown enp65s0f0np0, leaving the bond running on the remaining healthy port. All Ceph connections stabilized immediately. The DAC cable will be replaced on the next datacenter visit.