1

Our k8s cluster uses VMware VM as worker nodes. K8s version is 1.22.9, network is flannel, kube-proxy mode is iptables.

A worker node with storage was moved with vMotion. After the migration is complete, the pod ip on the worker node cannot be accessed. 50% of TCP connections to nodeport 30800 of this worker node cannot be established. No response was received after SYN packets were sent.

The nodeport svc is ingress-nginx, nodeport is 30800.I cannot access port 30800 of this worker node, but access 30800 of other worker nodes is OK.

The malfunction lasted for about 1 hour and automatically recovered without any intervention.

I suspect that vmotion may have caused the malfunction, but we don't know the exact cause.

The ping test network was almost not interrupted during the vMotion process, and ports 22 10050 of the VM could be accessed normally after the migration. I suspect that the migration caused problems in kube-proxy or flannel.

I tested it in a test environment, but it was not repeated.

Improve this question
New contributor
zys is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
3
  • 3
    If you only suspect it, do a manual vmotion and validate your assumption. Monitor the logs while you do so.
    – Gerald Schneider
    Dec 4 at 6:59
  • I think you have a misconfiguration of either you physical or virtual networking that the vMotion is simply showing up. Try something like a ping-loop to the moving VM, see if once it moves you lose the pings - if so then perhaps it's an issue with the rARPs that the 'receiveing' ESXi host sends out at the then of the vM process. If that's what happens then the next step will be to look at the MAC/CAM/IP tables on the switches to see if and where the MACs and IPs move.
    – Chopper3
    2 days ago
  • You should be able to manually vMotion a guest at any time. You should have already done this many times, while the guest is heavily loaded, as part of testing. If, you have not, and aren't going to test it further, you should prevent it from being moved by vMotion. vMotion is a fundamental aspect of moving guests around, so don't turn it on unless you are prepared to test and follow the necessary steps.
    – Greg Askew
    2 days ago

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .