this is a network related issue. not sure what specifically could be causing this, bad cable, many lost packets from ISP, highly doubt the network card went bad but sure thats a possibility. basically this happens when your packets sequence is out of order. Couldnt honestly even be the workstation your sshing in from (client issue).
If it were any of my computers, then it would be a problem on both MiaB servers and it surely wouldn’t be happening on machines local to my rack in the data center and two different machines at home.
If it were my ISP or my data center then it would be happening to both MiaB servers. My other MiaB stays up for weeks on end without issue.
Thank you for the input. Definitely going to bring a new ethernet cable with me tonight. I really hope it is not the network card.
I assume it isn’t because I am still on Ubuntu 18.x…but if it is, that is fine as I am doing this to move to Ubuntu 22.04.
So after an all night Monday night installing Ubuntu 22.04 from scratch and moving 125 gigs of mail and backups from the server to another server and then back after the upgrade and no problems at home, I put it back in the data center and broken pipe immediately.
So I brought the server home and did an all-nighter Tuesday night to repeat the process but move everything to a different server. However, the new set up had started to run rather slowly and got stuck on the found 1.5 million emails in the trash of an account that has a filter that deletes the messages (well no it actually doesn’t!). In the process I found two more automated accounts with similar filters that also had nearly a million messages each. Finally figured out how to delete the millions of messages (sudo rm * failed because there were too many files!) and then deleted all the backups for the last 90 days and then compressed and moved the files.
Took the new server to the data center but put it on a different IP address because I didn’t want to deal with the possibility that it was a firewall issue (that I had already spent hours trying to find but couldn’t).
So I am leaning against the rack, using my phone to remote to my house to remote into the data center to confirm all was well and I look up and I notice that a MacMini that was decommissioned probably two years ago but I left it in the data center because I had no use for it at home and well maybe I would come up with something for it was turned on! it turns out that the data center had a power problem on a circuit that half my rack was on and they had to swap me over to a new circuit a few weeks ago and half my machines got powered off. When powered was restored, this MacMini apparently also came back on. it was previously set to the same IP address as my email server (new since then). It was a fixed address machine and since then I moved to all DHCP. My guess is the MacMini was indicating a competing device was on the network and what did I want to do about it.
I turned it off and I am not going to use that internal IP address either in the mean time. Going to remove it next time I go to the data center.
Thanks to all of the folks that gave suggestions and made other efforts to help me on this. Much appreciated.