What is the build of your Xenserver? is it the latest? as your xen might have an issue with your NIC drivers, did you checked for that!