I can get the targets working fine if I am on my local network or if I use my VPN connection so I am not sure if mt NIC is the cause. My internet speeds are 8Mb down and 1Mb up so again I should be allright.
I just can't get a direct connection while using the public IP. I want to use the public IP becauase the remote machines are going to backup important account data to my server and the VPN makes the connection quite slow. Also I don't want the servers bouncing all their internet requests through my machine!