Todd -
Thanks again for taking time last night to help me out. I have done what you suggested and killed my bond, changed the IP of the iscsi interface to a different subnet, and then setup the virtual IP again.
Now, what I noticed was the speed tests were a little more consistent with each other after about 20 minutes (a lot of traffic initially because the VM's were down for about an hour). Of course, if I ran more than one test at a time the gig iscsi interface on the SAN couldn't keep up. I was seeing an average of ~105MB/s seq read, ~55MB/s seq write on both the RAID6 external lun and the RAID10 internal LUN.
I then created a balance-rr bond with all 4 of the Intel GIG iscsi interfaces. The tests were a little better than the 802.3ad, but not as "bursty." Once again, the speeds vary from lun to lun and when they are run. I have seen the low 90's on the read and even in the 70's on the writes. I think were getting closer. I will let it run more during the day before sending some logs. I have uploaded the logs from both servers when it was in a single, no bond, gig interface.
I'll keep you posted and possibly get some performance numbers once I can find some consistency.
Jason - Thanks for the update, seems we are getting closer. The bonds will start kicking in when there are many systems or requests coming then it will do better. You can also try the ALB bond. I heard that is doing well. Looks like we will work on the 802.3ad now.