Hi All, Had a Major incident this morning when the ISCSI crashed on DSS V6. it has been running 100% for months serving ESX VM images for Clients and it Crashed out this morning and only can back on after a reboot. If it happens again which im dreading what's around the next corner, I may as well pack my bags due to the hassle it caused can anyone Tell me why the following has occurred and How i can stop it happening again.

Server reports the following errors at 9:05 this morning and instantly access to the ISCSI target failed. I restarted the SAN at 10:02 and the same errors arisen + a few more.


2009-10-21 10:02:14 [<ffffffff8069c592>] ?... (0/4)
2009/10/21 10:02:13 [] ? __down_read+0x12/0xa0
2009/10/21 10:02:13 [] ? __down_read+0x12/0xa0
2009/10/21 09:04:37 [] ? __down_read+0x12/0xa0
2009/10/21 09:04:37 [] ? __down_read+0x12/0xa0
2009-10-21 10:02:14 [<ffffffff804256c8>] ?... (0/10)
2009/10/21 10:02:13 [] ? __down_write_trylock+0x48/0x60
2009/10/21 10:02:13 [] ? __down_write_trylock+0x48/0x60
2009/10/21 10:02:13 [] ? __down_write_trylock+0x48/0x60
2009/10/21 10:02:13 [] ? __down_write_trylock+0x48/0x60
2009/10/21 10:02:13 [] ? __down_write_trylock+0x48/0x60
2009/10/21 09:04:37 [] ? __down_write_trylock+0x48/0x60
2009/10/21 09:04:37 [] ? __down_write_trylock+0x48/0x60
2009/10/21 09:04:37 [] ? __down_write_trylock+0x48/0x60
2009/10/21 09:04:37 [] ? __down_write_trylock+0x48/0x60
2009/10/21 09:04:37 [] ? __down_write_trylock+0x48/0x60
2009-10-21 10:02:14 [<ffffffffa001d663>] ?... (0/4)
2009/10/21 10:02:13 [] ? di_read_unlock+0x73/0x130 [aufs]
2009/10/21 10:02:13 [] ? di_read_unlock+0x73/0x130 [aufs]
2009/10/21 09:04:37 [] ? di_read_unlock+0x73/0x130 [aufs]
2009/10/21 09:04:37 [] ? di_read_unlock+0x73/0x130 [aufs]
2009-10-21 10:02:14 [<ffffffffa001c497>] ?... (0/4)
2009/10/21 10:02:13 [] ? h_d_revalidate+0x4c7/0x6b0 [aufs]
2009/10/21 10:02:13 [] ? h_d_revalidate+0x4c7/0x6b0 [aufs]
2009/10/21 09:04:37 [] ? h_d_revalidate+0x4c7/0x6b0 [aufs]
2009/10/21 09:04:37 [] ? h_d_revalidate+0x4c7/0x6b0 [aufs]
2009-10-21 10:02:14 [<ffffffff80425701>] ?... (0/6)
2009/10/21 10:02:13 [] ? __up_read+0x21/0xb0
2009/10/21 10:02:13 [] ? __up_read+0x21/0xb0
2009/10/21 10:02:13 [] ? __up_read+0x21/0xb0
2009/10/21 09:04:37 [] ? __up_read+0x21/0xb0
2009/10/21 09:04:37 [] ? __up_read+0x21/0xb0
2009/10/21 09:04:37 [] ? __up_read+0x21/0xb0
2009-10-21 10:02:14 [<ffffffffa001ce09>] ?... (0/2)
2009/10/21 10:02:13 [] ? aufs_d_revalidate+0x249/0x4f0 [aufs]
2009/10/21 09:04:37 [] ? aufs_d_revalidate+0x249/0x4f0 [aufs]
2009-10-21 10:02:14 [<ffffffff802d0bef>] ?... (0/2)
2009/10/21 10:02:13 [] ? compat_sys_ioctl+0x31f/0x3f0
2009/10/21 09:04:37 [] ? compat_sys_ioctl+0x31f/0x3f0
2009-10-21 10:02:14 [<ffffffff80284efe>] ?... (0/2)
2009/10/21 10:02:13 [] ? do_munmap+0x26e/0x310
2009/10/21 09:04:37 [] ? do_munmap+0x26e/0x310
2009-10-21 10:02:14 [<ffffffff80228282>] ?... (0/2)
2009/10/21 10:02:13 [] ? ia32_sysret+0x0/0xa
2009/10/21 09:04:37 [] ? ia32_sysret+0x0/0xa
2009-10-21 10:02:14 ... (0/4)
2009/10/21 10:02:13
2009/10/21 10:02:13
2009/10/21 09:04:37
2009/10/21 09:04:37
0:02:14 Code: 48 c7 41 08 00 02 20 00 48 c7 83 a8 10 00 00 00 01 10 00 48 8b bb c8 10 00 00 e8 d4 74 0a e0 48... (0/2)
2009-10-21 10:02:14 RSP... (0/2)
2009-10-21 10:02:13 ------------[ cut... (0/2)
2009-10-21 10:02:13 CPU 2... (0/2)
2009-10-21 10:02:13 RIP: 0010:[<ffffffffa01ead78>] ... (0/2)
2009-10-21 10:02:13 R10: 0000000000000000 R11:... (0/2)
2009-10-21 10:02:13 FS: 0000000000000000(0000)... (2/2)
2009-10-21 10:02:13 CS: 0010 DS: 002b... (2/2)
2009-10-21 10:02:13 DR0: 0000000000000000 DR1:... (2/2)
2009-10-21 10:02:13 DR3: 0000000000000000 DR6:... (2/2)
2009-10-21 10:02:13 Process iscsi-scstd (pid: 12212,... (2/2)
2009-10-21 10:02:13 Call... (2/2)
2009-10-21 10:02:13 [<ffffffffa01eb0d9>] ?... (2/2)
2009-10-21 10:02:13 [<ffffffffa01e97b8>] ?... (1/2)
2009-10-21 10:02:13 [<ffffffff8022e6e5>] ?... (1/2)
2009-10-21 10:02:13 [<ffffffff8022b8d0>] ?... (1/2)
2009-10-21 10:02:13 Pid: 12212, comm: iscsi-scstd Tainted: G ... (0/1)
2009-10-21 10:02:13 RSP: 0000:ffff880248e2b578 ... (1/1)
2009-10-21 10:02:13 RAX: ffff88013cb95088 RBX:... (0/1)
2009-10-21 10:02:13 RDX: ffff88013cb95000 RSI:... (1/1)
2009-10-21 10:02:13 RBP: ffff88033a9dec00 R08:... (1/1)
2009-10-21 10:02:13 R13: 0000000000000003 R14:... (0/1)
2009-10-21 10:02:13 CR2: 00000000f6e96284 CR3:... (1/1)
2009-10-21 10:02:13 Stack: ffff88033a9dec20... (1/1)
2009-10-21 10:02:13 ffff88033a9dec08 ffff88033dbda200... (0/1)
2009-10-21 10:02:13 0000000000000008 ffff88032e5cda80... (1/1)
2009-10-21 10:02:13 [<ffffffffa01eb77d>] ?... (1/1)
2009-10-21 10:02:13 [<ffffffffa01eb877>] ?... (1/1)
2009-10-21 10:02:13 [<ffffffffa01e9889>] ?... (1/1)
2009-10-21 10:02:13 [<ffffffff80297ef8>] ?... (0/1)
2009-10-21 10:02:13 [<ffffffff802961e9>] ?... (1/1)
2009-10-21 10:02:13 [<ffffffff80237390>] ?... (1/1)
2009-10-21 10:02:13 [<ffffffff80237a9b>] ?... (1/1)
2009-10-21 10:02:13 [<ffffffff8069cf27>] ?... (1/1)
2009-10-21 10:02:13 [<ffffffff8020e106>] ?... (0/1)
2009-10-21 10:02:13 [<ffffffffa01ead78>] ?... (0/2)
2009-10-21 10:02:13 [<ffffffffa01d74c4>] ?... (1/1)
2009-10-21 10:02:13 [<ffffffffa01b70e4>] ?... (1/1)
2009-10-21 10:02:13 [<ffffffff8069cab9>] ?... (0/1)
2009-10-21 10:02:13 [<ffffffffa01eb209>] ?... (0/1)