WARNING: at fs/buffer.c:1183 mark_buffer_dirty+0x96/0xa0()
Modules linked in: iscsi_trgt scst_vdisk scst drbd bonding iscsi_tcp libiscsi scsi_transport_iscsi 3w_9xxx e1000 button ftdi_sio usbserial
Pid: 30727, comm: smbd Not tainted 2.6.25.13-oe32-00000-g0c9122d #23
[<c012234e>] warn_on_slowpath+0x3e/0x60
[<c0433517>] dm_unplug_all+0x17/0x30
[<c01190c9>] task_rq_lock+0x29/0x50
[<c011bf77>] try_to_wake_up+0xc7/0xf0
[<c011dca6>] __wake_up_common+0x36/0x60
[<c011dcf7>] __wake_up+0x27/0x40
[<c012303c>] wake_up_klogd+0x2c/0x30
[<c0192277>] __wait_on_buffer+0x17/0x20
[<c0195a74>] sync_dirty_buffer+0x54/0xc0
[<c01932e6>] mark_buffer_dirty+0x96/0xa0
[<c01c2bca>] ext3_commit_super+0x3a/0x50
[<c01c012f>] ext3_handle_error+0x3f/0xa0
[<c01c0293>] __ext3_std_error+0x33/0x60
[<c01c007e>] __ext3_journal_stop+0x2e/0x40
[<c018e6c7>] __mark_inode_dirty+0xc7/0x150
[<c01867ec>] touch_atime+0x8c/0xe0
[<c0153093>] generic_file_mmap+0x43/0x50
[<c01643f0>] mmap_region+0x170/0x420
[<c0153050>] generic_file_mmap+0x0/0x50
[<c0164063>] do_mmap_pgoff+0x1d3/0x370
[<c0106e8c>] sys_mmap2+0x5c/0x80
[<c0103a42>] syscall_call+0x7/0xb
=======================
---[ end trace 0c9a357e48a06970 ]---
sd 1:0:0:0: rejecting I/O to offline device
Buffer I/O error on device dm-10, logical block 0
lost page write due to I/O error on dm-10
ext3_abort called.
EXT3-fs error (device dm-10): ext3_journal_start_sb: Detected aborted journal
Remounting filesystem read-only
EXT3-fs error (device dm-10) in ext3_reserve_inode_write: Journal has aborted
EXT3-fs error (device dm-10) in ext3_reserve_inode_write: Journal has aborted
sd 1:0:0:0: rejecting I/O to offline device
Buffer I/O error on device dm-10, logical block 0
lost page write due to I/O error on dm-10
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
Buffer I/O error on device dm-10, logical block 163845
lost page write due to I/O error on dm-10
sd 1:0:0:0: rejecting I/O to offline device
Buffer I/O error on device dm-10, logical block 163848
lost page write due to I/O error on dm-10
3w-9xxx: scsi1: WARNING: (0x06:0x0037): Character ioctl (0x108) timed out, resetting card.
3w-9xxx: scsi1: AEN: WARNING (0x04:0x0039): Buffer ECC error corrected:address=0x17C820.
3w-9xxx: scsi1: AEN: WARNING (0x04:0x0039): Buffer ECC error corrected:address=0x17C820.
3w-9xxx: scsi1: AEN: WARNING (0x04:0x0039): Buffer ECC error corrected:address=0x17C820.
3w-9xxx: scsi1: AEN: ERROR (0x04:0x005F): Cache synchronization failed; some data lost:unit=0.
3w-9xxx: scsi1: AEN: ERROR (0x04:0x005F): Cache synchronization failed; some data lost:unit=1.
3w-9xxx: scsi1: AEN: INFO (0x04:0x000B): Rebuild started:unit=0.
3w-9xxx: scsi1: AEN: INFO (0x04:0x0005): Rebuild completed:unit=0.
3w-9xxx: scsi1: AEN: WARNING (0x04:0x002F): Verify not started; unit never initialized:Raid5/6 subunit=0.
3w-9xxx: scsi1: AEN: INFO (0x04:0x0029): Verify started:unit=0.
3w-9xxx: scsi1: AEN: WARNING (0x04:0x0023): Sector repair completed ort=3, LBA=0x15E0C8.
3w-9xxx: scsi1: AEN: INFO (0x04:0x0029): Verify started:unit=0.
3w-9xxx: scsi1: AEN: INFO (0x04:0x000B): Rebuild started:unit=0.
st: Version 20080221, fixed bufsize 32768, s/g segs 256
Driver 'st' needs updating - please use bus_type methods
sd 0:0:0:0: Attached scsi generic sg0 type 0
sd 1:0:0:0: Attached scsi generic sg1 type 0
sd 1:0:1:0: Attached scsi generic sg2 type 0
sd 1:0:0:0: rejecting I/O to offline device
xfs_force_shutdown(dm-9,0x1) called from line 420 of file fs/xfs/xfs_rw.c. Return address = 0xc02a2650
Filesystem "dm-9": I/O Error Detected. Shutting down filesystem: dm-9
Please umount the filesystem, and rectify the problem(s)
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
Buffer I/O error on device dm-10, logical block 167938
EXT3-fs error (device dm-10): ext3_readdir: directory #81946 contains a hole at offset 0
sd 1:0:0:0: rejecting I/O to offline device
Buffer I/O error on device dm-10, logical block 167939
EXT3-fs error (device dm-10): ext3_readdir: directory #81947 contains a hole at offset 0
sd 1:0:0:0: rejecting I/O to offline device
EXT3-fs error (device dm-10): ext3_get_inode_loc: unable to read inode block - inode=81984, block=163845
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
Buffer I/O error on device dm-10, logical block 182301
EXT3-fs error (device dm-10): ext3_readdir: directory #82013 contains a hole at offset 0
sd 1:0:0:0: rejecting I/O to offline device
Buffer I/O error on device dm-10, logical block 176154
EXT3-fs error (device dm-10): ext3_readdir: directory #82045 contains a hole at offset 0
sd 1:0:0:0: rejecting I/O to offline device
Buffer I/O error on device dm-10, logical block 176155
EXT3-fs error (device dm-10): ext3_readdir: directory #82046 contains a hole at offset 0
sd 1:0:0:0: rejecting I/O to offline device
EXT3-fs error (device dm-10): ext3_get_inode_loc: unable to read inode block - inode=82064, block=163848
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
3w-9xxx: scsi1: AEN: WARNING (0x04:0x0023): Sector repair completed ort=3, LBA=0x8AD396.
3w-9xxx: scsi1: AEN: INFO (0x04:0x000B): Rebuild started:unit=0.
3w-9xxx: scsi1: AEN: INFO (0x04:0x0055): Battery charging started:.
3w-9xxx: scsi1: AEN: INFO (0x04:0x0056): Battery charging completed:.
3w-9xxx: scsi1: AEN: INFO (0x04:0x0055): Battery charging started:.
3w-9xxx: scsi1: AEN: INFO (0x04:0x0056): Battery charging completed:.
3w-9xxx: scsi1: AEN: WARNING (0x04:0x004B): Battery temperature is high:.
3w-9xxx: scsi1: AEN: INFO (0x04:0x0005): Rebuild completed:unit=0.
3dm2[15921] general protection ip:b7fe5bf2 sp:bfeb67e0 error:0 in libc-2.3.2.so[b7f74000+12a000]
xfs_force_shutdown(dm-9,0x1) called from line 420 of file fs/xfs/xfs_rw.c. Return address = 0xc02a2650
3w-9xxx: scsi1: AEN: ERROR (0x04:0x004D): Battery temperature is too high:.
3w-9xxx: scsi1: AEN: WARNING (0x04:0x004B): Battery temperature is high:.
3w-9xxx: scsi1: AEN: INFO (0x04:0x0029): Verify started:unit=0.
3w-9xxx: scsi1: AEN: WARNING (0x04:0x0023): Sector repair completed ort=3, LBA=0x8A90F3.
3w-9xxx: scsi1: AEN: ERROR (0x04:0x004D): Battery temperature is too high:.
3w-9xxx: scsi1: AEN: WARNING (0x04:0x004B): Battery temperature is high:.
3w-9xxx: scsi1: AEN: INFO (0x04:0x002B): Verify completed:unit=0.
xfs_force_shutdown(dm-9,0x1) called from line 420 of file fs/xfs/xfs_rw.c. Return address = 0xc02a2650
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
program scsiinfo is using a deprecated SCSI ioctl, please convert it to SG_IO
program scsiinfo is using a deprecated SCSI ioctl, please convert it to SG_IO
program scsiinfo is using a deprecated SCSI ioctl, please convert it to SG_IO
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
program scsiinfo is using a deprecated SCSI ioctl, please convert it to SG_IO
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
program scsiinfo is using a deprecated SCSI ioctl, please convert it to SG_IO
xfs_force_shutdown(dm-9,0x1) called from line 420 of file fs/xfs/xfs_rw.c. Return address = 0xc02a2650
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
sd 1:0:0:0: rejecting I/O to offline device
program scsiinfo is using a deprecated SCSI ioctl, please convert it to SG_IO
There is for sure something wrong w/ the 3Ware, Buffer IO errors, ECC errors (related to memory).... If you can really say that the RAID is ok then you can try running the Repair Filesystem from the Extended tools section (but you will need Console access). I would really get 3Ware involved to have a look at those outputs.
All the best,
Todd Maxwell
The customer was hardbooting the NAS server which caused those errors. The RAID rebuilt fine, but after the rebuild, NAS-R3 started having the problems described.
Then if all is ok w/ RAID array then I would run the Repair Filesystem from the Extended tools section.
All the best,
Todd Maxwell
Posting Permissions
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
Forum Rules