dm-dust ======= This target emulates the behavior of bad sectors at arbitrary locations, and the ability to enable the emulation of the failures at an arbitrary time. This target behaves similarly to a linear target. At a given time, the user can send a message to the target to start failing read requests on specific blocks (to emulate the behavior of a hard disk drive with bad sectors). When the failure behavior is enabled (i.e.: when the output of "dmsetup status" displays "fail_read_on_bad_block"), reads of blocks in the "bad block list" will fail with EIO ("Input/output error"). Writes of blocks in the "bad block list will result in the following: 1. Remove the block from the "bad block list". 2. Successfully complete the write. This emulates the "remapped sector" behavior of a drive with bad sectors. Normally, a drive that is encountering bad sectors will most likely encounter more bad sectors, at an unknown time or location. With dm-dust, the user can use the "addbadblock" and "removebadblock" messages to add arbitrary bad blocks at new locations, and the "enable" and "disable" messages to modulate the state of whether the configured "bad blocks" will be treated as bad, or bypassed. This allows the pre-writing of test data and metadata prior to simulating a "failure" event where bad sectors start to appear. Table parameters ---------------- <device_path> <offset> <blksz> Mandatory parameters: <device_path>: Path to the block device. <offset>: Offset to data area from start of device_path <blksz>: Block size in bytes (minimum 512, maximum 1073741824, must be a power of 2) Usage instructions ------------------ First, find the size (in 512-byte sectors) of the device to be used:: $ sudo blockdev --getsz /dev/vdb1 33552384 Create the dm-dust device: (For a device with a block size of 512 bytes) :: $ sudo dmsetup create dust1 --table '0 33552384 dust /dev/vdb1 0 512' (For a device with a block size of 4096 bytes) :: $ sudo dmsetup create dust1 --table '0 33552384 dust /dev/vdb1 0 4096' Check the status of the read behavior ("bypass" indicates that all I/O will be passed through to the underlying device; "verbose" indicates that bad block additions, removals, and remaps will be verbosely logged):: $ sudo dmsetup status dust1 0 33552384 dust 252:17 bypass verbose $ sudo dd if=/dev/mapper/dust1 of=/dev/null bs=512 count=128 iflag=direct 128+0 records in 128+0 records out $ sudo dd if=/dev/zero of=/dev/mapper/dust1 bs=512 count=128 oflag=direct 128+0 records in 128+0 records out Adding and removing bad blocks ------------------------------ At any time (i.e.: whether the device has the "bad block" emulation enabled or disabled), bad blocks may be added or removed from the device via the "addbadblock" and "removebadblock" messages:: $ sudo dmsetup message dust1 0 addbadblock 60 kernel: device-mapper: dust: badblock added at block 60 $ sudo dmsetup message dust1 0 addbadblock 67 kernel: device-mapper: dust: badblock added at block 67 $ sudo dmsetup message dust1 0 addbadblock 72 kernel: device-mapper: dust: badblock added at block 72 These bad blocks will be stored in the "bad block list". While the device is in "bypass" mode, reads and writes will succeed:: $ sudo dmsetup status dust1 0 33552384 dust 252:17 bypass Enabling block read failures ---------------------------- To enable the "fail read on bad block" behavior, send the "enable" message:: $ sudo dmsetup message dust1 0 enable kernel: device-mapper: dust: enabling read failures on bad sectors $ sudo dmsetup status dust1 0 33552384 dust 252:17 fail_read_on_bad_block With the device in "fail read on bad block" mode, attempting to read a block will encounter an "Input/output error":: $ sudo dd if=/dev/mapper/dust1 of=/dev/null bs=512 count=1 skip=67 iflag=direct dd: error reading '/dev/mapper/dust1': Input/output error 0+0 records in 0+0 records out 0 bytes copied, 0.00040651 s, 0.0 kB/s ...and writing to the bad blocks will remove the blocks from the list, therefore emulating the "remap" behavior of hard disk drives:: $ sudo dd if=/dev/zero of=/dev/mapper/dust1 bs=512 count=128 oflag=direct 128+0 records in 128+0 records out kernel: device-mapper: dust: block 60 removed from badblocklist by write kernel: device-mapper: dust: block 67 removed from badblocklist by write kernel: device-mapper: dust: block 72 removed from badblocklist by write kernel: device-mapper: dust: block 87 removed from badblocklist by write Bad block add/remove error handling ----------------------------------- Attempting to add a bad block that already exists in the list will result in an "Invalid argument" error, as well as a helpful message:: $ sudo dmsetup message dust1 0 addbadblock 88 device-mapper: message ioctl on dust1 failed: Invalid argument kernel: device-mapper: dust: block 88 already in badblocklist Attempting to remove a bad block that doesn't exist in the list will result in an "Invalid argument" error, as well as a helpful message:: $ sudo dmsetup message dust1 0 removebadblock 87 device-mapper: message ioctl on dust1 failed: Invalid argument kernel: device-mapper: dust: block 87 not found in badblocklist Counting the number of bad blocks in the bad block list ------------------------------------------------------- To count the number of bad blocks configured in the device, run the following message command:: $ sudo dmsetup message dust1 0 countbadblocks A message will print with the number of bad blocks currently configured on the device:: countbadblocks: 895 badblock(s) found Querying for specific bad blocks -------------------------------- To find out if a specific block is in the bad block list, run the following message command:: $ sudo dmsetup message dust1 0 queryblock 72 The following message will print if the block is in the list:: dust_query_block: block 72 found in badblocklist The following message will print if the block is not in the list:: dust_query_block: block 72 not found in badblocklist The "queryblock" message command will work in both the "enabled" and "disabled" modes, allowing the verification of whether a block will be treated as "bad" without having to issue I/O to the device, or having to "enable" the bad block emulation. Clearing the bad block list --------------------------- To clear the bad block list (without needing to individually run a "removebadblock" message command for every block), run the following message command:: $ sudo dmsetup message dust1 0 clearbadblocks After clearing the bad block list, the following message will appear:: dust_clear_badblocks: badblocks cleared If there were no bad blocks to clear, the following message will appear:: dust_clear_badblocks: no badblocks found Listing the bad block list -------------------------- To list all bad blocks in the bad block list (using an example device with blocks 1 and 2 in the bad block list), run the following message command:: $ sudo dmsetup message dust1 0 listbadblocks 1 2 If there are no bad blocks in the bad block list, the command will execute with no output:: $ sudo dmsetup message dust1 0 listbadblocks Message commands list --------------------- Below is a list of the messages that can be sent to a dust device: Operations on blocks (requires a <blknum> argument):: addbadblock <blknum> queryblock <blknum> removebadblock <blknum> ...where <blknum> is a block number within range of the device (corresponding to the block size of the device.) Single argument message commands:: countbadblocks clearbadblocks listbadblocks disable enable quiet Device removal -------------- When finished, remove the device via the "dmsetup remove" command:: $ sudo dmsetup remove dust1 Quiet mode ---------- On test runs with many bad blocks, it may be desirable to avoid excessive logging (from bad blocks added, removed, or "remapped"). This can be done by enabling "quiet mode" via the following message:: $ sudo dmsetup message dust1 0 quiet This will suppress log messages from add / remove / removed by write operations. Log messages from "countbadblocks" or "queryblock" message commands will still print in quiet mode. The status of quiet mode can be seen by running "dmsetup status":: $ sudo dmsetup status dust1 0 33552384 dust 252:17 fail_read_on_bad_block quiet To disable quiet mode, send the "quiet" message again:: $ sudo dmsetup message dust1 0 quiet $ sudo dmsetup status dust1 0 33552384 dust 252:17 fail_read_on_bad_block verbose (The presence of "verbose" indicates normal logging.) "Why not...?" ------------- scsi_debug has a "medium error" mode that can fail reads on one specified sector (sector 0x1234, hardcoded in the source code), but it uses RAM for the persistent storage, which drastically decreases the potential device size. dm-flakey fails all I/O from all block locations at a specified time frequency, and not a given point in time. When a bad sector occurs on a hard disk drive, reads to that sector are failed by the device, usually resulting in an error code of EIO ("I/O error") or ENODATA ("No data available"). However, a write to the sector may succeed, and result in the sector becoming readable after the device controller no longer experiences errors reading the sector (or after a reallocation of the sector). However, there may be bad sectors that occur on the device in the future, in a different, unpredictable location. This target seeks to provide a device that can exhibit the behavior of a bad sector at a known sector location, at a known time, based on a large storage device (at least tens of gigabytes, not occupying system memory).