This article describes a method to identify bad sectors and isolate them. Then use CoreStorage to merge non-contiguous partitions into one logical volume that appears as a single disk to users and applications in macOS.
I have a 1TB hard disk. After a couple of years' use, some bad sectors have developed that render the whole disk unreliable. I know it's generally recommended to replace the disk if used for few years as the situation may deteriorate quickly. That's a sound advice, especially if the disk holds important data. Most of the time, though, bad sectors are tiny compared to today's tera-byte disks. A disk also has an automatic process in place to relocate and not use bad sectors.
Sometimes the automatic process isn't effective for unknown reasons. For example, long thrusts of disk thrashing when it happens. I figured a way to systematically isolate all bad sectors and surround them with a margin of safety. Then continue to use the remaining disk for less critical missions as a smaller disk.
Identify Bad Clusters
Bad sectors develop in clusters. When you encounter one bad sector, sectors around it are bad or likely to go bad. So we first identify locations of bad sectors. Mark them and enough space around them as bad regions.
To identify bad sectors, we perform a read (as it's faster) from the beginning of the disk until seeing the first I/O error. We then mark the 5 Gbyte before it and 5 Gbyte after it as one bad region. Five Gbyte is a margin of safety I use and you may choose a different size. Then continue to read from there until seeing the next I/O error. Repeat the process until the end of the disk.
You should always back up important data first before any operations. Then optionally format the disk as a single partition. In my case, it's physical disk,
/dev/disk0. I use Unix utility
dd to perform disk read. In Terminal, I issue:
$ sudo dd if=/dev/disk0 of=/dev/null bs=400m
I ran into the first I/O error some time later:
dd: /dev/disk0: Input/output error 1264+0 records in 1264+0 records out 530160025600 bytes transferred in 11423.480904 secs (46409674 bytes/sec)
This shows that 1264 full-size records (plus zero partial record) are read from the input device (
if=/dev/disk0) and written to the output device (
of=/dev/null, which means throwing away). A record size is defined as 400 Mbyte with
bs=400m. We resume the task but continue from 5 Gbyte (i.e. 13 records = 5*1024/400) after the bad sector. Hence, I issue:
sudo dd if=/dev/disk0 of=/dev/null bs=400m iseek=1277
We skip the first 1264 + 13 records with
iseek=1277 on the input device. Note that I use a record size of 400 Mbyte. I consider the whole record is bad when a single bad sector, 512 bytes, is found within this 400 Mbyte chunk. We want to overestimate bad sectors. Repeat the above process until the end.
For my 1TB hard disk, the above process helped me to identify the following contiguous regions (from the start of the disk):
For bad regions, it indicates at least one 512-byte bad sector in the middle of the region. For successive small good regions, I merged them into a larger bad region, for examples the 75 GB, 55 GB and 44 GB. Then I used Disk Utility and partitioned
/dev/disk0 into ten partitions of the given sizes in the same order.
Create CoreStorage Volume
CoreStorage was introduced in MacOS 10.7 Lion and is also known as logical volume management. For example, Fusion Drives are one CoreStorage volume spanning two physical disks. Also, FileVault 2.0 is one encrypted CoreStorage volume that houses an unencrypted HFS+ filesystem.
From the previous section, we have created five good partitions out of my 1TB disk. We want to merge them into a single unencrypted CoreStorage volume. We need convert each into a CoreStorage partition and then add all five into one CoreStorage group. The following command line will do both tasks in one go:
$ sudo diskutil cs create DATACS disk0s2 disk0s4 disk0s6 disk0s8 disk0s10 Started CoreStorage operation Unmounting disk0s2 Touching partition type on disk0s2 Adding disk0s2 to Logical Volume Group Unmounting disk0s4 Touching partition type on disk0s4 Adding disk0s4 to Logical Volume Group Unmounting disk0s6 Touching partition type on disk0s6 Adding disk0s6 to Logical Volume Group Unmounting disk0s8 Touching partition type on disk0s8 Adding disk0s8 to Logical Volume Group Unmounting disk0s10 Touching partition type on disk0s10 Adding disk0s10 to Logical Volume Group Creating Core Storage Logical Volume Group Switching disk0s2 to Core Storage Switching disk0s4 to Core Storage Switching disk0s6 to Core Storage Switching disk0s8 to Core Storage Switching disk0s10 to Core Storage Waiting for Logical Volume Group to appear Discovered new Logical Volume Group "D28CA321-01DD-464C-BA6C-ED0C0E738395" Core Storage LVG UUID: D28CA321-01DD-464C-BA6C-ED0C0E738395 Finished CoreStorage operation
DATACS is the name of the new CoreStorage group containing five CoreStorage partitions. Now you can think of this CoreStorage group as a single unit that you can re-partition into one or more volumes that will appear to macOS as one or more disks. For me, I want a single volume of the whole unit. So I issue:
$ sudo diskutil cs createVolume DATACS jhfs+ DATA 100% The Core Storage Logical Volume Group UUID is D28CA321-01DD-464C-BA6C-ED0C0E738395 Started CoreStorage operation Waiting for Logical Volume to appear Formatting file system for Logical Volume Initialized /dev/rdisk6 as a 803.5 GB case-insensitive HFS Plus volume with a 73728k journal Mounting disk Core Storage LV UUID: 6C126F02-38EE-4758-8BBD-C001975CE1E5 Core Storage disk: disk4 Finished CoreStorage operation
This creates a single volume named "DATA" formatted with journaled HFS+ that utilises all space from the "DATACS" CoreStorage group. Note to create two or more volumes, adjust the percentage and issue multiple times. My new disk is now known as
disk4 which is considered internal but virtual:
$ diskutil list /dev/disk4 (internal, virtual): #: TYPE NAME SIZE IDENTIFIER 0: Apple_HFS DATA +803.5 GB disk4 Logical Volume on disk0s2, disk0s5, ... 6C126F02-38EE-4758-8BBD-C001975CE1E5 Unencrypted
One caveat with CoreStorage is that Disk Utility doesn't support most of its operations. You have to operate from command lines like I've shown. Disk Utility does support one operation, re-formatting of CoreStorage volumes.
Final Sanity Check
To ensure that the new disk is free of bad sectors. We repeat the bad sector identification process described above:
$ sudo dd if=/dev/disk4 of=/dev/null bs=400m
This operation may take few hours for a 800 Gbyte disk. It finished without any I/O errors. Initial test looked good but I still didn't feel confident. I fired Disk Utility and performed a re-format with DOE-compliant 3-pass security erasure under "Security Options..". This will erase the whole disk with writing random data to the disk in the first two passes, and writing known data to the disk in the third pass and read back for check. Be warn that it's a very long process. Took two days non-stop for my 800 Gbyte disk. So be patient but definitely worth the check.
The whole process may appear time consuming though most of the time simply waiting. Plan ahead spanning a couple of days and providing sporadic inputs from you. In the end, you get a fresh disk, slightly smaller but healthier. It may well continue as a work horse for a few more years. Let's see and I'll report back with another blog post in due time!