Merge Multiple Partitions into One Logical Volume on MacOS

This article describes a method to identify bad sectors and isolate them. Then use CoreStorage to merge non-contiguous partitions into one logical volume that appears as a single disk to users and applications in macOS.

The Problem

I have a 1TB hard disk. After a couple of years' use, some bad sectors have developed that render the whole disk unreliable. I know it's generally recommended to replace the disk if used for few years as the situation may deteriorate quickly. That's a sound advice, especially if the disk holds important data. Most of the time, though, bad sectors are tiny compared to today's tera-byte disks. A  disk also has an automatic process in place to relocate and not use bad sectors.

Sometimes the automatic process isn't effective for unknown reasons. For example,  long thrusts of disk thrashing when it happens. I figured a way to systematically isolate all bad sectors and surround them with a margin of safety. Then continue to use the remaining disk for less critical missions as a smaller disk.

Identify Bad Clusters

Bad sectors develop in clusters. When you encounter one bad sector, sectors around it are bad or likely to go bad. So we first identify locations of bad sectors. Mark them and enough space around them as bad regions.

To identify bad sectors, we perform a read (as it's faster) from the beginning of the disk until seeing the first I/O error. We then mark the 5 Gbyte before it and 5 Gbyte after it as one bad region. Five Gbyte is a margin of safety I use and you may choose a different size. Then continue to read from there until seeing the next I/O error. Repeat the process until the end of the disk.

You should always back up important data first before any operations. Then optionally format the disk as a single partition. In my case, it's physical disk, /dev/disk0. I use Unix utility dd to perform disk read. In Terminal, I issue:

$ sudo dd if=/dev/disk0 of=/dev/null bs=400m

I ran into the first I/O error some time later:

dd: /dev/disk0: Input/output error
1264+0 records in
1264+0 records out
530160025600 bytes transferred in 11423.480904 secs (46409674 bytes/sec)

This shows that 1264 full-size records (plus zero partial record) are read from the input device (if=/dev/disk0) and written to the output device (of=/dev/null, which means throwing away). A record size is defined as 400 Mbyte with bs=400m. We resume the task but continue from 5 Gbyte (i.e. 13 records = 5*1024/400) after the bad sector. Hence, I issue:

sudo dd if=/dev/disk0 of=/dev/null bs=400m iseek=1277

We skip the first 1264 + 13 records with iseek=1277 on the input device. Note that I use a record size of 400 Mbyte. I consider the whole record is bad when a single bad sector, 512 bytes, is found within this 400 Mbyte chunk. We want to overestimate bad sectors. Repeat the above process until the end.

For my 1TB hard disk, the above process helped me to identify the following contiguous regions (from the start of the disk):

Region Status
255 GB Good
10 GB Bad
235 GB Good
75 GB Bad
95 GB Good
10 GB Bad
90 GB Good
55 GB Bad
130 GB Good
44 GB Bad

For bad regions, it indicates at least one 512-byte bad sector in the middle of the region. For successive small good regions, I merged them into a larger bad region, for examples the 75 GB, 55 GB and 44 GB. Then I used Disk Utility and partitioned /dev/disk0 into ten partitions of the given sizes in the same order.

Create CoreStorage Volume

CoreStorage was introduced in MacOS 10.7 Lion and is also known as logical volume management. For example, Fusion Drives are one CoreStorage volume spanning two physical disks. Also, FileVault 2.0 is one encrypted CoreStorage volume that houses an unencrypted HFS+ filesystem.

From the previous section, we have created five good partitions out of my 1TB disk. We want to merge them into a single unencrypted CoreStorage volume. We need convert each into a CoreStorage partition and then add all five into one  CoreStorage group. The following command line will do both tasks in one go:

$ sudo diskutil cs create DATACS disk0s2 disk0s4 disk0s6 disk0s8 disk0s10
Started CoreStorage operation
Unmounting disk0s2
Touching partition type on disk0s2
Adding disk0s2 to Logical Volume Group
Unmounting disk0s4
Touching partition type on disk0s4
Adding disk0s4 to Logical Volume Group
Unmounting disk0s6
Touching partition type on disk0s6
Adding disk0s6 to Logical Volume Group
Unmounting disk0s8
Touching partition type on disk0s8
Adding disk0s8 to Logical Volume Group
Unmounting disk0s10
Touching partition type on disk0s10
Adding disk0s10 to Logical Volume Group
Creating Core Storage Logical Volume Group
Switching disk0s2 to Core Storage
Switching disk0s4 to Core Storage
Switching disk0s6 to Core Storage
Switching disk0s8 to Core Storage
Switching disk0s10 to Core Storage
Waiting for Logical Volume Group to appear
Discovered new Logical Volume Group "D28CA321-01DD-464C-BA6C-ED0C0E738395"
Core Storage LVG UUID: D28CA321-01DD-464C-BA6C-ED0C0E738395
Finished CoreStorage operation

DATACS is the name of the new CoreStorage group containing five CoreStorage partitions. Now you can think of this CoreStorage group as a single unit that you can re-partition into one or more volumes that will appear to macOS as one or more disks. For me, I want a single volume of the whole unit. So I issue:

$ sudo diskutil cs createVolume DATACS jhfs+ DATA 100%
The Core Storage Logical Volume Group UUID is D28CA321-01DD-464C-BA6C-ED0C0E738395
Started CoreStorage operation
Waiting for Logical Volume to appear
Formatting file system for Logical Volume
Initialized /dev/rdisk6 as a 803.5 GB case-insensitive HFS Plus volume with a 73728k journal
Mounting disk
Core Storage LV UUID: 6C126F02-38EE-4758-8BBD-C001975CE1E5
Core Storage disk: disk4
Finished CoreStorage operation

This creates a single volume named "DATA" formatted with journaled HFS+ that utilises all space from the "DATACS" CoreStorage group. Note to create two or more volumes, adjust the percentage and issue multiple times. My new disk is now known as disk4 which is considered internal but virtual:

$ diskutil list
/dev/disk4 (internal, virtual):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                  Apple_HFS DATA                   +803.5 GB   disk4
                                 Logical Volume on disk0s2, disk0s5, ...
                                 6C126F02-38EE-4758-8BBD-C001975CE1E5
                                 Unencrypted

One caveat with CoreStorage is that Disk Utility doesn't support most of its operations. You have to operate from command lines like I've shown. Disk Utility does support one operation, re-formatting of CoreStorage volumes.

Final Sanity Check

To ensure that the new disk is free of bad sectors. We repeat the bad sector identification process described above:

$ sudo dd if=/dev/disk4 of=/dev/null bs=400m

This operation may take few hours for a 800 Gbyte disk. It finished without any I/O errors. Initial test looked good but I still didn't feel confident. I fired Disk Utility and performed a re-format with DOE-compliant 3-pass security erasure under "Security Options..". This will erase the whole disk with writing random data to the disk in the first two passes, and writing known data to the disk in the third pass and read back for check. Be warn that it's a very long process. Took two days non-stop for my 800 Gbyte disk. So be patient but definitely worth the check.

Conclusion

The whole process may appear time consuming though most of the time simply waiting. Plan ahead spanning a couple of days and providing sporadic inputs from you. In the end, you get a fresh disk, slightly smaller but healthier. It may well continue as a work horse for a few more years. Let's see and I'll report back with another blog post in due time!

comments powered by Disqus