One way to expand the capacity of a zpool is to replace each disk with a larger disk; once the last disk is replaced the pool can be expanded (or will auto-expand, depending on your pool settings). To do this we do the following:
zpool replace [poolname] [old drive] [new drive]
e.g.:
zpool replace kepler ata-WDC_WD15EARX-00PASB0_WD-WCAZAA512624 ata-WDC_WD20EARX-00PASB0_WD-WCAZAA637471
If you then check on the pool’s status via zpool status, we see:
NAME STATE READ WRITE CKSUM
kepler ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
ata-WDC_WD20EARX-00PASB0_WD-WMAZA7352703 ONLINE 0 0 0
replacing-1 ONLINE 0 0 0
ata-WDC_WD15EARX-00PASB0_WD-WCAZAA512624 ONLINE 0 0 0
ata-WDC_WD20EARX-00PASB0_WD-WCAZAA637471 ONLINE 0 0 0 (resilvering)
The pool will resilver and copy the drive contents across. If you have large drives which are reasonably full this resilver process can take quite a few hours. You can do this with multiple drives at once; here’s a zpool
NAME STATE READ WRITE CKSUM
kepler ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
ata-WDC_WD20EARX-00PASB0_WD-WMAZA7352703 ONLINE 0 0 0
replacing-1 ONLINE 0 0 0
ata-WDC_WD15EARX-00PASB0_WD-WCAZAA512624 ONLINE 0 0 0
ata-WDC_WD20EARX-00PASB0_WD-WCAZAA637471 ONLINE 0 0 0 (resilvering)
replacing-2 ONLINE 0 0 0
ata-ST2000DM001-9YN164_W24009TB ONLINE 0 0 0
ata-WDC_WD20EARS-00MVWB0_WD-WCAZAC389099 ONLINE 0 0 0 (resilvering)
ata-WDC_WD20EFRX-68AX9N0_WD-WMC300005367 ONLINE 0 0 0
ata-WDC_WD20EARX-00MMMB0_WD-WCAWZ0842974 ONLINE 0 0 0
ata-WDC_WD20EARX-00PASB0_WD-WMAZA7482198 ONLINE 0 0 0
Please don’t disconnect the old drive before inserting the new one – this can cause issues with some setups where ZFS complains that it cannot find the old drive to replace.
Once your resilver is complete on the final drive you can expand the vdev by running:
zpool online -e [poolname]
or you can turn on automatic expansion with the following settings:
zfs set autoexpand=on [poolname]
If you are using zpool online -e the pool does not have to be offline for it to work. Now sit back and enjoy your increased space!