Bug#784673: cloud-utils: growpart fails with gpt

[Frederic Bonnard]

Package: cloud-utils
Version: 0.27-1
Severity: normal
User: debian-***@lists.debian.org
Usertags: ppc64el

Dear Maintainer,

I did a ppc64el installation with the latest Jessie ; then on a 100G disk, put
the prep partition, the swap and a third partition for / leaving ~90G of free
space after.
Then after booting, I did this :
---
***@vm6:/tmp# growpart -v /dev/sda 3
max_end=0 tot=209712510 pt_end=0 pt_start=0 pt_size=0
NOCHANGE: partition 3 is size 0. it cannot be grown
---

The error seems related to the fact that the output of sfdisk is unusable :
growpart tries to parse : sfdisk -d /dev/sda which gives :
---
***@vm6:/tmp# sfdisk -C 13054 -H 255 -S 63 -uS -d /dev/sda
# partition table of /dev/sda
unit: sectors

/dev/sda1 : start= 1, size=209715199, Id=ee
/dev/sda2 : start= 0, size= 0, Id= 0
/dev/sda3 : start= 0, size= 0, Id= 0
/dev/sda4 : start= 0, size= 0, Id= 0
---

I did some checks with the latest util-linux 2.26.2 and it seems sfdisk has now
better support for gpt :
---
***@vm6:/tmp/util-linux-2.26.2/.libs# LD_LIBRARY_PATH=. ./sfdisk -d /dev/sda
label: gpt
label-id: 98B51944-1718-4DD3-9949-7C82C5BB03E3
device: /dev/sda
unit: sectors
first-lba: 34
last-lba: 209715166

/dev/sda1 : start= 2048, size= 14336, type=9E1A2D38-C612-4316-AA26-8B49521E5A8B, uuid=F667C2D9-0D4D-4FA4-8027-F218887A9062
/dev/sda2 : start= 16384, size= 7813120, type=0657FD6D-A4AB-43C4-84E5-0933C84B4F4F, uuid=9973294F-5AA6-4BF9-950C-5E35F8041AF4
/dev/sda3 : start= 7829504, size= 5859328, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4, uuid=BB1DAB6B-3455-42BC-979C-0EDBE5EF660F
---

Then I adapted growpart (as sfdisk had some options changed) and ran it :
---
***@vm6:/tmp/util-linux-2.26.2/.libs# growpart -v /dev/sda 3
max_end=209712510 tot=209712510 pt_end=13688832 pt_start=7829504 pt_size=5859328
CHANGED: partition=3 start=7829504 old: size=5859328 end=13688832 new: size=201883006,end=209712510

***@vm6:/tmp/util-linux-2.26.2/.libs# LD_LIBRARY_PATH=. ./sfdisk -uS -d /dev/sda
label: gpt
label-id: 98B51944-1718-4DD3-9949-7C82C5BB03E3
device: /dev/sda
unit: sectors
first-lba: 34
last-lba: 209715166

/dev/sda1 : start= 2048, size= 14336, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4, uuid=F667C2D9-0D4D-4FA4-8027-F218887A9062
/dev/sda2 : start= 16384, size= 7813120, type=0657FD6D-A4AB-43C4-84E5-0933C84B4F4F, uuid=9973294F-5AA6-4BF9-950C-5E35F8041AF4
/dev/sda3 : start= 7829504, size= 201883006, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4, uuid=BB1DAB6B-3455-42BC-979C-0EDBE5EF660F
---

I looked for those needed changes and :
- NEW/experimental: 2.26.2-2 : https://tracker.debian.org/pkg/util-linux
- https://code.launchpad.net/~smoser/cloud-utils/growpart-sfdisk-2.26
: "initial start at sfdisk 2.26 support"

Everything seems on its way.
So, that'd be nice, when possible to update util-linux to 2.26.2 and have
cloud-utils with growpart and support for sfdisk 2.26.
Thanks,

Fred


-- System Information:
Debian Release: stretch/sid
APT prefers unstable
APT policy: (500, 'unstable')
Architecture: ppc64el (ppc64le)

Kernel: Linux 3.16.0-4-powerpc64le (SMP w/80 CPU cores)
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)
Shell: /bin/sh linked to /bin/dash
Init: unable to detect

Versions of packages cloud-utils depends on:
ii e2fsprogs 1.42.12-1.1
ii euca2ools 3.1.0-1
ii file 1:5.22+15-2
ii gnupg 1.4.19-2
ii python 2.7.9-1
ii python-paramiko 1.15.1-1
ii python-yaml 3.11-2
ii util-linux 2.25.2-6

Versions of packages cloud-utils recommends:
ii distro-info 0.14
ii python-distro-info 0.14

cloud-utils suggests no packages.

-- no debconf information

[Tiago Ilieve]

Hi Frederic,

I've tried to reproduce this issue using cloud-utils (0.26-2) on Jessie:

***@virtualbox:~# growpart -v /dev/sda 1
geometry is -C 2610 -H 255 -S 63. total size=41929650
max_end=41929650 tot=41929650 pt_end=41943040 pt_start=1 pt_size=41943039
NOCHANGE: partition 1 could only be grown by -13390 [fudge=20480]

And the output was a little bit different, but the result is the same,
as it could not grow the partition.

Then I've tried using cloud-utils (0.27-2) from Stretch on Jessie:

***@virtualbox:~# growpart -v /dev/sda 1
update-partition set to true
FAILED: GPT partition found but no sgdisk

And after installing "gdisk":

***@virtualbox:~# growpart -v /dev/sda 1
update-partition set to true
found GPT partition table (id = ee)
disk=/dev/sda partition=1: original sgdisk info:
Partition GUID code: 0FC63DAF-8483-4772-8E79-3D69D8477DE4 (Linux filesystem)
Partition unique GUID: 4F846BCA-2671-486C-AFD7-6DBA157FDDD1
First sector: 2048 (at 1024.0 KiB)
Last sector: 15624191 (at 7.5 GiB)
Partition size: 15622144 sectors (7.4 GiB)
Attribute flags: 0000000000000000
Partition name: ''
Disk /dev/sda: 41943040 sectors, 20.0 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): D9C74796-7ED9-49AA-84C7-808685BC8ED7
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 41943006
Partitions will be aligned on 2048-sector boundaries
Total free space is 26320829 sectors (12.6 GiB)

Number Start (sector) End (sector) Size Code Name
1 2048 15624191 7.4 GiB 8300
disk=/dev/sda partition=1: pretend sgdisk info
Disk /dev/sda: 41943040 sectors, 20.0 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): D9C74796-7ED9-49AA-84C7-808685BC8ED7
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 41943006
Partitions will be aligned on 2048-sector boundaries
Total free space is 26320829 sectors (12.6 GiB)

Number Start (sector) End (sector) Size Code Name
1 2048 15624191 7.4 GiB 8300
disk=/dev/sda partition=1: pt_start=2048 pt_end=15624191
pt_size=15622143 pt_max=41943006 last=41943006
disk=/dev/sda partition=1: code=0FC63DAF-8483-4772-8E79-3D69D8477DE4
guid=4F846BCA-2671-486C-AFD7-6DBA157FDDD1 name=''
CHANGED: disk=/dev/sda partition=1: start=2048 old:
size=15622143,end=15624191 new: size=41940958,end=41943006

Everything went fine and "resize2fs" is now able to expand the filesystem:

***@virtualbox:~# resize2fs /dev/sda1
resize2fs 1.42.12 (29-Aug-2014)
Filesystem at /dev/sda1 is mounted on /; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 2
The filesystem on /dev/sda1 is now 5242619 (4k) blocks long.

***@virtualbox:~# df -h | grep sda1
/dev/sda1 20G 951M 18G 5% /

Then I've restored a snapshot, upgraded the machine to sid/unstable
and tried again.

Using cloud-utils (0.27-2) from Sid:

***@virtualbox:~# growpart -v /dev/sda 1
update-partition set to true
found MBR partition table (id = 0FC63DAF-8483-4772-8E79-3D69D8477DE4)
total number of sectors of /dev/sda is 41943040
## sfdisk --dump /dev/sda
label: gpt
label-id: D9C74796-7ED9-49AA-84C7-808685BC8ED7
device: /dev/sda
unit: sectors
first-lba: 34
last-lba: 41943006

/dev/sda1 : start= 2048, size= 15622144,
type=0FC63DAF-8483-4772-8E79-3D69D8477DE4,
uuid=4F846BCA-2671-486C-AFD7-6DBA157FDDD1
max_end=41943040 tot=41943040 pt_end=15624192 pt_start=2048 pt_size=15622144
attempt to resize /dev/sda failed. sfdisk output below:
| The last usable GPT sector is 41943006, but 41943039 is requested.
| Failed to add partition: Invalid argument
| Backup files:
| PMBR (offset 0, size 512):
/tmp/growpart.YN5dSL/backup-sda-0x00000000.bak
| GPT Header (offset 512, size 512):
/tmp/growpart.YN5dSL/backup-sda-0x00000200.bak
| GPT Entries (offset 1024, size 16384):
/tmp/growpart.YN5dSL/backup-sda-0x00000400.bak
|
| Disk /dev/sda: 20 GiB, 21474836480 bytes, 41943040 sectors
| Units: sectors of 1 * 512 = 512 bytes
| Sector size (logical/physical): 512 bytes / 512 bytes
| I/O size (minimum/optimal): 512 bytes / 512 bytes
| Disklabel type: gpt
| Disk identifier: D9C74796-7ED9-49AA-84C7-808685BC8ED7
|
| Old situation:
|
| Device Start End Sectors Size Type
| /dev/sda1 2048 15624191 15622144 7.5G Linux filesystem
|
| >>> Script header accepted.
| >>> Script header accepted.
| >>> Script header accepted.
| >>> Script header accepted.
| >>> Script header accepted.
| >>> Script header accepted.
| >>> Created a new GPT disklabel (GUID: D9C74796-7ED9-49AA-84C7-808685BC8ED7).
| Leaving.
|
FAILED: failed to resize
***** WARNING: Resize failed, attempting to revert ******
512+0 records in
512+0 records out
512 bytes copied, 0.00159423 s, 321 kB/s
***** Appears to have gone OK ****

The partition could not be grown and now "resize2fs" can't do anything:

***@virtualbox:~# resize2fs /dev/sda1
resize2fs 1.42.13 (17-May-2015)
The filesystem is already 1952768 (4k) blocks long. Nothing to do!

It has detected the partition table as MBR, although it is GPT, and
failed because it tried to grow it using "sfdisk" instead of "sgdisk".
You can see how the detection is done in the line 574 of
"bin/growpart"[1]. The problem is that the option "--id" from "sfdisk"
is not only deprecated:

***@virtualbox:~# sfdisk --id /dev/sda 1
sfdisk: --id is deprecated in favour of --part-type
0FC63DAF-8483-4772-8E79-3D69D8477DE4

As its behaviour is different from older (< 2.26) versions of
"sfdisk", where it used to return "ee" if a GPT partition was
detected, as you can see in the output from Jessie:

***@virtualbox:~# sfdisk --id /dev/sda 1
ee

Upstream developers are aware of this and had already published a
series of patches[2] to properly support newer (>= 2.26) versions of
"fdisk", but there wasn't a new release of "cloud-utils" since then.
We have a patch[3] applied to our last (0.27-2) package version,
created a week before the one from upstream, but as we can see it
doesn't work for GPT partitions.

Last night I've tried to apply a patch from the revision 267[4] with
no luck (it doesn't apply cleanly). I'm still looking at how this can
be done, if fixing the detection will be enough or if we'll have to
import the entire series (or at least most of it) of upstream patches.
Then I guess it ill be a matter of adding "gdisk" as a package
dependency.

I'll keep you posted.

P.s.: I already knew the Jessie situation since I did a triage for
#784004[5]. The problem with Stretch/Sid is new to me.

Regards,
Tiago.

[1]: https://anonscm.debian.org/cgit/collab-maint/cloud-utils.git/tree/bin/growpart?id=c92ff059a96548c6a752a8f3ef2e0fbe74cb08ec#n574
[2]: http://bazaar.launchpad.net/~cloud-utils-dev/cloud-utils/trunk/changes?filter_file_id=growpart-20110225134600-d84xgz6209r194ob-1
[3]: https://anonscm.debian.org/cgit/collab-maint/cloud-utils.git/tree/debian/patches/also-support-sfdisk-2.26-and-higher.patch?id=8962cfc94acfe9d8a59fbea3d99ed0049152b710
[4]: http://bazaar.launchpad.net/~cloud-utils-dev/cloud-utils/trunk/revision/267
[5]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=784004#20

[Charles Plessy]

Hi Tiago,

this reminded your request for sponsoring the backport of cloud-utils; I just
uploaded it. (Build logs attached for the record).

Cheers,

[Tiago Ilieve]

Hi Charles,

Post by Charles Plessy
Hi Tiago,
this reminded your request for sponsoring the backport of cloud-utils; I just
uploaded it. (Build logs attached for the record).

This is awesome. Thank you very much!

It is on the NEW queue[1] right now. :-)

[1]: https://ftp-master.debian.org/backports-new.html