F58 pax-newline-in-comment-then-sizeA newline in a pax comment, which split on lines leaves a size record before the last one.
Accept(draft) Every implementation must accept this archive, and extract the expected tree below.
F58-pax-newline-in-comment-then-size.tar |
test archive, 3,072 bytes
sha256 f7ae82f4501ccbabb194a46a17f3dd7af1b550e88284dac5680682c746228f63 |
F58-pax-newline-in-comment-then-size.json | expected result |
F58-pax-newline-in-comment-then-size.txt | annotated blocks |
F58-pax-newline-in-comment-then-size.py | generator script |
The archive has a file a. Its pax header has two records: a comment with a newline in it, x\n10 size=0, then size=512. Its header has the size 512 too. The 512 bytes after the header are the header of an empty file smuggled.
pax records are delimited by their length, so the comment is one record. In POSIX, a comment is ignored, and the pax size overrides the size field: a contains the 512 bytes. This case is not ambiguous. A reader that splits records on lines finds a record size=0 before size=512. In POSIX, the last of two records applies, and a reader that takes the first extracts an empty a, then smuggled.
The expected tree:
| path | kind | contents |
|---|---|---|
a | file | 512 bytes, sha256 917dba8e |
Seen in: none of the scanned tarballs.
Every block of 512 bytes, to scale, with its offset. Hover over a field for its bytes.
F58-pax-newline-in-comment-then-size: must accept, and extract its tree
A newline in a pax `comment`, which split on lines leaves a `size` record before the last one.
The archive has a file `a`. Its pax header has two records: a `comment` with a newline in it, `x\n10
size=0`, then `size=512`. Its header has the size 512 too. The 512 bytes after the header are the
header of an empty file `smuggled`.
pax records are delimited by their length, so the comment is one record. In POSIX,
a comment is ignored, and the pax size overrides the size field: `a`
contains the 512 bytes. This case is not ambiguous. A reader that splits records on lines finds a
record `size=0` before `size=512`. In POSIX, the last of two records applies, and a
reader that takes the first extracts an empty `a`, then `smuggled`.
Seen in: none of the scanned tarballs.
See also: F28, a `size` record in a comment on a file without a pax size.
0x000000 header: pax extended header, a comment with a newline, then size=512
name [ 0,100) "PaxHeader" + 91 NUL
mode [100,108) "0000644" + NUL 644 (octal)
uid [108,116) "0000000" + NUL 0
gid [116,124) "0000000" + NUL 0
size [124,136) "00000000043" + NUL 35
mtime [136,148) "00000000000" + NUL 0
chksum [148,156) "011076\0 " 4670: the unsigned sum of the header, this field as spaces
typeflag [156,157) "x" pax extended header
linkname [157,257) 100 NUL
magic [257,263) "ustar" + NUL
version [263,265) "00"
uname [265,297) 32 NUL
gname [297,329) 32 NUL
devmajor [329,337) "0000000" + NUL 0
devminor [337,345) "0000000" + NUL 0
prefix [345,500) 155 NUL
pad [500,512) 12 NUL
0x000200 data: pax records
35 bytes of pax records, then 477 NUL
"23 comment=x\n10 size=0\n"
"12 size=512\n"
0x000400 header: regular file, size 512
name [ 0,100) "a" + 99 NUL
mode [100,108) "0000644" + NUL 644 (octal)
uid [108,116) "0000000" + NUL 0
gid [116,124) "0000000" + NUL 0
size [124,136) "00000001000" + NUL 512
mtime [136,148) "00000000000" + NUL 0
chksum [148,156) "007337\0 " 3807: the unsigned sum of the header, this field as spaces
typeflag [156,157) "0" regular file
linkname [157,257) 100 NUL
magic [257,263) "ustar" + NUL
version [263,265) "00"
uname [265,297) 32 NUL
gname [297,329) 32 NUL
devmajor [329,337) "0000000" + NUL 0
devminor [337,345) "0000000" + NUL 0
prefix [345,500) 155 NUL
pad [500,512) 12 NUL
0x000600 header: regular file, the data of a
name [ 0,100) "smuggled" + 92 NUL
mode [100,108) "0000644" + NUL 644 (octal)
uid [108,116) "0000000" + NUL 0
gid [116,124) "0000000" + NUL 0
size [124,136) "00000000000" + NUL 0
mtime [136,148) "00000000000" + NUL 0
chksum [148,156) "010725\0 " 4565: the unsigned sum of the header, this field as spaces
typeflag [156,157) "0" regular file
linkname [157,257) 100 NUL
magic [257,263) "ustar" + NUL
version [263,265) "00"
uname [265,297) 32 NUL
gname [297,329) 32 NUL
devmajor [329,337) "0000000" + NUL 0
devminor [337,345) "0000000" + NUL 0
prefix [345,500) 155 NUL
pad [500,512) 12 NUL
0x000800 2 zero blocks: the end-of-archive marker
0x000c00 end of file
As xxd writes it: xxd -r makes the archive of it.
00000000: 5061 7848 6561 6465 7200 0000 0000 0000 PaxHeader....... 00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000060: 0000 0000 3030 3030 3634 3400 3030 3030 ....0000644.0000 00000070: 3030 3000 3030 3030 3030 3000 3030 3030 000.0000000.0000 00000080: 3030 3030 3034 3300 3030 3030 3030 3030 0000043.00000000 00000090: 3030 3000 3031 3130 3736 0020 7800 0000 000.011076. x... 000000a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000000b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000000c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000000d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000000e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000000f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000100: 0075 7374 6172 0030 3000 0000 0000 0000 .ustar.00....... 00000110: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000120: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000130: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000140: 0000 0000 0000 0000 0030 3030 3030 3030 .........0000000 00000150: 0030 3030 3030 3030 0000 0000 0000 0000 .0000000........ 00000160: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000170: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000180: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000190: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000001a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000001b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000001c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000001d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000001e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000001f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000200: 3233 2063 6f6d 6d65 6e74 3d78 0a31 3020 23 comment=x.10 00000210: 7369 7a65 3d30 0a31 3220 7369 7a65 3d35 size=0.12 size=5 00000220: 3132 0a00 0000 0000 0000 0000 0000 0000 12.............. 00000230: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000240: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000250: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000260: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000270: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000280: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000290: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000002a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000002b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000002c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000002d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000002e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000002f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000300: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000310: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000320: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000330: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000340: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000350: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000360: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000370: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000380: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000390: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000003a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000003b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000003c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000003d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000003e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000003f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000400: 6100 0000 0000 0000 0000 0000 0000 0000 a............... 00000410: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000420: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000430: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000440: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000450: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000460: 0000 0000 3030 3030 3634 3400 3030 3030 ....0000644.0000 00000470: 3030 3000 3030 3030 3030 3000 3030 3030 000.0000000.0000 00000480: 3030 3031 3030 3000 3030 3030 3030 3030 0001000.00000000 00000490: 3030 3000 3030 3733 3337 0020 3000 0000 000.007337. 0... 000004a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000004b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000004c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000004d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000004e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000004f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000500: 0075 7374 6172 0030 3000 0000 0000 0000 .ustar.00....... 00000510: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000520: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000530: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000540: 0000 0000 0000 0000 0030 3030 3030 3030 .........0000000 00000550: 0030 3030 3030 3030 0000 0000 0000 0000 .0000000........ 00000560: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000570: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000580: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000590: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000005a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000005b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000005c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000005d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000005e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000005f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000600: 736d 7567 676c 6564 0000 0000 0000 0000 smuggled........ 00000610: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000620: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000630: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000640: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000650: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000660: 0000 0000 3030 3030 3634 3400 3030 3030 ....0000644.0000 00000670: 3030 3000 3030 3030 3030 3000 3030 3030 000.0000000.0000 00000680: 3030 3030 3030 3000 3030 3030 3030 3030 0000000.00000000 00000690: 3030 3000 3031 3037 3235 0020 3000 0000 000.010725. 0... 000006a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000006b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000006c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000006d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000006e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000006f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000700: 0075 7374 6172 0030 3000 0000 0000 0000 .ustar.00....... 00000710: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000720: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000730: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000740: 0000 0000 0000 0000 0030 3030 3030 3030 .........0000000 00000750: 0030 3030 3030 3030 0000 0000 0000 0000 .0000000........ 00000760: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000770: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000780: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000790: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000007a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000007b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000007c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000007d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000007e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000007f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000800: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000810: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000820: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000830: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000840: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000850: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000860: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000870: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000880: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000890: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000008a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000008b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000008c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000008d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000008e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000008f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000900: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000910: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000920: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000930: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000940: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000950: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000960: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000970: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000980: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000990: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000009a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000009b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000009c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000009d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000009e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000009f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000a00: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000a10: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000a20: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000a30: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000a40: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000a50: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000a60: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000a70: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000a80: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000a90: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000aa0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000ab0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000ac0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000ad0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000ae0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000af0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000b00: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000b10: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000b20: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000b30: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000b40: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000b50: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000b60: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000b70: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000b80: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000b90: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000ba0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000bb0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000bc0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000bd0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000be0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000bf0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
In the outcome column, ✅ means that the implementation follows the default policy.
| implementation | version | outcome | what it makes of it |
|---|---|---|---|
| GNU tar | tar (GNU tar) 1.35 | ✅ accepts | the expected tree |
| bsdtar | bsdtar 3.8.11 - libarchive 3.8.11 | ✅ accepts | the expected tree |
| BusyBox tar | BusyBox v1.38.0, defconfig | ✅ accepts | the expected tree |
| 7-Zip | 7-Zip (z) 26.05 (x64) | ✅ accepts | the expected tree |
| Python tarfile, filter=data, as in pip | CPython 3.15.2 | ✅ accepts | the expected tree |
| Python tarfile, filter=tar | CPython 3.15.2 | ✅ accepts | the expected tree |
| Python tarfile, filter=fully_trusted | CPython 3.15.2 | ✅ accepts | the expected tree |
| node-tar | tar 7.5.22 | ✅ accepts | the expected tree |
| node-tar, strict | tar 7.5.22 | ✅ accepts | the expected tree |
| tar-fs | tar-fs 3.1.3, on tar-stream 3.2.2 | ✅ accepts | the expected tree |
| Java Commons Compress | Commons Compress 1.28.0, Expander, on Java 27.0.1.0.2 | ✅ accepts | the expected tree |
| .NET System.Formats.Tar | System.Formats.Tar of .NET 10.0.14, ExtractToDirectory | ❌ another tree, or rejects | error: InvalidDataException: The extended header contains invalid records. |
| PHP PharData, as in Composer | PharData of PHP 8.5.12, as Composer's TarDownloader calls it | ✅ accepts | the expected tree |
| Go containerd archive.Apply | containerd v2.4.2, archive.Apply, of go1.27.3 | ✅ accepts | the expected tree |
| Rust tar | tar 0.4.46, unpack | ✅ accepts | the expected tree |
| Rust astral-tokio-tar | astral-tokio-tar 0.7.1, unpack | ✅ accepts | the expected tree |
| Rust tar-codec, as in uv | tar-codec 0.0.16, with the policies of uv 0.13 | ✅ accepts | the expected tree |
| RubyGems | RubyGems 4.0.20 of Ruby 4.0.9, extract_tar_gz | ✅ accepts | the expected tree |
| R untar, internal | utils::untar of R 4.6.2, tar = "internal" | ❌ another tree, or rejects | tree A: a (0 B, e3b0c442), smuggled (0 B, e3b0c442) |
| Perl Archive::Tar | Archive::Tar 3.12 of Perl v5.44.0 | ✅ accepts | the expected tree |
| Haskell tar | tar 0.7.2.0, extract, of GHC 9.14 | ✅ accepts | the expected tree |
| Julia Tar.jl | Tar 1.10.0 of Julia 1.13.3 | ✅ accepts | the expected tree |
| live-bootstrap untar | mescc-tools-extra a151c245e512, as live-bootstrap pins it | ❌ another tree, or rejects | tree B: PaxHeader (35 B, 988f9a93), a (512 B, 917dba8e) |
| live-bootstrap untar, --non-strict | mescc-tools-extra a151c245e512, as live-bootstrap pins it | ❌ another tree, or rejects | tree B: PaxHeader (35 B, 988f9a93), a (512 B, 917dba8e) |
The table below shows how many release tarballs have the features of this test case.
| ecosystem | tarballs | with it | share |
|---|---|---|---|
| Spack | 37,553 | 0 | none |
| nixpkgs | 18,990 | 0 | none |
| Guix | 10,168 | 0 | none |
| PyPI | 10,000 | 0 | none |
| crates.io | 5,000 | 0 | none |
| RubyGems | 5,000 | 0 | none |
| live-bootstrap | 171 | 0 | none |
| total (distinct) | 82,586 | 0 | none |
F28: a size record in a comment on a file without a pax size.