Working draft

F16 type-unknown

An unknown typeflag with a size: data, or no data?

Reject(draft) A strict parser must reject this archive.

Category
framing
Default policy
rejects it
Files
F16-type-unknown.tar test archive, 2,048 bytes sha256 55102ceba42d5005546725dd1e11f86922928f14479491ae55fed777c2dd342d
F16-type-unknown.jsonexpected result
F16-type-unknown.txtannotated blocks
F16-type-unknown.pygenerator script

The case

The archive has a member unknown of typeflag Z, which is not defined anywhere, with the size 512. The 512 bytes after its header also form a valid header, of an empty file smuggled.

POSIX says an unknown typeflag is read as a regular file, so those 512 bytes are the data of unknown. This case is ambiguous in practice because a reader that skips the member, rather than extract it as a file, has to decide whether it has data. If it does, the 512 bytes are skipped with it; if not, they are the header of the next member, and smuggled is extracted.

Decision point: whether an unknown typeflag has data.

Seen in: none of the scanned tarballs.

The archive

Every block of 512 bytes, to scale, with its offset. Hover over a field for its bytes.

F16-type-unknown: every block of the archive0x000000header: unknown type, typeflag Z, size 512name [0,100): "unknown" + 93 NULnamemode [100,108): "0000644" + NUL, 644 (octal)uid [108,116): "0000000" + NUL, 0gid [116,124): "0000000" + NUL, 0size [124,136): "00000001000" + NUL, 512mtime [136,148): "00000000000" + NUL, 0chksum [148,156): "010670\0 ", 4536: the unsigned sum of the header, this field as spacestypeflag [156,157): "Z", unknown typeflaglinkname [157,257): 100 NULlinknamemagic [257,263): "ustar" + NULversion [263,265): "00"uname [265,297): 32 NULunamegname [297,329): 32 NULgnamedevmajor [329,337): "0000000" + NUL, 0devminor [337,345): "0000000" + NUL, 0prefix [345,500): 155 NULprefixpad [500,512): 12 NULname "unknown" + 93 NUL0x000200header: regular file, the data of the unknown member, read as a header by somename [0,100): "smuggled" + 92 NULnamemode [100,108): "0000644" + NUL, 644 (octal)uid [108,116): "0000000" + NUL, 0gid [116,124): "0000000" + NUL, 0size [124,136): "00000000000" + NUL, 0mtime [136,148): "00000000000" + NUL, 0chksum [148,156): "010725\0 ", 4565: the unsigned sum of the header, this field as spacestypeflag [156,157): "0", regular filelinkname [157,257): 100 NULlinknamemagic [257,263): "ustar" + NULversion [263,265): "00"uname [265,297): 32 NULunamegname [297,329): 32 NULgnamedevmajor [329,337): "0000000" + NUL, 0devminor [337,345): "0000000" + NUL, 0prefix [345,500): 155 NULprefixpad [500,512): 12 NULname "smuggled" + 92 NUL0x0004002 zero blocks: the end-of-archive markerbytes 0 to 512 of 1024: 512 NUL0x0006002 zero blocks, continuedbytes 512 to 1024 of 1024: 512 NUL0x000800end of fileNULprintable ASCIIspacecontrol character, such as a newlinebyte over 0x7fwritten by the case itself
Every block, as text
F16-type-unknown: must reject

An unknown typeflag with a size: data, or no data?

The archive has a member `unknown` of typeflag `Z`, which is not defined anywhere, with the size
512. The 512 bytes after its header also form a valid header, of an empty file `smuggled`.

POSIX says an unknown typeflag is read as a regular file, so those 512 bytes are the
data of `unknown`. This case is ambiguous in practice because a reader that skips the member, rather
than extract it as a file, has to decide whether it has data. If it does, the 512 bytes are skipped
with it; if not, they are the header of the next member, and `smuggled` is extracted.

Decision point: whether an unknown typeflag has data.
Seen in: none of the scanned tarballs.

0x000000  header: unknown type, typeflag Z, size 512
          name     [  0,100)  "unknown" + 93 NUL
          mode     [100,108)  "0000644" + NUL  644 (octal)
          uid      [108,116)  "0000000" + NUL  0
          gid      [116,124)  "0000000" + NUL  0
          size     [124,136)  "00000001000" + NUL  512
          mtime    [136,148)  "00000000000" + NUL  0
          chksum   [148,156)  "010670\0 "  4536: the unsigned sum of the header, this field as spaces
          typeflag [156,157)  "Z"  unknown typeflag
          linkname [157,257)  100 NUL
          magic    [257,263)  "ustar" + NUL
          version  [263,265)  "00"
          uname    [265,297)  32 NUL
          gname    [297,329)  32 NUL
          devmajor [329,337)  "0000000" + NUL  0
          devminor [337,345)  "0000000" + NUL  0
          prefix   [345,500)  155 NUL
          pad      [500,512)  12 NUL
0x000200  header: regular file, the data of the unknown member, read as a header by some
          name     [  0,100)  "smuggled" + 92 NUL
          mode     [100,108)  "0000644" + NUL  644 (octal)
          uid      [108,116)  "0000000" + NUL  0
          gid      [116,124)  "0000000" + NUL  0
          size     [124,136)  "00000000000" + NUL  0
          mtime    [136,148)  "00000000000" + NUL  0
          chksum   [148,156)  "010725\0 "  4565: the unsigned sum of the header, this field as spaces
          typeflag [156,157)  "0"  regular file
          linkname [157,257)  100 NUL
          magic    [257,263)  "ustar" + NUL
          version  [263,265)  "00"
          uname    [265,297)  32 NUL
          gname    [297,329)  32 NUL
          devmajor [329,337)  "0000000" + NUL  0
          devminor [337,345)  "0000000" + NUL  0
          prefix   [345,500)  155 NUL
          pad      [500,512)  12 NUL
0x000400  2 zero blocks: the end-of-archive marker
0x000800  end of file
Every byte, as a hex dump

As xxd writes it: xxd -r makes the archive of it.

00000000: 756e 6b6e 6f77 6e00 0000 0000 0000 0000  unknown.........
00000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 3030 3030 3634 3400 3030 3030  ....0000644.0000
00000070: 3030 3000 3030 3030 3030 3000 3030 3030  000.0000000.0000
00000080: 3030 3031 3030 3000 3030 3030 3030 3030  0001000.00000000
00000090: 3030 3000 3031 3036 3730 0020 5a00 0000  000.010670. Z...
000000a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000100: 0075 7374 6172 0030 3000 0000 0000 0000  .ustar.00.......
00000110: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000120: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000130: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000140: 0000 0000 0000 0000 0030 3030 3030 3030  .........0000000
00000150: 0030 3030 3030 3030 0000 0000 0000 0000  .0000000........
00000160: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000170: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000180: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000190: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000200: 736d 7567 676c 6564 0000 0000 0000 0000  smuggled........
00000210: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000220: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000230: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000240: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000250: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000260: 0000 0000 3030 3030 3634 3400 3030 3030  ....0000644.0000
00000270: 3030 3000 3030 3030 3030 3000 3030 3030  000.0000000.0000
00000280: 3030 3030 3030 3000 3030 3030 3030 3030  0000000.00000000
00000290: 3030 3000 3031 3037 3235 0020 3000 0000  000.010725. 0...
000002a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000300: 0075 7374 6172 0030 3000 0000 0000 0000  .ustar.00.......
00000310: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000320: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000330: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000340: 0000 0000 0000 0000 0030 3030 3030 3030  .........0000000
00000350: 0030 3030 3030 3030 0000 0000 0000 0000  .0000000........
00000360: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000370: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000380: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000390: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000400: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000410: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000420: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000430: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000440: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000450: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000460: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000470: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000480: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000490: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000500: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000510: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000520: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000530: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000540: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000550: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000560: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000570: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000580: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000590: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000600: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000610: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000620: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000630: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000640: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000650: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000660: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000670: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000680: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000690: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000700: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000710: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000720: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000730: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000740: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000750: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000760: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000770: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000780: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000790: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000007a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000007b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000007c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000007d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000007e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000007f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................

What implementations make of it

In the outcome column, ✅ means that the implementation follows the default policy.

implementationversionoutcomewhat it makes of it
GNU tartar (GNU tar) 1.35⚠️ acceptstree A: unknown (512 B, 917dba8e)
bsdtarbsdtar 3.8.11 - libarchive 3.8.11⚠️ acceptstree A: unknown (512 B, 917dba8e)
BusyBox tarBusyBox v1.38.0, defconfig✅ rejectserror: tar: unknown typeflag: 0x5a
7-Zip7-Zip (z) 26.05 (x64)⚠️ acceptstree A: unknown (512 B, 917dba8e)
Python tarfile, filter=data, as in pipCPython 3.15.2✅ rejectserror: SpecialFileError: 'unknown' is a special file
Python tarfile, filter=tarCPython 3.15.2⚠️ acceptstree A: unknown (512 B, 917dba8e)
Python tarfile, filter=fully_trustedCPython 3.15.2⚠️ acceptstree A: unknown (512 B, 917dba8e)
node-tartar 7.5.22⚠️ acceptstree B: nothing
node-tar, stricttar 7.5.22⚠️ acceptstree B: nothing
tar-fstar-fs 3.1.3, on tar-stream 3.2.2✅ rejectserror: Error: unsupported type for DEST/unknown (null)
Java Commons CompressCommons Compress 1.28.0, Expander, on Java 27.0.1.0.2⚠️ acceptstree A: unknown (512 B, 917dba8e)
.NET System.Formats.TarSystem.Formats.Tar of .NET 10.0.14, ExtractToDirectory✅ rejectserror: InvalidOperationException: Entry type '90' not supported for extraction.
PHP PharData, as in ComposerPharData of PHP 8.5.12, as Composer's TarDownloader calls it⚠️ acceptstree A: unknown (512 B, 917dba8e)
Go containerd archive.Applycontainerd v2.4.2, archive.Apply, of go1.27.3✅ rejectserror: unhandled tar header type 90
Rust tartar 0.4.46, unpack⚠️ acceptstree A: unknown (512 B, 917dba8e)
Rust astral-tokio-tarastral-tokio-tar 0.7.1, unpack⚠️ acceptstree A: unknown (512 B, 917dba8e)
Rust tar-codec, as in uvtar-codec 0.0.16, with the policies of uv 0.13✅ rejectserror: at byte 0: unsupported tar typeflag 90
RubyGemsRubyGems 4.0.20 of Ruby 4.0.9, extract_tar_gz⚠️ acceptstree B: nothing
R untar, internalutils::untar of R 4.6.2, tar = "internal"✅ rejectserror: unsupported entry type ‘Z’
Perl Archive::TarArchive::Tar 3.12 of Perl v5.44.0⚠️ acceptstree A: unknown (512 B, 917dba8e)
Haskell tartar 0.7.2.0, extract, of GHC 9.14⚠️ acceptstree B: nothing
Julia Tar.jlTar 1.10.0 of Julia 1.13.3✅ rejectserror: unsupported entry type Tar.Header("unknown", :Z, 0o644, 512, "")
live-bootstrap untarmescc-tools-extra a151c245e512, as live-bootstrap pins it⚠️ acceptstree A: unknown (512 B, 917dba8e)
live-bootstrap untar, --non-strictmescc-tools-extra a151c245e512, as live-bootstrap pins it⚠️ acceptstree A: unknown (512 B, 917dba8e)

Prevalence in release tarballs

The table below shows how many release tarballs have the features of this test case.

ecosystemtarballswith itshare
Spack37,5530none
nixpkgs18,9900none
Guix10,1680none
PyPI10,0000none
crates.io5,0000none
RubyGems5,0000none
live-bootstrap1710none
total (distinct)82,5860none