Working draft

N18 name-not-utf8

A name that is not UTF-8.

Reject(draft) A strict parser must reject this archive.

Category
name
Default policy
rejects it
Files
N18-name-not-utf8.tar test archive, 2,048 bytes sha256 3f2c99cc1686f96bc25ef636815c9f06d875310a3251bb96e84f5864ef582f8a
N18-name-not-utf8.jsonexpected result
N18-name-not-utf8.txtannotated blocks
N18-name-not-utf8.pygenerator script

The case

The archive has one file, caf\xe9, whose name is Latin-1.

In POSIX, the header is in ISO/IEC 646, that is ASCII, so the archive does not conform, and the encoding of other characters in names is implementation-defined. This case is ambiguous because the name on disk is either those bytes, or the name with the byte replaced, by U+FFFD or by ?.

Decision point: how a name that is not UTF-8 is decoded.

Seen in: aspell-is 0.51.1-0 and aspell-nb 0.50.1-0.

The archive

Every block of 512 bytes, to scale, with its offset. Hover over a field for its bytes.

N18-name-not-utf8: every block of the archive0x000000header: regular filename [0,100): "caf\xe9" + 96 NULnamemode [100,108): "0000644" + NUL, 644 (octal)uid [108,116): "0000000" + NUL, 0gid [116,124): "0000000" + NUL, 0size [124,136): "00000000010" + NUL, 8mtime [136,148): "00000000000" + NUL, 0chksum [148,156): "010221\0 ", 4241: the unsigned sum of the header, this field as spacestypeflag [156,157): "0", regular filelinkname [157,257): 100 NULlinknamemagic [257,263): "ustar" + NULversion [263,265): "00"uname [265,297): 32 NULunamegname [297,329): 32 NULgnamedevmajor [329,337): "0000000" + NUL, 0devminor [337,345): "0000000" + NUL, 0prefix [345,500): 155 NULprefixpad [500,512): 12 NULname "caf\xe9" + 96 NUL0x000200data8 bytes, then 504 NUL to the end of the block; "Latin-1\n""Latin-1\n"0x0004002 zero blocks: the end-of-archive markerbytes 0 to 512 of 1024: 512 NUL0x0006002 zero blocks, continuedbytes 512 to 1024 of 1024: 512 NUL0x000800end of fileNULprintable ASCIIspacecontrol character, such as a newlinebyte over 0x7fwritten by the case itself
Every block, as text
N18-name-not-utf8: must reject

A name that is not UTF-8.

The archive has one file, `caf\xe9`, whose name is Latin-1.

In POSIX, the header is in ISO/IEC 646, that is ASCII, so the archive does not
conform, and the encoding of other characters in names is implementation-defined.
This case is ambiguous because the name on disk is either those bytes, or the name with the byte
replaced, by U+FFFD or by `?`.

Decision point: how a name that is not UTF-8 is decoded.
Seen in: aspell-is 0.51.1-0 and aspell-nb 0.50.1-0.

0x000000  header: regular file
          name     [  0,100)  "caf\xe9" + 96 NUL
          mode     [100,108)  "0000644" + NUL  644 (octal)
          uid      [108,116)  "0000000" + NUL  0
          gid      [116,124)  "0000000" + NUL  0
          size     [124,136)  "00000000010" + NUL  8
          mtime    [136,148)  "00000000000" + NUL  0
          chksum   [148,156)  "010221\0 "  4241: the unsigned sum of the header, this field as spaces
          typeflag [156,157)  "0"  regular file
          linkname [157,257)  100 NUL
          magic    [257,263)  "ustar" + NUL
          version  [263,265)  "00"
          uname    [265,297)  32 NUL
          gname    [297,329)  32 NUL
          devmajor [329,337)  "0000000" + NUL  0
          devminor [337,345)  "0000000" + NUL  0
          prefix   [345,500)  155 NUL
          pad      [500,512)  12 NUL
0x000200  data
          8 bytes, then 504 NUL to the end of the block
          "Latin-1\n"
0x000400  2 zero blocks: the end-of-archive marker
0x000800  end of file
Every byte, as a hex dump

As xxd writes it: xxd -r makes the archive of it.

00000000: 6361 66e9 0000 0000 0000 0000 0000 0000  caf.............
00000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 3030 3030 3634 3400 3030 3030  ....0000644.0000
00000070: 3030 3000 3030 3030 3030 3000 3030 3030  000.0000000.0000
00000080: 3030 3030 3031 3000 3030 3030 3030 3030  0000010.00000000
00000090: 3030 3000 3031 3032 3231 0020 3000 0000  000.010221. 0...
000000a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000100: 0075 7374 6172 0030 3000 0000 0000 0000  .ustar.00.......
00000110: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000120: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000130: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000140: 0000 0000 0000 0000 0030 3030 3030 3030  .........0000000
00000150: 0030 3030 3030 3030 0000 0000 0000 0000  .0000000........
00000160: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000170: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000180: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000190: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000200: 4c61 7469 6e2d 310a 0000 0000 0000 0000  Latin-1.........
00000210: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000220: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000230: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000240: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000250: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000260: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000270: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000280: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000290: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000300: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000310: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000320: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000330: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000340: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000350: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000360: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000370: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000380: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000390: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000400: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000410: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000420: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000430: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000440: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000450: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000460: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000470: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000480: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000490: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000500: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000510: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000520: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000530: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000540: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000550: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000560: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000570: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000580: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000590: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000600: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000610: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000620: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000630: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000640: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000650: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000660: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000670: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000680: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000690: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000700: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000710: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000720: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000730: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000740: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000750: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000760: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000770: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000780: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000790: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000007a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000007b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000007c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000007d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000007e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000007f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................

What implementations make of it

In the outcome column, ✅ means that the implementation follows the default policy.

implementationversionoutcomewhat it makes of it
GNU tartar (GNU tar) 1.35⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
bsdtarbsdtar 3.8.11 - libarchive 3.8.11⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
BusyBox tarBusyBox v1.38.0, defconfig⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
7-Zip7-Zip (z) 26.05 (x64)⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
Python tarfile, filter=data, as in pipCPython 3.15.2⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
Python tarfile, filter=tarCPython 3.15.2⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
Python tarfile, filter=fully_trustedCPython 3.15.2⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
node-tartar 7.5.22⚠️ acceptstree B: caf� (8 B, ff3f1fa3)
node-tar, stricttar 7.5.22⚠️ acceptstree B: caf� (8 B, ff3f1fa3)
tar-fstar-fs 3.1.3, on tar-stream 3.2.2⚠️ acceptstree B: caf� (8 B, ff3f1fa3)
Java Commons CompressCommons Compress 1.28.0, Expander, on Java 27.0.1.0.2⚠️ acceptstree C: caf? (8 B, ff3f1fa3)
.NET System.Formats.TarSystem.Formats.Tar of .NET 10.0.14, ExtractToDirectory⚠️ acceptstree B: caf� (8 B, ff3f1fa3)
PHP PharData, as in ComposerPharData of PHP 8.5.12, as Composer's TarDownloader calls it⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
Go containerd archive.Applycontainerd v2.4.2, archive.Apply, of go1.27.3⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
Rust tartar 0.4.46, unpack⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
Rust astral-tokio-tarastral-tokio-tar 0.7.1, unpack⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
Rust tar-codec, as in uvtar-codec 0.0.16, with the policies of uv 0.13✅ rejectserror: at byte 0: path is not valid UTF-8
RubyGemsRubyGems 4.0.20 of Ruby 4.0.9, extract_tar_gz⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
R untar, internalutils::untar of R 4.6.2, tar = "internal"⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
Perl Archive::TarArchive::Tar 3.12 of Perl v5.44.0⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
Haskell tartar 0.7.2.0, extract, of GHC 9.14⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
Julia Tar.jlTar 1.10.0 of Julia 1.13.3⚠️ acceptstree A: caf\xe9 (8 B, ff3f1fa3)
live-bootstrap untarmescc-tools-extra a151c245e512, as live-bootstrap pins it✅ rejectserror: Checksum failure The extraction of suite/archives/N18-name-not-utf8.tar produced errors
live-bootstrap untar, --non-strictmescc-tools-extra a151c245e512, as live-bootstrap pins it✅ rejectserror: non-strict mode enabled, preparing for chaos Checksum failure The extraction of suite/archives/N18-name-not-utf8.tar produced errors

Prevalence in release tarballs

The table below shows how many release tarballs have the features of this test case. The default policy rejects all of them.

ecosystemtarballswith itshare
Spack37,5530none
nixpkgs18,99020.01%
Guix10,1680none
PyPI10,0000none
crates.io5,0000none
RubyGems5,0000none
live-bootstrap1710none
total (distinct)82,5862<0.01%

See the list of these 2 tarballs, with the sha256 of each and an example from it.