N32 linkname-not-utf8A symlink whose target in the header is not UTF-8.
Reject(draft) A strict parser must reject this archive.
N32-linkname-not-utf8.tar |
test archive, 1,536 bytes
sha256 6a52259a774d44d053561baacba2d7926255ab9696e7d37749102b12f63ba6a2 |
N32-linkname-not-utf8.json | expected result |
N32-linkname-not-utf8.txt | annotated blocks |
N32-linkname-not-utf8.py | generator script |
The archive has a symlink link, whose header has the target caf\xe9, café in Latin-1, and which does not have a pax header.
The fields of a ustar header are bytes, and in POSIX the characters of a header are those of ISO/IEC 646, of which \xe9 is not one, so the archive does not conform, and the encoding of other characters in names is implementation-defined. A reader decodes the target as UTF-8, as another encoding, or not at all. This case is ambiguous because the target is either the bytes as they are, caf\xe9, or a decoding of them, such as café or caf followed by U+FFFD.
Decision point: how a link target that is not UTF-8 is decoded.
Seen in: none of the scanned tarballs.
Every block of 512 bytes, to scale, with its offset. Hover over a field for its bytes.
N32-linkname-not-utf8: must reject
A symlink whose target in the header is not UTF-8.
The archive has a symlink `link`, whose header has the target `caf\xe9`, `café` in Latin-1, and
which does not have a pax header.
The fields of a ustar header are bytes, and in POSIX the characters of a header are
those of ISO/IEC 646, of which `\xe9` is not one, so the archive does not conform, and the encoding
of other characters in names is implementation-defined. A reader decodes the target
as UTF-8, as another encoding, or not at all. This case is ambiguous because the target is either
the bytes as they are, `caf\xe9`, or a decoding of them, such as `café` or `caf` followed by U+FFFD.
Decision point: how a link target that is not UTF-8 is decoded.
Seen in: none of the scanned tarballs.
See also: N31, the same target in a pax `linkpath`, and N18, a name that is not UTF-8.
0x000000 header: symlink, a target that is not UTF-8
name [ 0,100) "link" + 96 NUL
mode [100,108) "0000644" + NUL 644 (octal)
uid [108,116) "0000000" + NUL 0
gid [116,124) "0000000" + NUL 0
size [124,136) "00000000000" + NUL 0
mtime [136,148) "00000000000" + NUL 0
chksum [148,156) "011100\0 " 4672: the unsigned sum of the header, this field as spaces
typeflag [156,157) "2" symlink
linkname [157,257) "caf\xe9" + 96 NUL
magic [257,263) "ustar" + NUL
version [263,265) "00"
uname [265,297) 32 NUL
gname [297,329) 32 NUL
devmajor [329,337) "0000000" + NUL 0
devminor [337,345) "0000000" + NUL 0
prefix [345,500) 155 NUL
pad [500,512) 12 NUL
0x000200 2 zero blocks: the end-of-archive marker
0x000600 end of file
As xxd writes it: xxd -r makes the archive of it.
00000000: 6c69 6e6b 0000 0000 0000 0000 0000 0000 link............ 00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000060: 0000 0000 3030 3030 3634 3400 3030 3030 ....0000644.0000 00000070: 3030 3000 3030 3030 3030 3000 3030 3030 000.0000000.0000 00000080: 3030 3030 3030 3000 3030 3030 3030 3030 0000000.00000000 00000090: 3030 3000 3031 3131 3030 0020 3263 6166 000.011100. 2caf 000000a0: e900 0000 0000 0000 0000 0000 0000 0000 ................ 000000b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000000c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000000d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000000e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000000f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000100: 0075 7374 6172 0030 3000 0000 0000 0000 .ustar.00....... 00000110: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000120: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000130: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000140: 0000 0000 0000 0000 0030 3030 3030 3030 .........0000000 00000150: 0030 3030 3030 3030 0000 0000 0000 0000 .0000000........ 00000160: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000170: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000180: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000190: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000001a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000001b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000001c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000001d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000001e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000001f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000200: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000210: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000220: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000230: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000240: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000250: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000260: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000270: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000280: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000290: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000002a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000002b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000002c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000002d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000002e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000002f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000300: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000310: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000320: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000330: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000340: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000350: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000360: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000370: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000380: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000390: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000003a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000003b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000003c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000003d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000003e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000003f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000400: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000410: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000420: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000430: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000440: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000450: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000460: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000470: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000480: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000490: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000004a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000004b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000004c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000004d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000004e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000004f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000500: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000510: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000520: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000530: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000540: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000550: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000560: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000570: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000580: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000590: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000005a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000005b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000005c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000005d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000005e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000005f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
In the outcome column, ✅ means that the implementation follows the default policy.
| implementation | version | outcome | what it makes of it |
|---|---|---|---|
| GNU tar | tar (GNU tar) 1.35 | ⚠️ accepts | tree A: link → caf\xe9 |
| bsdtar | bsdtar 3.8.11 - libarchive 3.8.11 | ⚠️ accepts | tree A: link → caf\xe9 |
| BusyBox tar | BusyBox v1.38.0, defconfig | ⚠️ accepts | tree A: link → caf\xe9 |
| 7-Zip | 7-Zip (z) 26.05 (x64) | ⚠️ accepts | tree A: link → caf\xe9 |
| Python tarfile, filter=data, as in pip | CPython 3.15.2 | ⚠️ accepts | tree A: link → caf\xe9 |
| Python tarfile, filter=tar | CPython 3.15.2 | ⚠️ accepts | tree A: link → caf\xe9 |
| Python tarfile, filter=fully_trusted | CPython 3.15.2 | ⚠️ accepts | tree A: link → caf\xe9 |
| node-tar | tar 7.5.22 | ⚠️ accepts | tree B: link → caf� |
| node-tar, strict | tar 7.5.22 | ⚠️ accepts | tree B: link → caf� |
| tar-fs | tar-fs 3.1.3, on tar-stream 3.2.2 | ⚠️ accepts | tree B: link → caf� |
| Java Commons Compress | Commons Compress 1.28.0, Expander, on Java 27.0.1.0.2 | ⚠️ accepts | tree C: link (0 B, e3b0c442) |
| .NET System.Formats.Tar | System.Formats.Tar of .NET 10.0.14, ExtractToDirectory | ⚠️ accepts | tree B: link → caf� |
| PHP PharData, as in Composer | PharData of PHP 8.5.12, as Composer's TarDownloader calls it | ⚠️ accepts | tree C: link (0 B, e3b0c442) |
| Go containerd archive.Apply | containerd v2.4.2, archive.Apply, of go1.27.3 | ⚠️ accepts | tree A: link → caf\xe9 |
| Rust tar | tar 0.4.46, unpack | ⚠️ accepts | tree A: link → caf\xe9 |
| Rust astral-tokio-tar | astral-tokio-tar 0.7.1, unpack | ⚠️ accepts | tree A: link → caf\xe9 |
| Rust tar-codec, as in uv | tar-codec 0.0.16, with the policies of uv 0.13 | ✅ rejects | error: at byte 0: linkpath is not valid UTF-8 |
| RubyGems | RubyGems 4.0.20 of Ruby 4.0.9, extract_tar_gz | ⚠️ accepts | tree D: nothing |
| R untar, internal | utils::untar of R 4.6.2, tar = "internal" | ⚠️ accepts | tree A: link → caf\xe9 |
| Perl Archive::Tar | Archive::Tar 3.12 of Perl v5.44.0 | ⚠️ accepts | tree A: link → caf\xe9 |
| Haskell tar | tar 0.7.2.0, extract, of GHC 9.14 | ⚠️ accepts | tree A: link → caf\xe9 |
| Julia Tar.jl | Tar 1.10.0 of Julia 1.13.3 | ⚠️ accepts | tree A: link → caf\xe9 |
| live-bootstrap untar | mescc-tools-extra a151c245e512, as live-bootstrap pins it | ✅ rejects | error: Checksum failure The extraction of suite/archives/N32-linkname-not-utf8.tar produced errors |
| live-bootstrap untar, --non-strict | mescc-tools-extra a151c245e512, as live-bootstrap pins it | ✅ rejects | error: non-strict mode enabled, preparing for chaos Checksum failure The extraction of suite/archives/N32-linkname-not-utf8.tar produced errors |
The table below shows how many release tarballs have the features of this test case.
| ecosystem | tarballs | with it | share |
|---|---|---|---|
| Spack | 37,553 | – | – |
| nixpkgs | 18,990 | – | – |
| Guix | 10,168 | – | – |
| PyPI | 10,000 | – | – |
| crates.io | 5,000 | 0 | none |
| RubyGems | 5,000 | – | – |
| live-bootstrap | 171 | – | – |
| total (distinct) | 82,586 | – | – |
Note: cells with – were not measured. The count awaits a new scan of all tarballs.