F42 pax-comment-not-utf8 · Tar file extraction test suite

The archive has one file, a, whose pax header has one record, a comment of the two bytes \xff\xfe, which are not UTF-8.

In POSIX, the value of every specified keyword, comment among them, is in UTF-8, and every comment is ignored. The value can be seen either as bound by the rule for UTF-8, which makes the archive invalid, or as never read, as the comment is ignored. This case is ambiguous because the archive is either invalid or has the file a.

path	kind	contents
`a`	file	2 bytes, sha256 `87428fc5`

The archive¶

Every block of 512 bytes, to scale, with its offset. Hover over a field for its bytes.

What implementations make of it¶

In the outcome column, ✅ means that the implementation follows the default policy.

implementation	version	outcome	what it makes of it
GNU tar	tar (GNU tar) 1.35	➕ accepts	the expected tree
bsdtar	bsdtar 3.8.11 - libarchive 3.8.11	➕ accepts	the expected tree
BusyBox tar	BusyBox v1.38.0, defconfig	➕ accepts	the expected tree
7-Zip	7-Zip (z) 26.05 (x64)	➕ accepts	the expected tree
Python tarfile, filter=data, as in pip	CPython 3.15.2	➕ accepts	the expected tree
Python tarfile, filter=tar	CPython 3.15.2	➕ accepts	the expected tree
Python tarfile, filter=fully_trusted	CPython 3.15.2	➕ accepts	the expected tree
node-tar	tar 7.5.22	➕ accepts	the expected tree
node-tar, strict	tar 7.5.22	➕ accepts	the expected tree
tar-fs	tar-fs 3.1.3, on tar-stream 3.2.2	➕ accepts	the expected tree
Java Commons Compress	Commons Compress 1.28.0, Expander, on Java 27.0.1.0.2	➕ accepts	the expected tree
.NET System.Formats.Tar	System.Formats.Tar of .NET 10.0.14, ExtractToDirectory	➕ accepts	the expected tree
PHP PharData, as in Composer	PharData of PHP 8.5.12, as Composer's TarDownloader calls it	➕ accepts	the expected tree
Go containerd archive.Apply	containerd v2.4.2, archive.Apply, of go1.27.3	➕ accepts	the expected tree
Rust tar	tar 0.4.46, unpack	➕ accepts	the expected tree
Rust astral-tokio-tar	astral-tokio-tar 0.7.1, unpack	➕ accepts	the expected tree
Rust tar-codec, as in uv	tar-codec 0.0.16, with the policies of uv 0.13	➖ rejects	error: at byte 0: pax records contain invalid UTF-8 text
RubyGems	RubyGems 4.0.20 of Ruby 4.0.9, extract_tar_gz	➕ accepts	the expected tree
R untar, internal	utils::untar of R 4.6.2, tar = "internal"	➕ accepts	the expected tree
Perl Archive::Tar	Archive::Tar 3.12 of Perl v5.44.0	➕ accepts	the expected tree
Haskell tar	tar 0.7.2.0, extract, of GHC 9.14	➕ accepts	the expected tree
Julia Tar.jl	Tar 1.10.0 of Julia 1.13.3	➕ accepts	the expected tree
live-bootstrap untar	mescc-tools-extra a151c245e512, as live-bootstrap pins it	❌ another tree, or rejects	tree A: `PaxHeader` (14 B, eeb3db03), `a` (2 B, 87428fc5)
live-bootstrap untar, --non-strict	mescc-tools-extra a151c245e512, as live-bootstrap pins it	❌ another tree, or rejects	tree A: `PaxHeader` (14 B, eeb3db03), `a` (2 B, 87428fc5)

Prevalence in release tarballs¶

The table below shows how many release tarballs have the features of this test case.

`F42-pax-comment-not-utf8.tar`	test archive, 3,072 bytes sha256 `173e445d1f3612c27a46c6427c9cbc0ea7e12c202754dbf4c159c4956e88a2f9`
`F42-pax-comment-not-utf8.json`	expected result
`F42-pax-comment-not-utf8.txt`	annotated blocks
`F42-pax-comment-not-utf8.py`	generator script

ecosystem	tarballs	share
Spack	37,553	none
nixpkgs	18,990	none
Guix	10,168	none
PyPI	10,000	none
crates.io	5,000	none
RubyGems	5,000	none
live-bootstrap	171	none
total (distinct)	82,586	none

`F42` pax-comment-not-utf8

The case¶

The archive¶

What implementations make of it¶

Prevalence in release tarballs¶

See also¶