Working draft

F04 end-missing

No end-of-archive marker: the archive ends after a complete member.

Tolerate(draft) A strict parser may reject this archive. If it accepts it, it must extract the expected tree below.

Category
framing
Default policy
accepts it: implementations that extract it extract the same tree
Files
F04-end-missing.tar test archive, 1,024 bytes sha256 0201a877ec49d04139d7cb4c21e25c1ab17d01ac3fde8341c1902bcb80e791bc
F04-end-missing.jsonexpected result
F04-end-missing.txtannotated blocks
F04-end-missing.pygenerator script

The case

The archive has a file a, and ends right after its data, without zero blocks.

In POSIX, an archive ends with two zero blocks. Without them, an archive cannot be told apart from one cut short after a member. The default policy tolerates it, as it is rarely a problem in practice: whether an archive is complete is known by other means, such as a published checksum, or the decompression of a compressed archive.

The expected tree:

pathkindcontents
afile2 bytes, sha256 87428fc5

Decision point: whether an archive without an end-of-archive marker is complete.

Seen in: SDL2_image 2.8.8, ruby 4.0.5 and fastcap 2.0-18Sep92.

The archive

Every block of 512 bytes, to scale, with its offset. Hover over a field for its bytes.

F04-end-missing: every block of the archive0x000000header: regular filename [0,100): "a" + 99 NULnamemode [100,108): "0000644" + NUL, 644 (octal)uid [108,116): "0000000" + NUL, 0gid [116,124): "0000000" + NUL, 0size [124,136): "00000000002" + NUL, 2mtime [136,148): "00000000000" + NUL, 0chksum [148,156): "007340\0 ", 3808: the unsigned sum of the header, this field as spacestypeflag [156,157): "0", regular filelinkname [157,257): 100 NULlinknamemagic [257,263): "ustar" + NULversion [263,265): "00"uname [265,297): 32 NULunamegname [297,329): 32 NULgnamedevmajor [329,337): "0000000" + NUL, 0devminor [337,345): "0000000" + NUL, 0prefix [345,500): 155 NULprefixpad [500,512): 12 NULname "a" + 99 NUL0x000200data2 bytes, then 510 NUL to the end of the block; "a\n""a\n"0x000400end of fileNULprintable ASCIIspacecontrol character, such as a newlinebyte over 0x7fwritten by the case itself
Every block, as text
F04-end-missing: tolerated: a strict parser may reject it; if it accepts it, it must extract its tree, and the default policy accepts it

No end-of-archive marker: the archive ends after a complete member.

The archive has a file `a`, and ends right after its data, without zero blocks.

In POSIX, an archive ends with two zero blocks. Without them, an archive cannot be told apart
from one cut short after a member. The default policy tolerates it, as it is rarely a problem in
practice: whether an archive is complete is known by other means, such as a published checksum, or
the decompression of a compressed archive.

Decision point: whether an archive without an end-of-archive marker is complete.
Seen in: SDL2_image 2.8.8, ruby 4.0.5 and fastcap 2.0-18Sep92.
See also: F08, an archive cut short inside the data of a member, which is rejected.

0x000000  header: regular file
          name     [  0,100)  "a" + 99 NUL
          mode     [100,108)  "0000644" + NUL  644 (octal)
          uid      [108,116)  "0000000" + NUL  0
          gid      [116,124)  "0000000" + NUL  0
          size     [124,136)  "00000000002" + NUL  2
          mtime    [136,148)  "00000000000" + NUL  0
          chksum   [148,156)  "007340\0 "  3808: the unsigned sum of the header, this field as spaces
          typeflag [156,157)  "0"  regular file
          linkname [157,257)  100 NUL
          magic    [257,263)  "ustar" + NUL
          version  [263,265)  "00"
          uname    [265,297)  32 NUL
          gname    [297,329)  32 NUL
          devmajor [329,337)  "0000000" + NUL  0
          devminor [337,345)  "0000000" + NUL  0
          prefix   [345,500)  155 NUL
          pad      [500,512)  12 NUL
0x000200  data
          2 bytes, then 510 NUL to the end of the block
          "a\n"
0x000400  end of file
Every byte, as a hex dump

As xxd writes it: xxd -r makes the archive of it.

00000000: 6100 0000 0000 0000 0000 0000 0000 0000  a...............
00000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 3030 3030 3634 3400 3030 3030  ....0000644.0000
00000070: 3030 3000 3030 3030 3030 3000 3030 3030  000.0000000.0000
00000080: 3030 3030 3030 3200 3030 3030 3030 3030  0000002.00000000
00000090: 3030 3000 3030 3733 3430 0020 3000 0000  000.007340. 0...
000000a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000100: 0075 7374 6172 0030 3000 0000 0000 0000  .ustar.00.......
00000110: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000120: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000130: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000140: 0000 0000 0000 0000 0030 3030 3030 3030  .........0000000
00000150: 0030 3030 3030 3030 0000 0000 0000 0000  .0000000........
00000160: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000170: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000180: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000190: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000200: 610a 0000 0000 0000 0000 0000 0000 0000  a...............
00000210: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000220: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000230: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000240: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000250: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000260: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000270: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000280: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000290: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000300: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000310: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000320: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000330: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000340: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000350: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000360: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000370: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000380: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000390: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................

What implementations make of it

In the outcome column, ✅ means that the implementation follows the default policy.

implementationversionoutcomewhat it makes of it
GNU tartar (GNU tar) 1.35➕ acceptsthe expected tree
bsdtarbsdtar 3.8.11 - libarchive 3.8.11➕ acceptsthe expected tree
BusyBox tarBusyBox v1.38.0, defconfig➕ acceptsthe expected tree
7-Zip7-Zip (z) 26.05 (x64)➖ rejectserror: ERRORS: Unexpected end of archive
Python tarfile, filter=data, as in pipCPython 3.15.2➕ acceptsthe expected tree
Python tarfile, filter=tarCPython 3.15.2➕ acceptsthe expected tree
Python tarfile, filter=fully_trustedCPython 3.15.2➕ acceptsthe expected tree
node-tartar 7.5.22➕ acceptsthe expected tree
node-tar, stricttar 7.5.22➕ acceptsthe expected tree
tar-fstar-fs 3.1.3, on tar-stream 3.2.2➕ acceptsthe expected tree
Java Commons CompressCommons Compress 1.28.0, Expander, on Java 27.0.1.0.2➕ acceptsthe expected tree
.NET System.Formats.TarSystem.Formats.Tar of .NET 10.0.14, ExtractToDirectory➕ acceptsthe expected tree
PHP PharData, as in ComposerPharData of PHP 8.5.12, as Composer's TarDownloader calls it➕ acceptsthe expected tree
Go containerd archive.Applycontainerd v2.4.2, archive.Apply, of go1.27.3➕ acceptsthe expected tree
Rust tartar 0.4.46, unpack➕ acceptsthe expected tree
Rust astral-tokio-tarastral-tokio-tar 0.7.1, unpack➕ acceptsthe expected tree
Rust tar-codec, as in uvtar-codec 0.0.16, with the policies of uv 0.13➖ rejectserror: at byte 1024: missing two-block end-of-archive marker
RubyGemsRubyGems 4.0.20 of Ruby 4.0.9, extract_tar_gz➕ acceptsthe expected tree
R untar, internalutils::untar of R 4.6.2, tar = "internal"➕ acceptsthe expected tree
Perl Archive::TarArchive::Tar 3.12 of Perl v5.44.0➕ acceptsthe expected tree
Haskell tartar 0.7.2.0, extract, of GHC 9.14➖ rejectserror: TruncatedArchive
Julia Tar.jlTar 1.10.0 of Julia 1.13.3➕ acceptsthe expected tree
live-bootstrap untarmescc-tools-extra a151c245e512, as live-bootstrap pins it➖ rejectserror: Short read on suite/archives/F04-end-missing.tar: expected 512, got 0 The extraction of suite/archives/F04-end-missing.tar produced errors
live-bootstrap untar, --non-strictmescc-tools-extra a151c245e512, as live-bootstrap pins it➖ rejectserror: non-strict mode enabled, preparing for chaos Short read on suite/archives/F04-end-missing.tar: expected 512, got 0 The extraction of suite/archives/F04-end-missing.tar produced errors

Prevalence in release tarballs

The table below shows how many release tarballs have the features of this test case. The default policy accepts them.

ecosystemtarballswith itshare
Spack37,553120.03%
nixpkgs18,99040.02%
Guix10,16860.06%
PyPI10,0000none
crates.io5,0000none
RubyGems5,0000none
live-bootstrap1710none
total (distinct)82,586220.03%

See a sample of these 22 tarballs, with the sha256 of each and an example from it.

See also