Re: pg_basebackup failed to back up large file

On 2014-06-03 17:57:52 +0200, Magnus Hagander wrote:
> On Tue, Jun 3, 2014 at 5:42 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> > What we had better do, IMO, is fix things so that we don't have a filesize
> > limit in the basebackup format. After a bit of googling, I found out that
> > recent POSIX specs for tar format include "extended headers" that among
> > other things support member files of unlimited size [1]. Rather than
> > fooling with partial fixes, we should make the basebackup logic use an
> > extended header when the file size is over INT_MAX.

> Yeah, pax seems to be the way to go. It's at least supported by GNU tar -
> is it also supported on say BSD, or other popular platforms? (The size
> extension in the general ustar format seems to be, so it would be a shame
> if this one is less portable)

PG's tar.c already uses the ustar format and the referenced extension is
an extension to ustar as far as I understand it. So at least tarballs
with files < 8GB would still continue to be readable with all currently
working implementations.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services