Re: pg_dump end comment

This might seem a bit silly, but is there any chance we could add a
comment at the end of pg_dump text output that says '-- End of dump'?

Would make it useful for checking that you actually have a complete dump...

Chris

Christopher Kings-Lynne <chriskl(at)familyhealth(dot)com(dot)au> writes:
> This might seem a bit silly, but is there any chance we could add a
> comment at the end of pg_dump text output that says '-- End of dump'?

Sure --- while you're at it, put a "beginning of dump" at the start.

Is it worth adding the database name and/or other command-line
parameters given to pg_dump?

One thing to think about is the difference between a bare pg_dump and a
pg_dump/pg_restore sequence. Should these always generate identical
text output? (They do as of CVS tip, I believe, though this was not
always true before.)

Possibly this is all gilding the lily though...

regards, tom lane

Tom Lane wrote:
> Christopher Kings-Lynne <chriskl(at)familyhealth(dot)com(dot)au> writes:
> > This might seem a bit silly, but is there any chance we could add a
> > comment at the end of pg_dump text output that says '-- End of dump'?
>
> Sure --- while you're at it, put a "beginning of dump" at the start.
>
> Is it worth adding the database name and/or other command-line
> parameters given to pg_dump?
>
> One thing to think about is the difference between a bare pg_dump and a
> pg_dump/pg_restore sequence. Should these always generate identical
> text output? (They do as of CVS tip, I believe, though this was not
> always true before.)
>
> Possibly this is all gilding the lily though...

I like an end-of-dump marker for folks who want to check if the dump got
truncated somehow. I can see how to do that for text dumps, but what
about for tar or custom dumps?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On Tue, 30 Mar 2004, Bruce Momjian wrote:

> Tom Lane wrote:
> > Christopher Kings-Lynne <chriskl(at)familyhealth(dot)com(dot)au> writes:
> > > This might seem a bit silly, but is there any chance we could add a
> > > comment at the end of pg_dump text output that says '-- End of dump'?
> >
> > Sure --- while you're at it, put a "beginning of dump" at the start.
> >
> > Is it worth adding the database name and/or other command-line
> > parameters given to pg_dump?
> >
> > One thing to think about is the difference between a bare pg_dump and a
> > pg_dump/pg_restore sequence. Should these always generate identical
> > text output? (They do as of CVS tip, I believe, though this was not
> > always true before.)
> >
> > Possibly this is all gilding the lily though...
>
> I like an end-of-dump marker for folks who want to check if the dump got
> truncated somehow. I can see how to do that for text dumps, but what
> about for tar or custom dumps?

Wouldn't it be more effective to test for non zero return status as this
handles -Fc cases, etc, which would be non-trivial to test.

Gavin

Gavin Sherry wrote:
> On Tue, 30 Mar 2004, Bruce Momjian wrote:
>
> > Tom Lane wrote:
> > > Christopher Kings-Lynne <chriskl(at)familyhealth(dot)com(dot)au> writes:
> > > > This might seem a bit silly, but is there any chance we could add a
> > > > comment at the end of pg_dump text output that says '-- End of dump'?
> > >
> > > Sure --- while you're at it, put a "beginning of dump" at the start.
> > >
> > > Is it worth adding the database name and/or other command-line
> > > parameters given to pg_dump?
> > >
> > > One thing to think about is the difference between a bare pg_dump and a
> > > pg_dump/pg_restore sequence. Should these always generate identical
> > > text output? (They do as of CVS tip, I believe, though this was not
> > > always true before.)
> > >
> > > Possibly this is all gilding the lily though...
> >
> > I like an end-of-dump marker for folks who want to check if the dump got
> > truncated somehow. I can see how to do that for text dumps, but what
> > about for tar or custom dumps?
>
> Wouldn't it be more effective to test for non zero return status as this
> handles -Fc cases, etc, which would be non-trivial to test.

Yes, they have to check for a proper exit from pg_dump, but there is
still a file sitting around after the dump, with no way to tell if it is
accurate.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Gavin Sherry <swm(at)linuxworld(dot)com(dot)au> writes:
> On Tue, 30 Mar 2004, Bruce Momjian wrote:
>> I like an end-of-dump marker for folks who want to check if the dump got
>> truncated somehow. I can see how to do that for text dumps, but what
>> about for tar or custom dumps?

> Wouldn't it be more effective to test for non zero return status as this
> handles -Fc cases, etc, which would be non-trivial to test.

In the tar and custom archive formats, it's fairly obvious if the file
was truncated: you'll have table-of-contents entries that point past
the end of file. The only case that really needs an endmarker is
textual scripts --- both straight from pg_dump, and produced by
pg_restore.

regards, tom lane

At 12:13 AM 31/03/2004, Bruce Momjian wrote:
>Yes, they have to check for a proper exit from pg_dump, but there is
>still a file sitting around after the dump, with no way to tell if it is
>accurate.

Why don't we write a hash into the header or footer. Then use something like:

pg_restore --verify <dump-file>

if file integrity is the objective.

----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.B.N. 75 008 659 498) | /(@) ______---_
Tel: (+61) 0500 83 82 81 | _________ \
Fax: (+61) 03 5330 3172 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp.mit.edu:11371 |/

>>I like an end-of-dump marker for folks who want to check if the dump got
>>truncated somehow. I can see how to do that for text dumps, but what
>>about for tar or custom dumps?
>
> Wouldn't it be more effective to test for non zero return status as this
> handles -Fc cases, etc, which would be non-trivial to test.

That assumes you are there at the time of dump...

Chris

On Wed, 31 Mar 2004, Philip Warner wrote:

> At 12:13 AM 31/03/2004, Bruce Momjian wrote:
> >Yes, they have to check for a proper exit from pg_dump, but there is
> >still a file sitting around after the dump, with no way to tell if it is
> >accurate.
>
> Why don't we write a hash into the header or footer. Then use something like:
>
> pg_restore --verify <dump-file>
>
> if file integrity is the objective.

I like this idea. Nice to be able to check the md5 sig to make sure a
backup is not corrupted OR short.

Tom Lane wrote:
> Christopher Kings-Lynne <chriskl(at)familyhealth(dot)com(dot)au> writes:
> > This might seem a bit silly, but is there any chance we could add a
> > comment at the end of pg_dump text output that says '-- End of dump'?
>
> Sure --- while you're at it, put a "beginning of dump" at the start.
>
> Is it worth adding the database name and/or other command-line
> parameters given to pg_dump?
>
> One thing to think about is the difference between a bare pg_dump and a
> pg_dump/pg_restore sequence. Should these always generate identical
> text output? (They do as of CVS tip, I believe, though this was not
> always true before.)

I have added code to print end-of-dump markers for pg_dump and
pg_dumpall:

--
-- PostgreSQL database dump complete
--

--
-- PostgreSQL database cluster dump complete
--

This mirrors the markers used at the start of the files.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073