Re: BUG #13733: ~/.psql_history* corrupted

The following bug has been logged on the website:

Bug reference: 13733
Logged by: Emmanuel Sambo
Email address: news(at)emmanuelsambo(dot)com
PostgreSQL version: 9.4.5
Operating system: Apple Mac OS X El Capitan 10.11.1
Description:

Upgrading from PostgreSQL 9.4.4 to 9.4.5 deleted my entire psql history in
two of my ~/.psql_history-<db>
Looking at this file I can see:
- all the spaces are encoded as utf-8: "\040"
- each history entry is truncated at 1021 characters

news(at)emmanuelsambo(dot)com writes:
> The following bug has been logged on the website:
> Bug reference: 13733
> Logged by: Emmanuel Sambo
> Email address: news(at)emmanuelsambo(dot)com
> PostgreSQL version: 9.4.5
> Operating system: Apple Mac OS X El Capitan 10.11.1
> Description:

> Upgrading from PostgreSQL 9.4.4 to 9.4.5 deleted my entire psql history in
> two of my ~/.psql_history-<db>
> Looking at this file I can see:
> - all the spaces are encoded as utf-8: "\040"
> - each history entry is truncated at 1021 characters

This probably means you accidentally switched from using libedit to
libreadline, or vice versa, when you rebuilt Postgres. They're not
very compatible as to history file format :-(

regards, tom lane

I figured it out. The homebrew 9.4.5 El Capitan bottle is bad because of
Xcode. I documented it in https://github.com/Homebrew/homebrew/issues/45405

On Mon, Oct 26, 2015 at 11:08 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> news(at)emmanuelsambo(dot)com writes:
> > The following bug has been logged on the website:
> > Bug reference: 13733
> > Logged by: Emmanuel Sambo
> > Email address: news(at)emmanuelsambo(dot)com
> > PostgreSQL version: 9.4.5
> > Operating system: Apple Mac OS X El Capitan 10.11.1
> > Description:
>
> > Upgrading from PostgreSQL 9.4.4 to 9.4.5 deleted my entire psql history
> in
> > two of my ~/.psql_history-<db>
> > Looking at this file I can see:
> > - all the spaces are encoded as utf-8: "\040"
> > - each history entry is truncated at 1021 characters
>
> This probably means you accidentally switched from using libedit to
> libreadline, or vice versa, when you rebuilt Postgres. They're not
> very compatible as to history file format :-(
>
> regards, tom lane
>

On 10/26/15 12:08 PM, Tom Lane wrote:
> news(at)emmanuelsambo(dot)com writes:
>> The following bug has been logged on the website:
>> Bug reference: 13733
>> Logged by: Emmanuel Sambo
>> Email address: news(at)emmanuelsambo(dot)com
>> PostgreSQL version: 9.4.5
>> Operating system: Apple Mac OS X El Capitan 10.11.1
>> Description:
>
>> Upgrading from PostgreSQL 9.4.4 to 9.4.5 deleted my entire psql history in
>> two of my ~/.psql_history-<db>
>> Looking at this file I can see:
>> - all the spaces are encoded as utf-8: "\040"
>> - each history entry is truncated at 1021 characters
>
> This probably means you accidentally switched from using libedit to
> libreadline, or vice versa, when you rebuilt Postgres. They're not
> very compatible as to history file format :-(

I have noticed that building PostgreSQL with libxml support on El
Capitan by default ends up building against the system libedit, because
it has a symlink at /usr/lib/libreadline.dylib, and xml2-config points
to that directory, so it ends up early in the search path.

Since I don't have the pre-El Capitan installation anymore, and can't
verify whether this changed, but the problem is pretty clear.

For the time being, I'm working around it like this:

XML2_CONFIG=:
CPPFLAGS=-I/usr/include/libxml2

Of course, building without libxml would also work.

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> I have noticed that building PostgreSQL with libxml support on El
> Capitan by default ends up building against the system libedit, because
> it has a symlink at /usr/lib/libreadline.dylib, and xml2-config points
> to that directory, so it ends up early in the search path.

> Since I don't have the pre-El Capitan installation anymore, and can't
> verify whether this changed, but the problem is pretty clear.

Hm, I do not see anything like that on Yosemite:

$ which xml2-config
/usr/bin/xml2-config
$ xml2-config
Usage: xml2-config [OPTION]

Known values for OPTION are:

--prefix print the install prefix
--libs print library linking information
--cflags print pre-processor and compiler flags
--modules module support enabled
--help display this help and exit
--version output version information
$ xml2-config --libs
-lxml2
$ xml2-config --cflags
-I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/libxml2
$ xml2-config --prefix
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr

The /usr/lib/libreadline.dylib symlink has been there pretty much
forever, so that's not a new hazard.

You didn't say, but I assume what you're seeing is that "xml2-config
--cflags" now emits some -L switches along with -I?

Maybe we could fix that by inserting xml2-config's switches after the
user's, instead of before.

regards, tom lane

On 11/2/15 2:14 PM, Tom Lane wrote:
> $ xml2-config --libs
> -lxml2
> $ xml2-config --cflags
> -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/libxml2
> $ xml2-config --prefix
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr
>
> The /usr/lib/libreadline.dylib symlink has been there pretty much
> forever, so that's not a new hazard.
>
> You didn't say, but I assume what you're seeing is that "xml2-config
> --cflags" now emits some -L switches along with -I?

xml2-config --libs emits -L switches that point into Xcode, similar to
the -I switch shown above.

But after running

sudo xcode-select -s /Library/Developer/CommandLineTools

as suggested in the links mentioned earlier in the thread, this goes
away. Apparently, Apple fixed this in one of their minor software updates.

This has been fixed in https://github.com/Homebrew/homebrew/issues/45405
Thank you everyone for your valuable help

On Mon, Nov 2, 2015 at 3:03 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:

> On 11/2/15 2:14 PM, Tom Lane wrote:
> > $ xml2-config --libs
> > -lxml2
> > $ xml2-config --cflags
> >
> -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/libxml2
> > $ xml2-config --prefix
> >
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr
> >
> > The /usr/lib/libreadline.dylib symlink has been there pretty much
> > forever, so that's not a new hazard.
> >
> > You didn't say, but I assume what you're seeing is that "xml2-config
> > --cflags" now emits some -L switches along with -I?
>
> xml2-config --libs emits -L switches that point into Xcode, similar to
> the -I switch shown above.
>
> But after running
>
> sudo xcode-select -s /Library/Developer/CommandLineTools
>
> as suggested in the links mentioned earlier in the thread, this goes
> away. Apparently, Apple fixed this in one of their minor software updates.
>
>