Re: Correct Unicode sorting depends on how initdb was run

Lists: Postg윈 토토SQL : Postg윈 토토SQL 메일 링리스트 : 2003-08-11 이후 PGSQL-BUGS
From: Nils Philippsen <nphilipp(at)redhat(dot)com>
To: pgsql-bugs(at)postgresql(dot)org
Subject: Correct Unicode sorting depends on how initdb was run
Date: 2003-08-11 06:37:37
Message-ID: 1060583857.3676.7.camel@wombat.dialup.fht-esslingen.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-bugs

Hi there,

Recently I stumbled over a very strange problem: I had two very similar
setups (RHL9 with latest updates, pgsql-7.3.2, parameters in "show all"
the same, databases with encoding=UNICODE, loaded from the same database
dump) where the sorting on one was erroneous with regards to accented
characters.

After hours of fiddling I found out that the erroneous one was initdb'ed
with locale set to en_US, while the one correctly sorting was initdb'ed
with locale set to en_US.utf-8. I pg_dumpall'ed the wrong one, redid the
initdb with locale set to en_US.utf-8 and loaded the dumped databases,
now the sorting order was correct.

Is this expected behaviour (I do not think so)?

Nils
--
Nils Philippsen / Red Hat / nphilipp(at)redhat(dot)com
"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety." -- B. Franklin, 1759
PGP fingerprint: C4A8 9474 5C4C ADE3 2B8F 656D 47D8 9B65 6951 3011


From: Peter Eisentraut <peter_e(at)gmx(dot)net>
To: Nils Philippsen <nphilipp(at)redhat(dot)com>
Cc: pgsql-bugs(at)postgresql(dot)org
Subject: Re: Correct Unicode sorting depends on how initdb was run
Date: 2003-08-11 08:49:43
Message-ID: Pine.LNX.4.56.0308111049230.1333@krusty.localdomain
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: Postg윈 토토SQL : Postg윈 토토SQL 메일 링리스트 : 2003-08-11 이후 PGSQL-BUGS

Nils Philippsen writes:

> Is this expected behaviour

Yes.

--
Peter Eisentraut peter_e(at)gmx(dot)net


From: Nils Philippsen <nphilipp(at)redhat(dot)com>
To: Peter Eisentraut <peter_e(at)gmx(dot)net>
Cc: pgsql-bugs(at)postgresql(dot)org
Subject: Re: Correct Unicode sorting depends on how initdb was run
Date: 2003-08-11 11:27:37
Message-ID: 1060599991.3147.10.camel@gibraltar.stuttgart.redhat.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-bugs

On Mon, 2003-08-11 at 10:49, Peter Eisentraut wrote:
> Nils Philippsen writes:
>
> > Is this expected behaviour
>
> Yes.

Hmm. I ask myself whether this is desired behaviour, too.

Given that this isn't obviously documented (at least I didn't find it),
I'd expect sort order to be dependent on server_encoding or
client_encoding, but not on a locale setting that was present at
initialisation of the database structures (and which isn't changeable
except by dump&reload).

Nils
--
Nils Philippsen / Red Hat / nphilipp(at)redhat(dot)com
"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety." -- B. Franklin, 1759
PGP fingerprint: C4A8 9474 5C4C ADE3 2B8F 656D 47D8 9B65 6951 3011


From: Peter Eisentraut <peter_e(at)gmx(dot)net>
To: Nils Philippsen <nphilipp(at)redhat(dot)com>
Cc: pgsql-bugs(at)postgresql(dot)org
Subject: Re: Correct Unicode sorting depends on how initdb was run
Date: 2003-08-11 11:48:11
Message-ID: Pine.LNX.4.56.0308111347300.1333@krusty.localdomain
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-bugs

Nils Philippsen writes:

> On Mon, 2003-08-11 at 10:49, Peter Eisentraut wrote:
> > Nils Philippsen writes:
> >
> > > Is this expected behaviour
> >
> > Yes.
>
> Hmm. I ask myself whether this is desired behaviour, too.

No, but it will take a lot of work to fix this, such as implementing our
own locale library.

--
Peter Eisentraut peter_e(at)gmx(dot)net


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Peter Eisentraut <peter_e(at)gmx(dot)net>
Cc: Nils Philippsen <nphilipp(at)redhat(dot)com>, pgsql-bugs(at)postgresql(dot)org
Subject: Re: Correct Unicode sorting depends on how initdb was run
Date: 2003-08-11 14:52:59
Message-ID: 2442.1060613579@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-bugs

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> Nils Philippsen writes:
>> Hmm. I ask myself whether this is desired behaviour, too.

> No, but it will take a lot of work to fix this, such as implementing our
> own locale library.

We should, however, look into using C99-spec <wctype.h> routines where
available --- the existing logic that depends on <ctype.h> stuff cannot
work with multibyte encodings. I am not sure if this has any
user-visible effects beyond upper()/lower().

regards, tom lane