Re: [SQL] Question regarding multibyte.

Hi,

I am new to postgres. We are using 8.2 release of postgres. Recently we
converted our database to multibyte on our dev machine. we want to test the
following.

1) How will I insert multibyte from our php? Do we need to use any
special encoding?
2) I am using psql command line to get the data into a flat file and
push to our data warehouse
We use IBM redbrick warehouse.
3) If I have multibyte, I know the output from the psql will scatter to
many lines. How can we avoid that. What is the sql command ? I need 1
record per row to supply to our warehouse.

Can somebody help me?

Regards
skarthi

_________________________________________________________________
Valentines Day -- Shop for gifts that spell L-O-V-E at MSN Shopping
http://shopping.msn.com/content/shp/?ctId=8323,ptnrid=37,ptnrdata=24095&tcode=wlmtagline

Hi,

I am new to postgres. I asked a question regarding multibyte display. I
got only one response.
Hence, I am re-iterating the question again to a larger audience.

We are using 8.2 release of postgres. Recently we converted our database to
multibyte on our dev machine. we want to test the following.

1) How will I insert multibyte from our php? Do we need to use any
special encoding?
2) I am using psql command line to get the data into a flat file and push
to our data warehouse
We use IBM redbrick warehouse.
3) If I have multibyte, I know the output from the psql will scatter to
many lines. How can we avoid that. What is the sql command ? I need 1
record per row to supply to our warehouse.

Can somebody help me?

Regards
skarthi

_________________________________________________________________
Valentines Day -- Shop for gifts that spell L-O-V-E at MSN Shopping
http://shopping.msn.com/content/shp/?ctId=8323,ptnrid=37,ptnrdata=24095&tcode=wlmtagline

On Sun, Feb 04, 2007 at 01:34:41PM -0800, Karthikeyan Sundaram wrote:
> I am new to postgres. I asked a question regarding multibyte display. I
> got only one response.
> Hence, I am re-iterating the question again to a larger audience.

pgsql-general would probably be the largest audience; maybe somebody
with access to list statistics can confirm or correct.

I responded to your original message in pgsql-admin; maybe you
haven't seen the response yet:

http://archives.postgresql.org/pgsql-admin/2007-02/msg00043.php

If the comments aren't helpful then please provide more information
about the difficulties you're having. A minimal but complete example
that illustrates the problem might be useful.

--
Michael Fuhr

Karthikeyan Sundaram wrote:
> Hi,
>
> I am new to postgres. I asked a question regarding multibyte
> display. I got only one response.
> Hence, I am re-iterating the question again to a larger audience.
>
> We are using 8.2 release of postgres. Recently we converted our
> database to multibyte on our dev machine. we want to test the following.
>
> 1) How will I insert multibyte from our php? Do we need to use any
> special encoding?
> 2) I am using psql command line to get the data into a flat file and
> push to our data warehouse
> We use IBM redbrick warehouse.
> 3) If I have multibyte, I know the output from the psql will scatter
> to many lines. How can we avoid that. What is the sql command ? I need
> 1 record per row to supply to our warehouse.
>
> Can somebody help me?
>
> Regards
> skarthi

http://www.php.net/mbstring
http://www.php.net/iconv

Those are the two pages that will give you more information about your
needs. But these aren't the only factors when sending data collected
via php to postgres. There's also your webserver and the user's browser
to account for, both of which handle data according to the defaults of
the browser.

My solution to this problem was to standardize on utf-8 and then use
mbstring and iconv to convert everything to utf-8. It's still not
perfect as there are still characters that aren't converted properly.
But by and large, it's extremely rare considering the amount of data
we're dealing with.

We also chose to use PHP to pull the data out of db and write to file.
By doing this, we were able to make sure the data was encoded correctly.

Charley

Am Sonntag, 4. Februar 2007 22.34 schrieb Karthikeyan Sundaram:
> Hi,
>
> I am new to postgres. I asked a question regarding multibyte display.
> I got only one response.

Which was correct and about complet when taking the amount of information
given into account. So nobody else did see a need for an other answer.

> Hence, I am re-iterating the question again to a larger audience.
>
> We are using 8.2 release of postgres. Recently we converted our database
> to multibyte on our dev machine. we want to test the following.
>
> 1) How will I insert multibyte from our php? Do we need to use any
> special encoding?

You need to specify the client encoding of your php installation. (the last
time I used it I think it was Latin1 for me) So after the connection is
established send something like "set client_encoding='LATIN1'" to the server.
From then on the server will automatically convert between the client and the
server encoding for you.

> 2) I am using psql command line to get the data into a flat file and
> push to our data warehouse
> We use IBM redbrick warehouse.

Here you need to find a way which makes your shell and the data warehouse
happy. It depends on the data you have to transfer what will give the best
solution. Here a very simple one which might create troubles in some special
cases:
If you know the encoding your ware house is expecting just set the client
encoding of the psql to what you ware house expects. Now the ware house
should be able to read what from psql comes where mostpossible at the console
you will not get the correct display anymore whit this setup.

> 3) If I have multibyte, I know the output from the psql will scatter to
> many lines. How can we avoid that. What is the sql command ? I need 1
> record per row to supply to our warehouse.

This is most probably only a problem of the editor/console related to wrong
encoding settings and does not have anything to do with the real problem.
If this is not the case you have to escape the linebreakes in a way the
warehouse understands (iconv, recode,...).

>
> Can somebody help me?

I hope so.

Best regards
Ivo Rossacher
>
> Regards
> skarthi
>
> _________________________________________________________________
> Valentines Day -- Shop for gifts that spell L-O-V-E at MSN Shopping
> http://shopping.msn.com/content/shp/?ctId=8323,ptnrid=37,ptnrdata=24095&tco
>de=wlmtagline
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org

Hi Gurus,

I hae installed postgres 8.2 recently and when I open the psql command
line prompt and say \d <table name>. The psql abort abruptly with an error
message given below.

What is the cause and how to rectify it?

Error Message
=========
Welcome to psql 8.2.1, the PostgreSQL interactive terminal.

Type: \copyright for distribution terms
\h for help with SQL commands
\? for help with psql commands
\g or terminate with semicolon to execute query
\q to quit

podcast=> \d channel
psql: symbol lookup error: psql: undefined symbol: PQescapeStringConn

Regards
skarthi

_________________________________________________________________
Check out all that glitters with the MSN Entertainment Guide to the Academy
Awards http://movies.msn.com/movies/oscars2007/?icid=ncoscartagline2

"Karthikeyan Sundaram" <skarthi98(at)hotmail(dot)com> writes:
> podcast=> \d channel
> psql: symbol lookup error: psql: undefined symbol: PQescapeStringConn

It looks like your psql has linked to an obsolete version of libpq.so,
though it's not real clear why the thing would have started at all if
that were the situation ...

regards, tom lane

Hi,

I am new to postgres. I need some kind of template script or advise on
how to analyse the indexes. In our database, we do delete, insert, update
tons of rows.

Regards
skarthi

_________________________________________________________________
Refi Now: Rates near 39yr lows! $430,000 Mortgage for $1,399/mo - Calculate
new payment
http://www.lowermybills.com/lre/index.jsp?sourceid=lmb-9632-17727&moid=7581

Karthikeyan Sundaram wrote on 18.02.2007 09:15:
> Hi,
>
> I am new to postgres. I need some kind of template script or advise
> on how to analyse the indexes. In our database, we do delete, insert,
> update tons of rows.
>
http://www.postgresql.org/docs/8.2/interactive/routine-reindex.html

What version of Postgres are you using?

Charley

Karthikeyan Sundaram wrote:
> Hi,
>
> I am new to postgres. I need some kind of template script or advise
> on how to analyse the indexes. In our database, we do delete, insert,
> update tons of rows.
>
> Regards
> skarthi
>
> _________________________________________________________________
> Refi Now: Rates near 39yr lows! $430,000 Mortgage for $1,399/mo -
> Calculate new payment
> http://www.lowermybills.com/lre/index.jsp?sourceid=lmb-9632-17727&moid=7581
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend
>