Re: [Pljava-dev] built, now how to install?

so I've built the latest off git with mvn install, on CentOS 6, with
openjdk 1.8 and postgres 9.4.latest... the 'installing on linux'
instructions seem, well, old... I don't like the idea of putting
unpackaged files in directories 'owned' by packages, like $PGHOME/lib
so I'm trying to put them in /usr/share/pljava/ ...

I think these instructions https://github.com/tada/pljava/wiki/Deployer
might need some updating, and be integrated with or linked from the
'installing from linux' page....

on
https://github.com/tada/pljava/wiki/Installing-on-linux-%28or-other-%2Anix%29

I had to use ...

export JRE_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64/jre
export LD_LIBRARY_PATH=$JRE_HOME/lib/amd64:$JRE_HOME/lib/amd64/server

(doc said /client ...)

anyways, um, i"m currently stuck at the deployer...

$ java -cp
/usr/share/pljava/pljava.jar:/usr/share/pljava/postgresql-9.4.1207.jar
org.postgresql.pljava.deploy.Deployer -install
Error: Could not find or load main class
org.postgresql.pljava.deploy.Deployer

$ psql -c "select version();"
version
---------------------------------------------------------------------------------------------------------------
PostgreSQL 9.4.5 on x86_64-unknown-linux-gnu, compiled by gcc (GCC)
4.4.7 20120313 (Red Hat 4.4.7-16), 64-bit
(1 row)

$ java -version
java version "1.7.0_91"
OpenJDK Runtime Environment (rhel-2.6.2.2.el6_7-x86_64 u91-b00)
OpenJDK 64-Bit Server VM (build 24.91-b01, mixed mode)

some notes re:
https://github.com/tada/pljava/wiki/Installing-on-linux-%28or-other-%2Anix%29

I'm pretty sure those Solaris Notes are quite obsolete. Those were my
doing some years back. I'm pretty sure the include/pg_config.h in the
standard solaris build was long ago fixed. We're no longer developing
for Solaris in my environment, so someone else will need to follow up.

I'm also sure the first paragraph is completely obsolete with respect to
1.6 not working.

GCJ probably should be completely deprecated and dropped, as its a dead
project, having java-openjdk available pretty much negates gcj's raison
d'être. I suppose the gcj note on this page, and on the building page,
could be put on their own GCJ page, along with a note suggesting its all
deprecated.

I'd do some of these edits myself but I have no idea how to submit them
to git

--
john r pierce, recycling bits in santa cruz

_______________________________________________
Pljava-dev mailing list
Pljava-dev(at)lists(dot)pgfoundry(dot)org
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev

Um, hold on a sec ...

On 01/11/16 20:59, John R Pierce wrote:
> so I've built the latest off git with mvn install, on CentOS 6, with openjdk
> 1.8 and postgres 9.4.latest... the 'installing on linux' instructions
> seem, well, old... I don't like the idea of putting unpackaged files in
> directories 'owned' by packages, like $PGHOME/lib so I'm trying to put
> them in /usr/share/pljava/ ...
>
> I think these instructions https://github.com/tada/pljava/wiki/Deployer
> might need some updating, and be integrated with or linked from the
> 'installing from linux' page....
>
> on
> https://github.com/tada/pljava/wiki/Installing-on-linux-%28or-other-%2Anix%29
>
> I had to use ...
>
> export JRE_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64/jre
> export LD_LIBRARY_PATH=$JRE_HOME/lib/amd64:$JRE_HOME/lib/amd64/server
>
>
> (doc said /client ...)
>
> anyways, um, i"m currently stuck at the deployer...
>
> $ java -cp
> /usr/share/pljava/pljava.jar:/usr/share/pljava/postgresql-9.4.1207.jar
> org.postgresql.pljava.deploy.Deployer -install
> Error: Could not find or load main class org.postgresql.pljava.deploy.Deployer
>
> $ psql -c "select version();"
> version
> ---------------------------------------------------------------------------------------------------------------
>
> PostgreSQL 9.4.5 on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.4.7
> 20120313 (Red Hat 4.4.7-16), 64-bit
> (1 row)
>
> $ java -version
> java version "1.7.0_91"
> OpenJDK Runtime Environment (rhel-2.6.2.2.el6_7-x86_64 u91-b00)
> OpenJDK 64-Bit Server VM (build 24.91-b01, mixed mode)
>
>
>
>
>
> some notes re:
> https://github.com/tada/pljava/wiki/Installing-on-linux-%28or-other-%2Anix%29
>
> I'm pretty sure those Solaris Notes are quite obsolete. Those were my
> doing some years back. I'm pretty sure the include/pg_config.h in the
> standard solaris build was long ago fixed. We're no longer developing for
> Solaris in my environment, so someone else will need to follow up.
>
> I'm also sure the first paragraph is completely obsolete with respect to 1.6
> not working.
>
> GCJ probably should be completely deprecated and dropped, as its a dead
> project, having java-openjdk available pretty much negates gcj's raison
> d'être. I suppose the gcj note on this page, and on the building page,
> could be put on their own GCJ page, along with a note suggesting its all
> deprecated.
>
> I'd do some of these edits myself but I have no idea how to submit them to git
>
>
>
>
_______________________________________________
Pljava-dev mailing list
Pljava-dev(at)lists(dot)pgfoundry(dot)org
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev

I've pushed the latest documentation at http://tada.github.io/pljava/
and updates to the wiki at https://github.com/tada/pljava/wiki.

TL;DR Deployer is an ex-thing; install.sql and uninstall.sql are
ex-things.

If you had no special preferences about where things go, you would
simply "java -jar <the-jar-file-that-gets-created-in-pljava-packaging>.jar"
and then in psql say "SET pljava.libjvm_location TO something;" (where
"something" is in your distribution is too hard to predict), and then
"CREATE EXTENSION pljava;".

The "java -jar" command would have put the files in standard pgsql
places: the extension .control file in $sharedir/extension, the .so
in $libdir, etc. Yes, that is stuffing them into PostgreSQL-owned
directories, but CREATE EXTENSION strongly assumes some of that.
Other files for $sharedir are put in a new $sharedir/pljava directory,
and not stuffed straight into $sharedir.

>> I don't like the idea of putting unpackaged files in
>> directories 'owned' by packages, like $PGHOME/lib so I'm trying to put
>> them in /usr/share/pljava/ ...

Philosophically, I am kind of with you here, I like to put things in
their own places. You can do that, by the way, with -D options on the
"java -jar" command that extracts the pieces:

java -Dpgconfig.pkglibdir=/usr/lib/pljava \
-Dpgconfig.sharedir=/usr/share/pljava -jar the-package-jar.jar

If you do that, you won't have CREATE EXTENSION "just work" until you
have set dynamic_library_path to include /usr/lib/pljava and
pljava.classpath to /usr/share/pljava/pljava/pljava-${project.version}.jar.

In other words, the extraction code built into the jar is just a dumb
extractor; it will let you override where it puts things, but it will
not edit them during extraction to hardcode the new locations in; it's
just up to you to set the configuration variables to reflect the locations
you chose. (Hmm, if you like CREATE EXTENSION, you may also have to edit
the 'directory' setting in pljava.control.)

>> I think these instructions https://github.com/tada/pljava/wiki/Deployer

404 Deployer? What deployer? :)

>> $ java -cp
>> /usr/share/pljava/pljava.jar:/usr/share/pljava/postgresql-9.4.1207.jar
>> org.postgresql.pljava.deploy.Deployer -install
>> Error: Could not find or load main class org.postgresql.pljava.deploy.Deployer

Deployer hasn't actually vanished from the build (yet), but it builds in
its own jar, it is not merged into pljava.jar.

However, I do not foresee it being maintained ... it won't be usable at
all after the next time something changes in the pljava schema, and even
until then it is still doing things the hard way.

>> I'm pretty sure those Solaris Notes are quite obsolete. Those were my
>> doing some years back. I'm pretty sure the include/pg_config.h in the
>> standard solaris build was long ago fixed. We're no longer developing for
>> Solaris in my environment, so someone else will need to follow up.

The wiki installation page now points to the tada.github.io/pljava
installation page, which has a few "special topics" subpages. If there
is still anything special about Solaris, it can go there. I didn't see
anything that seemed to be still applicable.

>> GCJ probably should be completely deprecated and dropped, as its a dead
>> project, having java-openjdk available pretty much negates gcj's raison

Granted! :)

>> I'd do some of these edits myself but I have no idea how to submit
>> them to git

Yes, github wikis are strange that way, if not world-writable they are
actually harder to submit to than the regular code repo, where you can
easily do pull requests.

But I am trying to have current, authoritative documentation on the
tada.github.io/pljava site (you find all the doc sources in src/site
and can easily do a standard github pull request). There is still
worthwhile content on the wiki, that I'd like to see moved over
gradually as time permits.

Of course, please look over the new docs and wiki to see how much
of what you were thinking of changing still needs to be changed. :)

-Chap
_______________________________________________
Pljava-dev mailing list
Pljava-dev(at)lists(dot)pgfoundry(dot)org
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev

On 1/11/2016 7:03 PM, Chapman Flack wrote:
> Of course, please look over the new docs and wiki to see how much
> of what you were thinking of changing still needs to be changed.:)

thanks muchos for all of the above, I'll tackle it from scratch in the
morning... tonights project (its 9:40pm Monday here) is rebuilding my
college daughter's laptop before she returns to school on
wednesday.... ah, I just loved trashed windows systems (NOT!)

--
john r pierce, recycling bits in santa cruz

_______________________________________________
Pljava-dev mailing list
Pljava-dev(at)lists(dot)pgfoundry(dot)org
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev

On 1/11/2016 7:03 PM, Chapman Flack wrote:
> I've pushed the latest documentation athttp://tada.github.io/pljava/
> and updates to the wiki athttps://github.com/tada/pljava/wiki.

thanks, I successfully built and tested it yesterday, quite painlessly
(grrr, why isn't maven packaged as an RPM in one of the major rpm
repos?? the one I found wanted to drag in openjdk 1.6.0)

I wrote up the build procedure for deployment at one of our other sites
where they actually need it, and got back a report that the test failed
with what appear to be character encoding errors. my test server was
running en_US.UTF8, while I suspect this production site uses
POSIX/SQLASCII... I'll rebuild on a SQLASCII test system in my lab to
see if this is the case (unluckily, the production site has already gone
home for the weekend, so I won't know what server_encoding they are
using until Monday).

--
john r pierce, recycling bits in santa cruz

_______________________________________________
Pljava-dev mailing list
Pljava-dev(at)lists(dot)pgfoundry(dot)org
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev

On 01/15/2016 07:35 AM, John R Pierce wrote:

> (grrr, why isn't maven packaged as an RPM in one of the major rpm
> repos?? the one I found wanted to drag in openjdk 1.6.0)

I don't know, but because it's pure Java and will go out itself
and fetch everything else it needs the first time you run it,
there is really no need to fret over finding an installer for it.
You just grab a tarball or zipball (why, in 2016, are we still
asking you "tar or zip?" before you can download some Java code? :)
from http://maven.apache.org/download.cgi and extract it in your
favorite temp directory and run bin/mvn and watch the downloads fly.

If you edit your .m2/settings.xml to put the repository in temp
space too, it all goes away when you no longer need it.

> test failed
> with what appear to be character encoding errors. my test server was
> running en_US.UTF8, while I suspect this production site uses
> POSIX/SQLASCII...

If the server is SQL_ASCII and the only failing test was the
UnicodeRoundTripTest, then PL/Java is working and that's as good
as it's going to get. That particular test doesn't even have a
definable meaning when the server encoding is SQL_ASCII.

To elaborate, that test is the regression test for bug 21
https://github.com/tada/pljava/issues/21 and it's there to
confirm that PL/Java will faithfully roundtrip all Unicode
characters from and to the server encoding. But out of all
the available server encodings, SQL_ASCII is the one that says
"I am ASCII for codes in the ASCII range, and I allow codes
outside that range but _nobody defines what they mean_ except
maybe the application developer, one dares to hope."

(Come to think of it, I bet my test as written will only
succeed for server encoding UTF8. It should really test that
the mapping is faithful _for all characters representable in
the server encoding_ and then it would succeed with all of the
normal, defined encodings. It would still fail with SQL_ASCII
because that's an "encoding" that's only defined for a fraction
of its codes.)

So, this site's situation can be broken into cases:

Case 1 (easiest): server encoding is SQL_ASCII, database
doesn't contain any strings with non-ASCII data (or none that
will be poked at in Java code, anyway). Everything will work
fine.

Case 2: Server encoding is SQL_ASCII, database contains some
non-ASCII data, all of which actually is in some one standard
encoding. This begs the question why not set the server encoding
to *be* that encoding and then everything will just work, but
perhaps that would involve disruptive downtime. It could be
possible to add a PL/Java GUC for that situation, something like
pljava.i_guarantee_my_sqlascii_data_actually_happens_to_be='latin1'
or whatever. That doesn't exist at the moment but would not be
prohibitively hard.

Case 3: Server encoding is SQL_ASCII, database contains some
non-ASCII data in more than one encoding, where the application
somehow knows which encoding is used where. That's completely
possible since SQL_ASCII doesn't guarantee anything; it permits
the kind of encoding flexibility the SQL standard allows by using
N'' strings, only instead of the encoding being known for each
N'' string, it isn't known anywhere but within the application.
There is no way to expect PL/Java to make sense out of that
transparently, and the rigorous solution would be for the app
to use bytea as the type of things passed to and from Java,
and explicitly apply convert_to()/convert_from() where the
application knows the encoding. It's verbose but correct;
Python 3 had to walk the same road.

I should probably put this in the docs somewhere.

Do you know which case your site is dealing with?

-Chap
_______________________________________________
Pljava-dev mailing list
Pljava-dev(at)lists(dot)pgfoundry(dot)org
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev

On 1/15/2016 7:24 AM, Chapman Flack wrote:
>> test failed
>> >with what appear to be character encoding errors. my test server was
>> >running en_US.UTF8, while I suspect this production site uses
>> >POSIX/SQLASCII...
> If the server is SQL_ASCII and the only failing test was the
> UnicodeRoundTripTest, then PL/Java is working and that's as good
> as it's going to get. That particular test doesn't even have a
> definable meaning when the server encoding is SQL_ASCII.

whale...

I re-initdb'd my test vm with POSIX/C locale, and when I load the
examples.jar, I get...

CONTEXT: SQL statement "CREATE OR REPLACE FUNCTION
javatest.complex_send(javatest.complex) RETURNS bytea LANGUAGE java
IMMUTABLE RETURNS NULL ON NULL INPUT AS
'UDT[org.postgresql.pljava.example.annotation.ComplexScalar] SEND'"
ERROR: requested character too large for encoding: 256

all the other functions load, just that last one fails. this is the
same error production showed me a screenshot of when they ran the
examples. of course, since this fails, the whole sqlj.install_jar()
fails because hte transaction rollsback.

unluckily, the main application using this database /requires/ POSIX
collation for obscure and historic reasons. its all USASCII data
anyways. I believe they used a CREATE DATABASE ... ENCODING
'SQL_ASCII' on a cluster that was initdb'd with UTF8 for equally
historic reasons (this config is a prototype, 12 hour time zone
differences, and path of least resistance).

--
john r pierce, recycling bits in santa cruz

_______________________________________________
Pljava-dev mailing list
Pljava-dev(at)lists(dot)pgfoundry(dot)org
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev

On 01/15/2016 02:31 PM, John R Pierce wrote:

> CONTEXT: SQL statement "CREATE OR REPLACE FUNCTION
> javatest.complex_send(javatest.complex) RETURNS bytea LANGUAGE java
> IMMUTABLE RETURNS NULL ON NULL INPUT AS
> 'UDT[org.postgresql.pljava.example.annotation.ComplexScalar] SEND'"
> ERROR: requested character too large for encoding: 256

Erk, that's not anything I was expecting. Not even sure what
encoding it's talking about in a function involving no strings.
I'll look tonight.

-Chap

_______________________________________________
Pljava-dev mailing list
Pljava-dev(at)lists(dot)pgfoundry(dot)org
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev

On 1/15/2016 11:36 AM, Chapman Flack wrote:
> On 01/15/2016 02:31 PM, John R Pierce wrote:
>
>> >CONTEXT: SQL statement "CREATE OR REPLACE FUNCTION
>> >javatest.complex_send(javatest.complex) RETURNS bytea LANGUAGE java
>> >IMMUTABLE RETURNS NULL ON NULL INPUT AS
>> >'UDT[org.postgresql.pljava.example.annotation.ComplexScalar] SEND'"
>> >ERROR: requested character too large for encoding: 256
> Erk, that's not anything I was expecting. Not even sure what
> encoding it's talking about in a function involving no strings.
> I'll look tonight.

ok, upon some further testing, it only does this if the ENCODING is
SQL_ASCII, its fine if the server_encoding is UTF8 but the
lc_collation/ctype are 'C'.

I note there's a almost 1 second delay between that CONTEXT and that
ERROR, suggesting the error is NOT in that CREATE OR REPLACE FUNCTION.

I tried setting log_statement = 'all' but just got...

< 2016-01-15 14:47:01.895 PST >ERROR: requested character too large for
encoding: 256
< 2016-01-15 14:47:01.895 PST >STATEMENT: SELECT
sqlj.install_jar('file:///home/piercej/pljava/pljava-examples/target/pljava-examples-1.5.0-SNAPSHOT.jar','examples',
true);

--
john r pierce, recycling bits in santa cruz

_______________________________________________
Pljava-dev mailing list
Pljava-dev(at)lists(dot)pgfoundry(dot)org
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev

On 01/15/16 17:54, John R Pierce wrote:

> I note there's a almost 1 second delay between that CONTEXT and that ERROR,
> suggesting the error is NOT in that CREATE OR REPLACE FUNCTION.

Ah, I feel better now. That would be the unicode roundtrip test ... it does
have roughly a million Unicode codepoints to check. (That wouldn't take
nearly a second, unless you wrote it in, say, SQL with CTEs, and then it
would. :)

If you set log_error_verbosity to verbose, you'll see the LOCATION is the
chr function, which is used in the Unicode test.

That's actually illustrating a strange PostgreSQL quirk ... even though
in general the encodings are all about interconvertible subsets of Unicode
(with SQL_ASCII being the only weird, undefinable one) ... the chr function
(and U'' strings) are only fully usable in UTF8, period ... and crippled
/in every other encoding/. I've never asked why....

Since that code is just a regression test, I should just change it to be
skipped unless the server is using UTF8.

-Chap
_______________________________________________
Pljava-dev mailing list
Pljava-dev(at)lists(dot)pgfoundry(dot)org
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev

On 1/15/2016 3:18 PM, Chapman Flack wrote:
> On 01/15/16 17:54, John R Pierce wrote:
>
>> >I note there's a almost 1 second delay between that CONTEXT and that ERROR,
>> >suggesting the error is NOT in that CREATE OR REPLACE FUNCTION.
> Ah, I feel better now. That would be the unicode roundtrip test ... it does
> have roughly a million Unicode codepoints to check. (That wouldn't take
> nearly a second, unless you wrote it in, say, SQL with CTEs, and then it
> would.:)

its running on a small resource VM. and it probably wasn't even a whole
second, just a noticable delay in the console output from psql

anyways, I set the log error verbosity, and this was logged...

< 2016-01-15 15:25:37.347 PST >LOG: statement: set log_error_verbosity
= 'verbose';
< 2016-01-15 15:25:40.559 PST >LOG: 00000: statement: SELECT
sqlj.install_jar('file:///home/piercej/pljava/pljava-examples/target/pljava-examples-1.5.0-SNAPSHOT.jar','examples',
true);
< 2016-01-15 15:25:40.559 PST >LOCATION: exec_simple_query, postgres.c:914
< 2016-01-15 15:25:42.544 PST >ERROR: 54000: requested character too
large for encoding: 256
< 2016-01-15 15:25:42.544 PST >LOCATION: chr, oracle_compat.c:1008
< 2016-01-15 15:25:42.544 PST >STATEMENT: SELECT
sqlj.install_jar('file:///home/piercej/pljava/pljava-examples/target/pljava-examples-1.5.0-SNAPSHOT.jar','examples',
true);

hope that helps?

--
john r pierce, recycling bits in santa cruz

_______________________________________________
Pljava-dev mailing list
Pljava-dev(at)lists(dot)pgfoundry(dot)org
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev

On 01/15/16 18:27, John R Pierce wrote:

> anyways, I set the log error verbosity, and this was logged...
> ...
> < 2016-01-15 15:25:42.544 PST >LOCATION: chr, oracle_compat.c:1008

Yep, chr.

I've just pushed a commit that skips that test unless UTF8 is the
server encoding, so it shouldn't get in your way.

-Chap
_______________________________________________
Pljava-dev mailing list
Pljava-dev(at)lists(dot)pgfoundry(dot)org
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev

On 1/15/2016 5:48 PM, Chapman Flack wrote:
> I've just pushed a commit that skips that test unless UTF8 is the
> server encoding, so it shouldn't get in your way.

fyi, built it last night, and it works for me!

thanks muchos!!

if you're ever in the greater SF Bay Area, I owe ya a beer or three :)

--
john r pierce, recycling bits in santa cruz

_______________________________________________
Pljava-dev mailing list
Pljava-dev(at)lists(dot)pgfoundry(dot)org
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev

On 1/16/2016 10:41 AM, John R Pierce wrote:
> On 1/15/2016 5:48 PM, Chapman Flack wrote:
>> I've just pushed a commit that skips that test unless UTF8 is the
>> server encoding, so it shouldn't get in your way.
>
> fyi, built it last night, and it works for me!
>
> thanks muchos!!
>
> if you're ever in the greater SF Bay Area, I owe ya a beer or three :)

and my Asian counterparts successfully got the example running, so they
are off and ready to deploy our one app that depends on pljava, yaay!

make that a beer and dinner I owe you!!

oh, and I just chatted with devrim, packager of all things RPM for
fedora/centos/rhel/etc. he's waiting for a release version, he's
already got the rpm spec ready but wants a tarball released source
rather than a code-d-jour beta off git. so the sooner 1.5.0 can be
pushed through beta, meet some minimal level of acceptance and be
released, IMHO, the better!! :)

--
john r pierce, recycling bits in santa cruz

_______________________________________________
Pljava-dev mailing list
Pljava-dev(at)lists(dot)pgfoundry(dot)org
http://lists.pgfoundry.org/mailman/listinfo/pljava-dev