Re: 2019-03 CF Summary / Review - Tranche #1

Hi,

As last year [1], I'll try to summarize all commitfest items in 2019-03
to see which I think could realistically be put into 12.

Going through all non bugfix CF entries. Here's the summary for the
entries I could stomach today:

RFC: ready for committer
NR: needs review
WOA: waiting on author.

- pgbench - another attempt at tap test for time-related options

NR. This was already around last year. I think it'd be fair to argue
that there's not been a ton of push to get this committed.

- Psql patch to show access methods info

This adds \ commands to display many properties of [index ]access methods.

NR. This patch has gotten a fair bit of committer feedback via
Alvaro. I personally am not particularly convinced this is
functionally that many people are going to use.

- Show size of partitioned table

NR. There seems to have been plenty discussion over details. Feels
like this ought to be committable for v12?

- pgbench - add pseudo-random permutation function

WOA. I'm not clear as to why we'd want to add this to pgbench. To
revive a discussion from last year's thread, I feel like we're adding
more code to pgbench than we can actually usefully use.

- libpq host/hostaddr consistency

NR. I think the patches in this needs a few more committer eyes. It's
possible we just should fix the documentation, or go further and
change the behaviour. Feels like without more senior attention,
this'll not be resolved.

- pg_dump multi VALUES INSERT

NR. There seems to be some tentative agreement, excepting Tom, that
we probably want this feature. There has been plenty review /
improvements.

Seems like it ought to be possible to get this into v12.

- libpq trace log

NR. There seems to be considerable debate about what exactly this
feature should do, and the code hasn't yet seen a lot of review. I
think we ought to just target v13 here, there seems to be some
agreement that there's a problem, just not exactly what the solution
is.

Andres: punt to v13.

- pg_dumpall --exclude-database option

RFC, and author is committer.

- Add sqlstate output mode to VERBOSITY

RFC, there seems to be agreement.

- DECLARE STATEMENT syntax support in ECPG

NR. There seems to be some tentative agreement that this is
desirable. But the patch was only recently (2018-12-16) revived, and
hasn't yet gotten a ton of review. I pinged Michael Meskes, to make
him aware of this work.

Andres: punt to v13.

- A new data type 'bytea' for ECPG host variable

NR: This seems to be much closer to being ready than the
above. Michael has done a few review cycles. Not sure if it'll be
done by 12, but it seems to have a good chance.

- \describe: verbose commands in psql

NR: This seems like a relatively large change, and only submitted
2019-01-24. Per our policy to not add nontrivial work to the last CF,
I think we should not consider this a candidate for v12.

Andres: punt to v13.

- documenting signal handling with readme

WOA: I'm very unclear as to why this is something worth documenting in
this manner. Right now I'm not clear what the audience of this is
supposed to be.

- First SVG graphic

NR: My reading of the discussion makes this look like we'll probably
have graphics in v12's docs. Neat.

- Update INSTALL file

WOA: It seems we've not really come to much of a conclusion what this
ought to contain.

- Make installcheck-world in a clean environment

NR: I don't feel we really have agreement on what we want here. Thus
I'm doubtful this is likely to be resolvable in short order.

Andres: lightly in favor of punting to v13

- Avoid creation of the free space map for small tables

NR: the majority of this patch has been committed, I assume the
remaining pieces will too.

- Adding a TAP test checking data consistency on standby with
minRecoveryPoint

NR: Seems like it ought to be committable, Andrew Gierth did provide
some feedback.

- idle-in-transaction timeout error does not give a hint

NR: Seems trivial enough.

- commontype and commontypearray polymorphic types

NR: To me this seems like a potentially large change, with user
visible impact. As it was only submitted in December, and is clearly
not yet reviewed much, I think we ought to punt on this for v12.

Andres: punt to v13

- EXPLAIN with information about modified GUC values

NR: The patch seems to be getting closer to completion, but I'm not
sure how many actually agree that we want this.

Andres: aim for v12, but we probably should discuss soon whether we
actually want this.

- Include all columns in default names for foreign key constraints.

WOA: This patch has been submitted to the last CF first. I think it's
straddling the line after which we should just refuse that pretty
closely. Not sure.

- Shared-memory based stats collector

WOA: I think this patch has quite some promise and solve quite a few
issues, and it has been worked on for a while. But at the same time
the code isn't that close to being committable.

Andres: unless there's a new version cleaning up review comments PDQ,
I think we're unfortunately have to punt this to v13.

- timeout parameters in libpq

NR: This doesn't yet seem terribly well reviewed and designed.

Andres: +0.5 for punting to v13

- Log a sample of transactions

WOA: Seems like a useful enough feature to me, but there were a few
other senior community members that didn't quite agree. Issues in the
patch seem resolvable in time for v12. I'm wondering if this doesn't
need a more radical approach to avoid the overhead.

- monitoring CREATE INDEX [CONCURRENTLY]

NR: I think there's agreement on the desirability of the feature. But
while the patch has been submitted to CF 2019-01 that seems to
have been somewhat of a placeholder entry. OTOH, it's a committer's
project, so we can give more leeway there.

- pg_stat_statements should notice FOR UPDATE clauses

NR: This seems like to get in, given how sipmle it is. Some quibbles
about the appropriate approach aside.

Ok, my flight's about to land. So that's it for this round.

Greetings,

Andres Freund

[1] https://postgr.es/m/20180301110344.kyp3wejoxp2ipler%40alap3.anarazel.de

On Thu, Feb 14, 2019 at 12:37:52PM -0800, Andres Freund wrote:
> - Adding a TAP test checking data consistency on standby with
> minRecoveryPoint
>
> NR: Seems like it ought to be committable, Andrew Gierth did provide
> some feedback.

I am rather happy of the shape of the test, and was just waiting from
Andrew for a last confirmation. If there are no objections, I could
commit it as well.
--
Michael

On Fri, Feb 15, 2019 at 2:08 AM Andres Freund <andres(at)anarazel(dot)de> wrote:
>
> Hi,
>
> - Avoid creation of the free space map for small tables
>
> NR: the majority of this patch has been committed,
>

This is a correct assessment.

> I assume the
> remaining pieces will too.
>

Yes, I am busy these days with something else, but I will surely get
back to reviewing/committing the remaining stuff for PG12.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Hi,

On 2019-02-14 12:37:52 -0800, Andres Freund wrote:
> - pg_stat_statements should notice FOR UPDATE clauses
>
> NR: This seems like to get in, given how sipmle it is. Some quibbles
> about the appropriate approach aside.
>
>
> Ok, my flight's about to land. So that's it for this round.

- Protect syscache from bloating with negative cache entries

WOA: I think unless the feature is drastically scaled down in scope,
there's no chance to get anything into v12.

Andres: punt to v13, unless a smaller chunk can be split off

- SERIALIZABLE with parallel query

NR: This seems like it's pretty much committable, and the author is a
committer these days.

- Removing [Merge]Append nodes which contain a single subpath

NR: I can't quite tell the state of this patch just by reading the
thread. It's longstanding, and the code doesn't look terrible, but Tom
appears to still be a bit unhappy.

Andres: ???

- verify ALTER TABLE SET NOT NULL by valid constraints

NR: Seems like a pretty useful feature. The code isn't very
invasive. The patch has been lingering around for a while. We should
merge this.

- Reduce amount of WAL generated by CREATE INDEX for gist, gin and
sp-gist

NR: This unfortunately has barely gotten any review so far, and has
had a number of issues authors have discovered themselves. It's a bit
sad that a useful patch has gotten this little review, but I think
it's probably a stretch to get it into v12 unless some senior
reviewers show up.

- GiST VACUUM

NR: Has gotten a fair bit of review by Heikki, but there still seems
to be a number of unresolved issues. Not sure if there's cycles to get
this done unless Heikki has time.

- libpq compression

NR: While a very useful feature, the patch seems pretty raw, there's
disagreements about code structure, and some cryptographic risks need
to be discussed or at least better documented.

Andres: punt to v13

- Evaluate immutable functions during planning (in FROM clause)

NR: As far as I can tell this CF entry should have been either WO or
even rejected for the last two CFs. Even if the review feedback had
been addressed, it seems there's very substantial architecture
concerns that haven't been responded to.

Andres: no chance for v12, CF entry should probably be closed.

- Global shared meta cache

NR: This is extremely invasive, and is more PoC work than anything
else.

Andres: punt to v13

- Remove self join on a unique column

NR: This still seems pretty raw, and there's not been a ton of
detailed review (although plenty of more general discussion). I don't
quite see how we can get this into shape for v12, although some review
would be good to allow the feature to progress.

Andres: punt to v13

- Inline Common Table Expressions

NR: I think it's likely that Tom will commit this soon, we're mostly
debating syntax and similar things at this point (and man, I'm looking
forward to this).

- Autoprepare: implicitly replace literals with parameters and store
generalized plan

NR: I think there's no chance to get this into v12, given the state of
the patch. There's not even agreement that we want this feature
(although I think we can't avoid it for much longer), not to speak of
agreement on the architecture.

I think this needs a lot more attention to ever get anywhere.

Andres: punt to v13

- Tid scan improvements (ordering and range scan)

NR: The patch has been through recent significant architectural
changes, albeit to an architecture more similar to an earlier
approach. There's not been meaningful review since. On the other hand,
the patch isn't actually all that complex. Seems like a stretch to get
into v12, but possible if e.g. Tom wants to pick it up.

Andres: +0.5 for punting to v13

- Block level parallel vacuum

NR: Cool feature, but I don't think this has gotten even remotely
enough review to be mergable into v12.

Andres: punt to v13

- Speed up planning with partitions

NR: Important feature, but based on a skim of the discussion this
doesn't seem ready for v12.

Andres: punt to v13

- Make nbtree keys unique by appending heap TID, suffix truncation

NR: Important, seemingly carefully worked on feature. But it's
*complicated* stuff, and there's only been a bit of design review by
Heikki. The author's a committer. I don't feel qualified to judge
this.

- KNN for B-tree

NR: The patch still seems to lack a bit of review. Possible but a
stretch for v12. (While the thread is quite old, there've been
substantial gaps where it wasn't worked on, so I don't think it's one
of the really bad cases of not getting review.)

Andres: +0.5 for punting to v13

- New vacuum option to do only freezing

NR: Seems simple enough. We probably can merge this.

- Speed up transaction completion faster after many relations are
accessed in a transaction

NR: This has only been submitted 2019-02-12. While not a really
complicated patch, it's also not trivial. Therefore I'd say this falls
under our policy of not merging nontrivial patches first submitted to
the last CF.

Andres: punt to v13

- SortSupport implementation on inet/cdir

WOA: This is a substantial amount of code submitted first for the last
CF.

Andres: punt to v13

- Referential Integrity Checks with Statement Level Triggers

NR: This has only been submitted to this CF, and is a very substantial
change. There has been no review as of yet, and the author
acknowledges several shortcomings in the patch.

Andres: punt to v13

- postgres_fdw: Perform UPPERREL_ORDERED and UPPERREL_FINAL steps
remotely

WOA: This is a nontrivial change, and the design and review only
started in late December. It's probably not realistic to target v12.

Andres: punt to v13

- Delay locking partitions during query execution

NR: Important patch. Development has only started in December. Not a
ton of code though.

Andres: ???

- Delay locking partitions during INSERT and UPDATE

RFC: Looks reasonable enough, although there's some discussion related
to increased deadlock risk. Re-raised that on the thread.

- Prove IS NOT NULL inference for large arrays

NR: No idea.

- Detoast Compressed Datum Slice

NR: Probably just can get committed close to as-is. Pinged Stephen,
who mentioned he's interested in committing it.

- Ordered Partitioned Table Scans

NR: Patch has been through a few rounds, but probably needs a look
soon by somebody else with a fair bit of planner experience. Might be
doable.

- schema variables, LET command

NR: For a feature that's as user exposed as this, I don't think this
has gotten even remotely enough review.

Andres: punt to v13

- block level PRAGMA

NR: My reading of this thread is that the proposal is closer to being
rejected than merged.

Andres: reject or punt?

- get rid of StdRdOptions, use individual binary reloptions
representation for each relation kind instead

NR: I'm not sure I understand what this going to buy us.

Andres: ???

- Track the next xid using 64 bits

WOA: Can probably be merged, I posted a few relatively minor review
comments, but I assume Thomas is going to merge an updated version.

- Refactoring the checkpointer's fsync request queue

NR: There's been some design issues raised (by your's truly, in
person). And there's very likely not going to be an in-core user for
this in v12 (neither slru-via-bufmgr or undo seem likely to get
merged). So I'm not sure it's realistic to get this into v12, although
it certainly seems doable if a bit of elbow grease is put into it.

- Making WAL receiver startup rely on GUC context for primary_conninfo
and primary_slot_name

NR: I think this should be rejected. As noted in the thread, this
doesn't actually buy us much, and has some potential costs once we
make primary_conninfo PGC_SIGHUP.

Andres: Reject

- Respect client-initiated CopyDone during logical streaming replication

NR: I don't think this is ready.

Andres: punt to v13

- logical decoding of two-phase transactions

WOA: This is probably not going to be ready by v13, although I think
it could become so if somebody senior really started working on it.

Andres: punt to v13 :(

- logical streaming for large in-progress transactions

WOA: Tomas, at the dev meeting in Brussels, said he doesn't believe
this is going to be ready for v12.

Andres: punt to v13 :(

- Restricting maximum keep segments by repslots

WOA: This seems to need substantial polishing before being
ready. Might be doable before v13, but the author also is involved in
numerous other patches needing work...

- Synchronous replay mode for avoiding stale reads on hot standbys

NR: There's some disagreement about the desirability of the feature,
but plenty people signalled they want it. Seems like it ought to get
merged at some point, there's been review (but more probably wouldn't
hurt).

- Copy function for logical replication slots

WOA: Probably can be committed, was briefly marked RFC, but I found
some issues (which should be easy enough to rectify).

- pg_rewind: options to use restore_command from recovery.conf or
command line

WOA: Was previously marked as RFC, but I don't see how it is. Possibly
can be finished, but does require a good bit more work.

- create and use subscription for nonsuperuser

NR: This seems to need a good bit more work. I'm a bit doubtful this
can be finished for v12.

Andres: +0.25 for punting to v13

- online change primary_conninfo

WOA: Needs a bit more work, but probably can be finished for v12.

- Remove deprecated exclusive backup mode

NR: There clearly seems to be no concensus on making this change.

Andres: punt to v13 or something

- Add timeline to partial WAL segments

WOA: Seems to need a good bit more work, and touches sensitive bits.

Andres: +0.5 for punting to v13

- Synchronizing slots from primary to standby

NR: This clearly is just a POC at this point.

Andres: punt to v13

- pg_hba.conf : new auth option : clientcert=verify-full

NR: this should probably be RFC, as it was before needing to be
rebased. Looks like it should just get merged.

- GSSAPI encryption support

NR: Seems Stephen is pondering committing this. Not quite sure I like
the way it's integreated in fe-secure/be-secure.

- multivariate MCV lists and histograms

WOA: Seems the MCV bits might be realistic for v12, but the histogram
part not?

- Push aggregation down to base relations and joins

NR: Seems like this needs a few more review, and is probably not quite
going to be ready for v12. But it'd probably need some attention by
Tom for the author to be able to move forward.

Andres: push to v13

- Pluggable storage API

NR: I'm biased... I plan to push substantial portions of this
feature. There's a few later features that I'm not sure are going to
be ready (e.g. doing trigger lookups using a snapshot).

- Custom compression methods

WOA: Hm.

Andres: I think we need to make a call whether we actually want this,
rather than just continuing to punt this forward.

- BRIN bloom and multi-minmax indexes

NR: Unfortunately this doesn't seem to have gotten meaningful review
in the last year :(

- SQL/JSON: jsonpath

WOA: This seems to need substantial further work before being
committable

Andres: +0.5 for punting to v13

(okay, breaking open a bottle of wine here)

- Add enum relation option type

RFC: I think Alvaro probably can commit this? He's edited a few
versions of the patch, and set the target version to 12.

- Covering GiST indexes

RFC

- amcheck verification for GiST

WOA: Some locking changes are apparently needed, possibly a bit too
much to fix up for v12?

- DNS SRV support for LDAP authentication

NR: Looks like Thomas should just merge this

- Add Hook Functions for Disk Quota Extension

NR: This is in the early design stages, rather than realistically
targeting v12.

Andres: punt to v13

- Implement NULL-related checks in object address functions to prevent cache lookup errors, take two

NR: Seems like this should go into 12, although it'd be good if Alvaro
could take another peek before Michael pushes.

- Triggers on Materialized Views

NR: I think we need to provide useful feedback whether we actually
want this feature. But either way, it's not v12 material.

Andres: punt to v13, discuss whether to reject

- Ltree syntax improvement

NR: Given this is a nontrivial patch, and was submitted 2019-01-29,
it's clearly not v12 material.

Andres: punt to v13

- Skip table truncation at VACUUM (should be: allow to disable
truncations via a reloptions)

NR: The feature is near trivial, and avoids significant problems in
hot standby environments. It seems to need some language lawyering.

- SQL/JSON: functions

NR: Dependant on jsonpath which I have a hard time seeing in v12. And
it's barely reviewed (and still contains exciting PG_CATCH games that
I warned about at the tail end of v11...).

Andres: punt to v13

- SQL/JSON: JSON_TABLE

NR: Depends on previous, no review.

Andres: punt to v13

- chained transactions

NR: Looks like it ought to be committable

- conflict handling for COPY FROM

NR: Clearly not ready for v12, the whole business with requiring a log
file doesn't seem acceptable.

Andres: punt to v13

- FETCH FIRST clause PERCENT option

WOA: Clearly not ready for v12.

Andres: punt to v13

- ALTER TABLE on system catalogs

NR: I still think this is the wrong approach, but I can also live with
Peter hacking this up. But I think a call got to be made at some
point, rather than schlepping this around continuously.

- ATTACH/DETACH PARTITION CONCURRENTLY

NR: Seems to be getting closer to being mergable.

- FETCH FIRST clause WITH TIES option

NR: Doesn't quite seem ready, insufficient tests, new code probably
should be in separate function. But possibly could be fixed up for
v12?

- support VARIADIC arg for least/greatest functions

NR: There's debate whether we want this feature. Tom argues, and I'm
inclined to agree, that this should rather be separate array specific
functions. Pavel's position is that that's a separate thing, but I'm
not sure I agree.

Andres: reject?

- Temporary materialized views

NR: Some infrastructure work is needed before this can go in. Not sure
if that can be finished for v12?

- insensitive/non-deterministic collations

NR: Peter has stated that he's targeting v12, but I'm not sure this
had enough review? But it's not *that* complicated...

- Log10 and hyperbolic functions for SQL:2016 compliance

NR: Seems simple enough, we should just merge this.

- pg_upgrade: Pass -j option down to vacuumdb

NR: Seems simple enough, we should just merge this.

- Support huge_pages on AIX

NR: Probably can be merged.

There's a few patches that authors, rather than others, have market as
targeting v13, or where authors consented to that. I don't see a need to
go through those here.

- Generic type subscripting
- Transactions involving multiple postgres foreign servers
- Undo logs
- Undo worker and transaction rollback
- Index Skip Scan
- SERIALIZABLE on standby servers
- Advanced partition matching for partition-wise join

Comments?

Greetings,

Andres Freund

Hi

> - block level PRAGMA
>
> NR: My reading of this thread is that the proposal is closer to being
> rejected than merged.
>
> Andres: reject or punt?
>
>
This patch is very simple and has strong sense for users of
plpgsql_checks. For first moment, only plpgsql_check's users can get some
advance from it. But if we implement autonomous transactions, and I hope so
this feature will be implemented, then this code can be used for Oracle's
PL/SQL syntax compatible implementation. There is not any disadvantage - it
is clean, and compatible with ADA, PL/SQL .. I implemented just only block
level PRAGMA, and there was not any disagreement.

Regards

Pavel

On Fri, Feb 15, 2019 at 9:45 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
> - Make nbtree keys unique by appending heap TID, suffix truncation
>
> NR: Important, seemingly carefully worked on feature. But it's
> *complicated* stuff, and there's only been a bit of design review by
> Heikki. The author's a committer. I don't feel qualified to judge
> this.

I think that's fair. The best way of understanding the risks as a
non-expert is to think of the patch as having two distinct components:

1. Make nbtree entries unique + add suffix truncation -- the basic mechanism.

2. Make more intelligent decisions about where to split pages, to work
with suffix truncation, while still mostly caring about the balance of
free space among each half of the split, and caring about a number of
other new concerns besides these two.

You're right that some parts of the patch are very complicated, but
those are all contained in the second component. That has been the
main focus of Heikki's review, by far. This second component is
concerned with picking a split point that is already known to be legal
based on the *existing* criteria. If there were bugs here, they could
not result in data corruption. The worst outcome I can conceive of is
that an index would be bloated in a new and novel way. It would be
possible to correct that in a point release without breaking on-disk
compatibility. That would be painful, certainly, but it's far from the
worst outcome.

Granted, there are also one or two subtle things in the first, more
critical component, but these are also the things that were
established earliest and have received the most testing. And, amcheck
is now capable of doing point lookups using the same code paths as
index scans (calling _bt_search()) to relocate each and every tuple on
the leaf level, starting from the root. The first component does not
change anything about how crash recovery or VACUUM works, either. It's
all about how keys compare, and how new pivot tuples are generated --
it's mostly about the key space, while changing very little about the
physical on-disk representation. (It builds on the on-disk
representation changes added in Postgres 11, for INCLUDE indexes.)

> - SortSupport implementation on inet/cdir
>
> WOA: This is a substantial amount of code submitted first for the last
> CF.
>
> Andres: punt to v13

I was kind of involved here. I think that it's fair to punt, based on
the rule about submitting a big patch to the last CF.

> - amcheck verification for GiST
>
> WOA: Some locking changes are apparently needed, possibly a bit too
> much to fix up for v12?

I had hoped that Andrey Borodin would get back to me on this soon. It
does still seem unsettled.

> - insensitive/non-deterministic collations
>
> NR: Peter has stated that he's targeting v12, but I'm not sure this
> had enough review? But it's not *that* complicated...

I could help out here.

--
Peter Geoghegan

Andres Freund <andres(at)anarazel(dot)de> writes:
> - Inline Common Table Expressions

> NR: I think it's likely that Tom will commit this soon, we're mostly
> debating syntax and similar things at this point (and man, I'm looking
> forward to this).

Unless there are a bunch of votes real soon against the [NOT] MATERIALIZED
syntax, I'm going to commit it that way. "Real soon" means "probably
tomorrow".

regards, tom lane

On Fri, Feb 15, 2019 at 09:45:26PM -0800, Andres Freund wrote:
> - Making WAL receiver startup rely on GUC context for primary_conninfo
> and primary_slot_name
>
> NR: I think this should be rejected. As noted in the thread, this
> doesn't actually buy us much, and has some potential costs once we
> make primary_conninfo PGC_SIGHUP.
>
> Andres: Reject

I am not surprised by your opinion here, you stated it clearly on the
thread :) I have just marked it as rejected, as I don't have the
energy to fight for it.

> - online change primary_conninfo
>
> WOA: Needs a bit more work, but probably can be finished for v12.

Yep, agreed.

> - Add timeline to partial WAL segments
>
> WOA: Seems to need a good bit more work, and touches sensitive bits.
>
> Andres: +0.5 for punting to v13

The problem is not that easy, particularly to make things consistent
between the backend and pg_receivewal.

> - Implement NULL-related checks in object address functions to prevent cache lookup errors, take two
>
> NR: Seems like this should go into 12, although it'd be good if Alvaro
> could take another peek before Michael pushes.

I would prefer if Alvaro has an extra look at what I am proposing as
most of the stuff in objectaddress.c is originally his.

> - Temporary materialized views
>
> NR: Some infrastructure work is needed before this can go in. Not sure
> if that can be finished for v12?

I would suggest to move that to v13. Relation creation done by CTAS
needs to be refactored first, and my take is that we could have this
refactoring part for v12, moving the core portion of the proposal to
the next dev cycle.
--
Michael

On Sat, Feb 16, 2019 at 8:45 AM Andres Freund <andres(at)anarazel(dot)de> wrote:
> - SQL/JSON: jsonpath
>
> WOA: This seems to need substantial further work before being
> committable
>
> Andres: +0.5 for punting to v13

I'm going to post updated patchset next week. All the issues
highlighted will be resolved there. So, let's don't decide this too
early.

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Hi,

On February 16, 2019 11:22:32 AM PST, Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru> wrote:
>On Sat, Feb 16, 2019 at 8:45 AM Andres Freund <andres(at)anarazel(dot)de>
>wrote:
>> - SQL/JSON: jsonpath
>>
>> WOA: This seems to need substantial further work before being
>> committable
>>
>> Andres: +0.5 for punting to v13
>
>I'm going to post updated patchset next week. All the issues
>highlighted will be resolved there. So, let's don't decide this too
>early.

Well, given that the patches still have a lot of the same issues complained about a year ago, where people said we should try to get them into v11, I'm not sure that that's a realistic goal. Jsonb was a success, but also held up the release by several months.

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

On Sat, Feb 16, 2019 at 11:31 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
> On February 16, 2019 11:22:32 AM PST, Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru> wrote:
> >On Sat, Feb 16, 2019 at 8:45 AM Andres Freund <andres(at)anarazel(dot)de>
> >wrote:
> >> - SQL/JSON: jsonpath
> >>
> >> WOA: This seems to need substantial further work before being
> >> committable
> >>
> >> Andres: +0.5 for punting to v13
> >
> >I'm going to post updated patchset next week. All the issues
> >highlighted will be resolved there. So, let's don't decide this too
> >early.
>
> Well, given that the patches still have a lot of the same issues complained about a year ago, where people said we should try to get them into v11, I'm not sure that that's a realistic goal.

I'm sorry, a year ago I didn't understand this issue correctly.
Otherwise, I would push people to do something more productive during
this year.

If solution I'm going to post next week wouldn't be good enough, there
is a backup plan. We can wipe out error suppression completely. Then
we implement less part of standard, but still can get something very
useful into core.

> Jsonb was a success, but also held up the release by several months.

I don't ask to commit patchset in current shape and help up the
release because of it. I just say there is still time before FF. So,
let's not doom patchset too early.

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On Sun, Feb 17, 2019 at 1:09 AM Alexander Korotkov
<a(dot)korotkov(at)postgrespro(dot)ru> wrote:
>
> On Sat, Feb 16, 2019 at 11:31 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
> > On February 16, 2019 11:22:32 AM PST, Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru> wrote:
> > >On Sat, Feb 16, 2019 at 8:45 AM Andres Freund <andres(at)anarazel(dot)de>
> > >wrote:
> > >> - SQL/JSON: jsonpath
> > >>
> > >> WOA: This seems to need substantial further work before being
> > >> committable
> > >>
> > >> Andres: +0.5 for punting to v13
> > >
> > >I'm going to post updated patchset next week. All the issues
> > >highlighted will be resolved there. So, let's don't decide this too
> > >early.
> >
> > Well, given that the patches still have a lot of the same issues complained about a year ago, where people said we should try to get them into v11, I'm not sure that that's a realistic goal.
>
> I'm sorry, a year ago I didn't understand this issue correctly.
> Otherwise, I would push people to do something more productive during
> this year.
>
> If solution I'm going to post next week wouldn't be good enough, there
> is a backup plan. We can wipe out error suppression completely. Then
> we implement less part of standard, but still can get something very
> useful into core.

To be more clear. In both options I'm NOT proposing to commit any
PG_TRY/PG_CATCH anymore :)

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

> - Prove IS NOT NULL inference for large arrays
>
> NR: No idea.

As the fairly new author of this patch, my perspective is that this
patch got quite a bit of review, albeit without a formal "yes or no"
response.

I'm obviously interested in getting it committed, and I believe it's a
fairly simple patch to look at (though since it's in predtest probably
requires extra brain cycles to be careful we don't make spurious
assumptions in the optimizer). It's also an obvious performance win
for queries that can use partial indexes with almost no additional
optimizer overhead.

But I'm also interested in feedback on how patches like this work in
the review process -- particularly when given the fair amount of
discussion/cleanup but without a final word. As a new patch author a
lot of understanding how this works feels very much like a "learn as
you go", but since there's not a lot of information directly written
on it I figured asking explicitly is the best way to learn the process
better.

Thanks,
James Coleman

James Coleman <jtc331(at)gmail(dot)com> writes:
>> - Prove IS NOT NULL inference for large arrays
>>
>> NR: No idea.

> As the fairly new author of this patch, my perspective is that this
> patch got quite a bit of review, albeit without a formal "yes or no"
> response.

FWIW, I ran out of time for that patch during the January 'fest, but
I think it still has a good shot at getting committed during March.

regards, tom lane

From: Andres Freund [mailto:andres(at)anarazel(dot)de]
> - Protect syscache from bloating with negative cache entries
>
> WOA: I think unless the feature is drastically scaled down in scope,
> there's no chance to get anything into v12.
>
> Andres: punt to v13, unless a smaller chunk can be split off

At a glance, the patch set looks big divided in 5 patch files, but the actual size may get much smaller by merging those files and excluding 004 (statistics views). I'm reviewing this. I wish this could not be given up yet...

> - Speed up planning with partitions
>
> NR: Important feature, but based on a skim of the discussion this
> doesn't seem ready for v12.
>
> Andres: punt to v13

> - Speed up transaction completion faster after many relations are
> accessed in a transaction
>
> NR: This has only been submitted 2019-02-12. While not a really
> complicated patch, it's also not trivial. Therefore I'd say this falls
> under our policy of not merging nontrivial patches first submitted to
> the last CF.
>
> Andres: punt to v13

I hope these will be continued in PG 12, because these will make PostgreSQL comparable to a commercial database in OLTP workloads. I'd like to hear from Amit and David on the feasibility.

Regards
Takayuki Tsunakawa

Hi,

On 2019/02/18 11:13, Tsunakawa, Takayuki wrote:
> From: Andres Freund [mailto:andres(at)anarazel(dot)de]
>> - Speed up planning with partitions
>>
>> NR: Important feature, but based on a skim of the discussion this
>> doesn't seem ready for v12.
>>
>> Andres: punt to v13
>
>> - Speed up transaction completion faster after many relations are
>> accessed in a transaction
>>
>> NR: This has only been submitted 2019-02-12. While not a really
>> complicated patch, it's also not trivial. Therefore I'd say this falls
>> under our policy of not merging nontrivial patches first submitted to
>> the last CF.
>>
>> Andres: punt to v13
>
> I hope these will be continued in PG 12, because these will make PostgreSQL comparable to a commercial database in OLTP workloads. I'd like to hear from Amit and David on the feasibility.

As far as the first one is concerned (speed up planning with partitions),
although there's no new functionality [1], there is quite a bit of code
churn affecting somewhat complex logic of inheritance planning. As I
mentioned in the FOSDEM developer meeting's patch triage discussion,
there's a chance of moving this forward if Tom has time to look at some
portions of these patches. David's and Imai-san's review over the past
few months has been very helpful to get the patches to the state they are
in now.

As for the 2nd one, while I can say it's really helpful for workloads with
many partitions, I have to admit I can't give an expert opinion on the
code changes. Thank you for working on it.

Thanks,
Amit

[1] it does enable hash partition pruning for update/delete queries
though, which is a new feature

(2019/02/16 14:45), Andres Freund wrote:
> - postgres_fdw: Perform UPPERREL_ORDERED and UPPERREL_FINAL steps
> remotely
>
> WOA: This is a nontrivial change, and the design and review only
> started in late December. It's probably not realistic to target v12.
>
> Andres: punt to v13

I also think this needs more reviews, but I don't think it's unrealistic
to target v12, because 1) the patch is actually not that large (at least
in the latest version, most of the changes are in regression tests), and
2) IMO the patch is rather straightforward.

Best regards,
Etsuro Fujita

On Mon, 18 Feb 2019 at 15:50, Amit Langote
<Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp> wrote:
> On 2019/02/18 11:13, Tsunakawa, Takayuki wrote:
> > From: Andres Freund [mailto:andres(at)anarazel(dot)de]
> >> - Speed up planning with partitions
> >>
> >> NR: Important feature, but based on a skim of the discussion this
> >> doesn't seem ready for v12.
> >>
> >> Andres: punt to v13
> >
> >> - Speed up transaction completion faster after many relations are
> >> accessed in a transaction
> >>
> >> NR: This has only been submitted 2019-02-12. While not a really
> >> complicated patch, it's also not trivial. Therefore I'd say this falls
> >> under our policy of not merging nontrivial patches first submitted to
> >> the last CF.
> >>
> >> Andres: punt to v13
> >
> > I hope these will be continued in PG 12, because these will make PostgreSQL comparable to a commercial database in OLTP workloads. I'd like to hear from Amit and David on the feasibility.
>
> As far as the first one is concerned (speed up planning with partitions),
> although there's no new functionality [1], there is quite a bit of code
> churn affecting somewhat complex logic of inheritance planning.

I think we need to treat each patch in that series individually. Last
time I looked, the first 1 or 2 patches looked very close. I know
there's some tricky stuff in later patches in the series, but I don't
think that should stop earlier patches being committed for PG12.
However, I'd say that if the entire thing is to make PG12 then we'll
need to start ticking off earlier patches pretty soon, likely before
March.

--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 2/16/19 7:45 AM, Andres Freund wrote:

> - Add timeline to partial WAL segments
>
> WOA: Seems to need a good bit more work, and touches sensitive bits.
>
> Andres: +0.5 for punting to v13
I have labelled this patch v13 and I'll push it as soon as the CF app
allows me to do so.

I think this is important but there is a lot of tricky work to be done
in pg_receivewal and it's best not to rush that. Michael and I have an
agreement in principle but ...

Regards,
--
-David
david(at)pgmasters(dot)net

Hi,

On 2019-02-18 18:59:22 +0200, David Steele wrote:
> On 2/16/19 7:45 AM, Andres Freund wrote:
>
> > - Add timeline to partial WAL segments
> >
> > WOA: Seems to need a good bit more work, and touches sensitive bits.
> >
> > Andres: +0.5 for punting to v13
> I have labelled this patch v13 and I'll push it as soon as the CF app allows
> me to do so.
>
> I think this is important but there is a lot of tricky work to be done in
> pg_receivewal and it's best not to rush that. Michael and I have an
> agreement in principle but ...

FWIW, given that we now can filter by targetting v12 (although it'd be
nice if it were possible to filter by v12 or NULL), I don't think
there's any urgency to push to the next CF immediately.

Greetings,

Andres Freund

On 2/18/19 7:03 PM, Andres Freund wrote:
> Hi,
>
> On 2019-02-18 18:59:22 +0200, David Steele wrote:
>> On 2/16/19 7:45 AM, Andres Freund wrote:
>>
>>> - Add timeline to partial WAL segments
>>>
>>> WOA: Seems to need a good bit more work, and touches sensitive bits.
>>>
>>> Andres: +0.5 for punting to v13
>> I have labelled this patch v13 and I'll push it as soon as the CF app allows
>> me to do so.
>>
>> I think this is important but there is a lot of tricky work to be done in
>> pg_receivewal and it's best not to rush that. Michael and I have an
>> agreement in principle but ...
>
> FWIW, given that we now can filter by targetting v12 (although it'd be
> nice if it were possible to filter by v12 or NULL), I don't think
> there's any urgency to push to the next CF immediately.

Agreed. This new feature helps a lot.

Even so, I'll push it when I can to get it out of my hair, as it were.
I'll be spending a lot of time look at that list next month.

--
-David
david(at)pgmasters(dot)net

David Steele <david(at)pgmasters(dot)net> writes:
> Even so, I'll push it when I can to get it out of my hair, as it were.
> I'll be spending a lot of time look at that list next month.

Can't you do it now? The status summary line already shows one
patch as having been pushed to the next CF.

regards, tom lane

On 2019-02-18 12:37:27 -0500, Tom Lane wrote:
> David Steele <david(at)pgmasters(dot)net> writes:
> > Even so, I'll push it when I can to get it out of my hair, as it were.
> > I'll be spending a lot of time look at that list next month.
>
> Can't you do it now? The status summary line already shows one
> patch as having been pushed to the next CF.

It's CF app nannyism. One can't move a patch to the next CF that's
waiting-on-author. I've complained about that a number of times, but...

Andres Freund <andres(at)anarazel(dot)de> writes:
> On 2019-02-18 12:37:27 -0500, Tom Lane wrote:
>> Can't you do it now? The status summary line already shows one
>> patch as having been pushed to the next CF.

> It's CF app nannyism. One can't move a patch to the next CF that's
> waiting-on-author. I've complained about that a number of times, but...

So change it to another state, push it, change it again.

regards, tom lane

On 2/18/19 12:43 PM, Tom Lane wrote:
> Andres Freund <andres(at)anarazel(dot)de> writes:
>> On 2019-02-18 12:37:27 -0500, Tom Lane wrote:
>>> Can't you do it now? The status summary line already shows one
>>> patch as having been pushed to the next CF.
>> It's CF app nannyism. One can't move a patch to the next CF that's
>> waiting-on-author. I've complained about that a number of times, but...
> So change it to another state, push it, change it again.
>
>

I'm with Andres. I found this annoying six months ago.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Andrew Dunstan <andrew(dot)dunstan(at)2ndquadrant(dot)com> writes:
> On 2/18/19 12:43 PM, Tom Lane wrote:
>> Andres Freund <andres(at)anarazel(dot)de> writes:
>>> It's CF app nannyism. One can't move a patch to the next CF that's
>>> waiting-on-author. I've complained about that a number of times, but...

>> So change it to another state, push it, change it again.

> I'm with Andres. I found this annoying six months ago.

Oh, I agree the restriction is stupid. I'm just pointing out that
it can be gotten around.

regards, tom lane

On 2/19/19 1:23 AM, Tom Lane wrote:
> Andrew Dunstan <andrew(dot)dunstan(at)2ndquadrant(dot)com> writes:
>> On 2/18/19 12:43 PM, Tom Lane wrote:
>>> Andres Freund <andres(at)anarazel(dot)de> writes:
>>>> It's CF app nannyism. One can't move a patch to the next CF that's
>>>> waiting-on-author. I've complained about that a number of times, but...
>
>>> So change it to another state, push it, change it again.
>
>> I'm with Andres. I found this annoying six months ago.
>
> Oh, I agree the restriction is stupid. I'm just pointing out that
> it can be gotten around.

OK, yeah, that worked. For some reason I was having trouble moving
things out of the 2019-01 CF last month but this time it worked just
fine. I think more than one was marked as open then, not sure.

--
-David
david(at)pgmasters(dot)net

(2019/02/18 12:40), Etsuro Fujita wrote:
> (2019/02/16 14:45), Andres Freund wrote:
>> - postgres_fdw: Perform UPPERREL_ORDERED and UPPERREL_FINAL steps
>> remotely
>>
>> WOA: This is a nontrivial change, and the design and review only
>> started in late December. It's probably not realistic to target v12.
>>
>> Andres: punt to v13
>
> I also think this needs more reviews, but I don't think it's unrealistic
> to target v12, because 1) the patch is actually not that large (at least
> in the latest version, most of the changes are in regression tests), and
> 2) IMO the patch is rather straightforward.

There seems to be no objections, so I marked this as targeting v12.

Best regards,
Etsuro Fujita

On 16.02.2019 8:45, Andres Freund wrote:
> - pg_rewind: options to use restore_command from recovery.conf or
> command line
>
> WOA: Was previously marked as RFC, but I don't see how it is. Possibly
> can be finished, but does require a good bit more work.

Just sent new version of the patch to the thread [1], which removes all
unnecessary complexity. I am willing to address all new issues during
2019-03 CF if any.

[1]
/message-id/c9cfabce-8fb6-493f-68ec-e0a72d957bf4%40postgrespro.ru

Thanks

--
Alexey Kondratov