Re: Latest LLVM breaks our code again

Speaking of buildfarm breakage, seawasp has been failing for the
past several days. It looks like bleeding-edge LLVM has again
changed some APIs we depend on. First failure is here:

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=seawasp&dt=2022-01-28%2000%3A17%3A48

regards, tom lane

On Mon, Jan 31, 2022 at 3:38 PM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Speaking of buildfarm breakage, seawasp has been failing for the
> past several days. It looks like bleeding-edge LLVM has again
> changed some APIs we depend on. First failure is here:
>
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=seawasp&dt=2022-01-28%2000%3A17%3A48

Oops, I missed this message. The patch at
/message-id/CA%2BhUKG%2BaGPBkBT5_4czkvzMu6-D%2BJgPaVL7mX_WBPXgGRndtXA%40mail.gmail.com
fixes the first two of these problems, and the third needs some more
work:

1. There's a missing #include <new> (must have been transitively
included before).
2. There's an API change that I'd written about already but hadn't
committed yet because seawasp is running behind my laptop at tracking
LLVM and I figured we might as well 'nagel' these kinds of commits.
3. We'll need to look into how to switch LLVMBuildCall -> LLVMBuildCall2 etc.

> Speaking of buildfarm breakage, seawasp has been failing for the
> past several days. It looks like bleeding-edge LLVM has again
> changed some APIs we depend on. First failure is here:
>
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=seawasp&dt=2022-01-28%2000%3A17%3A48

Indeed.

I'm sorry for the late warning: clang could not compile with its previous
version at some point so my weekly recompilation got stuck and I did not
notice for some months. I repaired by switching compiling clang with gcc
instead. ::sigh::

--
Fabien.

Hi,

On 2022-01-30 21:38:16 -0500, Tom Lane wrote:
> Speaking of buildfarm breakage, seawasp has been failing for the
> past several days. It looks like bleeding-edge LLVM has again
> changed some APIs we depend on. First failure is here:

I'm doubtful that tracking development branches of LLVM is a good
investment. Their development model is to do changes in-tree much more than we
do. Adjusting to API changes the moment they're made will often end up with
further changes to the same / related lines. Once they open the relevant
release-NN branch, it's a different story.

Maybe it'd make sense to disable --with-llvm on seawasp and have a separate
animal that tracks the newest release branch?

Greetings,

Andres Freund

Hello Andres,

> I'm doubtful that tracking development branches of LLVM is a good
> investment. Their development model is to do changes in-tree much more than we
> do. Adjusting to API changes the moment they're made will often end up with
> further changes to the same / related lines. Once they open the relevant
> release-NN branch, it's a different story.
>
> Maybe it'd make sense to disable --with-llvm on seawasp>
> and have ppppppa separate animal that tracks the newest release branch?

The point of seawasp is somehow to have an early warning of upcoming
changes, especially the --with-llvm part. LLVM indeed is a moving target,
but they very rarely back down on their API changes… As pg version are
expected to run for 5 years, they will encounter newer compiler versions,
so being as ready as possible seems worthwhile.

ISTM that there is no actual need to fix LLVM breakage on the spot, but I
think it is pertinent to be ok near release time. This is why there is a
"EXPERIMENTAL" marker in the system description. There can be some noise
when LLVM development version is broken, this has happened in the past
with seawasp (llvm) and moonjelly (gcc), but not often.

About tracking releases: this means more setups and runs and updates, and
as I think they do care about compatibility *within* a release we would
not see breakages so it would not be very useful, and we would only get
the actual breakages at LLVM release time, which may be much later.

For these reasons, I'm inclined to let seawasp as it is.

--
Fabien.

Hi,

On 2022-02-03 10:44:11 +0100, Fabien COELHO wrote:
> > I'm doubtful that tracking development branches of LLVM is a good
> > investment. Their development model is to do changes in-tree much more than we
> > do. Adjusting to API changes the moment they're made will often end up with
> > further changes to the same / related lines. Once they open the relevant
> > release-NN branch, it's a different story.
> >
> > Maybe it'd make sense to disable --with-llvm on seawasp>
> > and have ppppppa separate animal that tracks the newest release branch?
>
> The point of seawasp is somehow to have an early warning of upcoming
> changes, especially the --with-llvm part. LLVM indeed is a moving target,
> but they very rarely back down on their API changes…

The point is less backing down, but making iterative changes that require
changing the same lines repeatedly...

> About tracking releases: this means more setups and runs and updates, and as
> I think they do care about compatibility *within* a release we would not see
> breakages so it would not be very useful, and we would only get the actual
> breakages at LLVM release time, which may be much later.

LLVM's release branches are created well before the release. E.g 13 was afaics
branched off 2021-07-27 [1], released 2021-09-30 [2].

> For these reasons, I'm inclined to let seawasp as it is.

I find seawasp tracking the development trunk compilers useful. Just not for
--with-llvm. The latter imo *reduces* seawasp's, because once there's an API
change we can't see whether there's e.g. a compiler codegen issue leading to
crashes or whatnot.

What I was proposing was to remove --with-llvm from seawasp, and have a
separate animal tracking the newest llvm release branch (I can run/host that
if needed).

Greetings,

Andres Freund

[1] git show $(git merge-base origin/release/13.x origin/main)
[2] git show llvmorg-13.0.0

On Fri, Feb 4, 2022 at 8:12 AM Andres Freund <andres(at)anarazel(dot)de> wrote:
> On 2022-02-03 10:44:11 +0100, Fabien COELHO wrote:
> > For these reasons, I'm inclined to let seawasp as it is.

It might be easier to use the nightly packages at
https://apt.llvm.org/. You could update daily and still save so much
CPU that ...

> I find seawasp tracking the development trunk compilers useful. Just not for
> --with-llvm. The latter imo *reduces* seawasp's, because once there's an API
> change we can't see whether there's e.g. a compiler codegen issue leading to
> crashes or whatnot.
>
> What I was proposing was to remove --with-llvm from seawasp, and have a
> separate animal tracking the newest llvm release branch (I can run/host that
> if needed).

... you could do a couple of variations like that ^ on the same budget :-)

FWIW 14 just branched. Vive LLVM 15.