Re: [BUGS] BUG #8177: initscript should create /var/run/postgresql

The following bug has been logged on the website:

Bug reference: 8177
Logged by: stronny
Email address: stronny(at)celestia(dot)ru
PostgreSQL version: 9.2.4
Operating system: wheezy
Description:

When installed from apt.postgresql.org Postgres fails to start on system
boot.

Wheezy changed /var/run to become memory-based so initscript should create
/var/run/postgresql if necessary.

On Mon, 2013-05-27 at 09:27 +0000, stronny(at)celestia(dot)ru wrote:
> The following bug has been logged on the website:
>
> Bug reference: 8177
> Logged by: stronny
> Email address: stronny(at)celestia(dot)ru
> PostgreSQL version: 9.2.4
> Operating system: wheezy
> Description:
>
> When installed from apt.postgresql.org Postgres fails to start on system
> boot.
>
> Wheezy changed /var/run to become memory-based so initscript should create
> /var/run/postgresql if necessary.

pgsql-pkg-debian(at)postgresql(dot)org is where the people who know about this
hang out.

On Mon, 2013-05-27 at 09:27 +0000, stronny(at)celestia(dot)ru wrote:
> The following bug has been logged on the website:
>
> Bug reference: 8177
> Logged by: stronny
> Email address: stronny(at)celestia(dot)ru
> PostgreSQL version: 9.2.4
> Operating system: wheezy
> Description:
>
> When installed from apt.postgresql.org Postgres fails to start on system
> boot.
>
> Wheezy changed /var/run to become memory-based so initscript should create
> /var/run/postgresql if necessary.

start() {
# create socket directory
if [ -d /var/run/postgresql ]; then
chmod 2775 /var/run/postgresql
else
install -d -m 2775 -o postgres -g postgres /var/run/postgresql
fi

do_ctl_all start "$1" "Starting PostgreSQL $1 database server"
}

We create the directory, what specifically doesn't work for you?

Christoph
--
cb(at)df7cb(dot)de | http://www.df7cb.de/

I was able to reproduce the problem.

Package versions
-----------------
ii postgresql-9.2 9.2.4-1.pgdg60+1
object-relational SQL database, version 9.2 server
ii postgresql-common 142.pgdg60+1
PostgreSQL database-cluster manager

The failure
-----------------
# /etc/init.d/postgresql stop
# rm -rf /var/run/postgresql
# /etc/init.d/postgresql start
Error: Cannot stat /var/run/postgresql
No PostgreSQL clusters exist; see "man pg_createcluster" ... (warning).

The config file
-----------------
# ls -lA /etc/postgresql/9.2
total 0
lrwxrwxrwx 1 root root 23 May 22 20:36 main -> /var/lib/postgresql/etc
# cat /etc/postgresql/9.2/main/postgresql.conf
data_directory = '/var/lib/postgresql/9.2/main'
hba_file = '/var/lib/postgresql/etc/pg_hba.conf'
external_pid_file = '/var/run/postgresql/9.2-main.pid'
ssl = true
ssl_cert_file = '/etc/ssl/certs/ssl-cert-snakeoil.pem'
ssl_key_file = '/etc/ssl/private/ssl-cert-snakeoil.key'
log_line_prefix = '%t '
log_timezone = 'Europe/Moscow'

listen_addresses = '*'
wal_level = hot_standby
max_wal_senders = 3
wal_keep_segments = 8

On Thu, May 30, 2013 at 6:58 AM, Christoph Berg <myon(at)debian(dot)org> wrote:

> On Mon, 2013-05-27 at 09:27 +0000, stronny(at)celestia(dot)ru wrote:
> > The following bug has been logged on the website:
> >
> > Bug reference: 8177
> > Logged by: stronny
> > Email address: stronny(at)celestia(dot)ru
> > PostgreSQL version: 9.2.4
> > Operating system: wheezy
> > Description:
> >
> > When installed from apt.postgresql.org Postgres fails to start on system
> > boot.
> >
> > Wheezy changed /var/run to become memory-based so initscript should
> create
> > /var/run/postgresql if necessary.
>
> start() {
> # create socket directory
> if [ -d /var/run/postgresql ]; then
> chmod 2775 /var/run/postgresql
> else
> install -d -m 2775 -o postgres -g postgres /var/run/postgresql
> fi
>
> do_ctl_all start "$1" "Starting PostgreSQL $1 database server"
> }
>
> We create the directory, what specifically doesn't work for you?
>
> Christoph
> --
> cb(at)df7cb(dot)de | http://www.df7cb.de/
>

Re: stronny red 2013-05-30 <CALOvDf7fpScGUJBai69S3=Mr5wC-Xj5BZEqnV4qQBjOHXcj+Zg(at)mail(dot)gmail(dot)com>
> I was able to reproduce the problem.
>
> Package versions
> -----------------
> ii postgresql-9.2 9.2.4-1.pgdg60+1
> object-relational SQL database, version 9.2 server
> ii postgresql-common 142.pgdg60+1
> PostgreSQL database-cluster manager
>
> The failure
> -----------------
> # /etc/init.d/postgresql stop
> # rm -rf /var/run/postgresql
> # /etc/init.d/postgresql start
> Error: Cannot stat /var/run/postgresql

Can you try "sh -x /etc/init.d/postgresql start" there and send us the
output?

Here it does:

+ [ -d /var/run/postgresql ]
+ install -d -m 2775 -o postgres -g postgres /var/run/postgresql

> No PostgreSQL clusters exist; see "man pg_createcluster" ... (warning).

Btw, this seems to be the actual problem here.

> The config file
> -----------------
> # ls -lA /etc/postgresql/9.2
> total 0
> lrwxrwxrwx 1 root root 23 May 22 20:36 main -> /var/lib/postgresql/etc

That's a non-standard setup. Could it be the case that this partition
wasn't mounted during your tests, or something like that?

Christoph
--
cb(at)df7cb(dot)de | http://www.df7cb.de/

> On Mon, 2013-05-27 at 09:27 +0000, stronny(at)celestia(dot)ru wrote:
> > The following bug has been logged on the website:
> >
> > Bug reference: 8177
> > Logged by: stronny
> > Email address: stronny(at)celestia(dot)ru
> > PostgreSQL version: 9.2.4
> > Operating system: wheezy
> > Description:
> >
> > When installed from apt.postgresql.org Postgres fails to start on system
> > boot.
> >
> > Wheezy changed /var/run to become memory-based so initscript should create
> > /var/run/postgresql if necessary.
>
> start() {
> # create socket directory
> if [ -d /var/run/postgresql ]; then
> chmod 2775 /var/run/postgresql
> else
> install -d -m 2775 -o postgres -g postgres /var/run/postgresql
> fi
>
> do_ctl_all start "$1" "Starting PostgreSQL $1 database server"
> }
>
> We create the directory, what specifically doesn't work for you?

I've been able to reproduce this as well, and have found the root
cause. Presently, the init script itself
(/etc/init.d/postgresql) does this:

start|stop|restart|reload)
if [ -z "`pg_lsclusters -h`" ]; then
log_warning_msg 'No PostgreSQL clusters exist; see "man pg_createcluster"'
exit 0
fi

pg_lsclusters is a Perl application that gets most of its logic
from PgCommon.pm. Its entry point into PgCommon.pm (the one we
care about, anyway) is the subroutine &cluster_info.
&cluster_info, at line 575, calls &get_cluster_socketdir. This
subroutine, at line 360, gets the 'unix_socket_directories' value
from the configuration file. If the value is defined in the
configuration file (the default), it immediately returns that
value to &cluster_info. This is important. If the value is
*not* defined, then &get_cluster_socketdir defaults it to
'/var/run/postgresql' and attempts to stat() this directory. As
'/var/run' is now a tmpfs in Debian, this fails, because no
attempt has been made to create that directory before the stat()
runs.

There are actually two bugs here. The first is attempting to
stat() a directory that the code has not ensured exists, which is
what this bug is about. The second, more general, is that the
code makes no attempts at creating or even verifying the
existence of any unix_socket_{directory,directories} before
returning. I've attached a patch to the init script that ensures
/var/run/postgresql actually exists before the code tries to
stat() it.

The reproduction steps for this bug are as follows:

1. install fresh Debian wheezy machine
2. Add official Postgres mirror to apt, installing the keyring
for same
3. Install the Postgres server
4. Remove any unix_socket_{directory,directories} directive from
/etc/postgresql/9.?/main/postgresql.conf
5. Reboot, observe Postgres startup failure

I've been able to reproduce this 100% of the time with the above
steps.

--
Chris Nehren

It would help if I actually attached the patch. My apologies.

On Fri, May 30, 2014 at 13:03:16 -0400, Chris Nehren wrote:
> > On Mon, 2013-05-27 at 09:27 +0000, stronny(at)celestia(dot)ru wrote:
> > > The following bug has been logged on the website:
> > >
> > > Bug reference: 8177
> > > Logged by: stronny
> > > Email address: stronny(at)celestia(dot)ru
> > > PostgreSQL version: 9.2.4
> > > Operating system: wheezy
> > > Description:
> > >
> > > When installed from apt.postgresql.org Postgres fails to start on system
> > > boot.
> > >
> > > Wheezy changed /var/run to become memory-based so initscript should create
> > > /var/run/postgresql if necessary.
> >
> > start() {
> > # create socket directory
> > if [ -d /var/run/postgresql ]; then
> > chmod 2775 /var/run/postgresql
> > else
> > install -d -m 2775 -o postgres -g postgres /var/run/postgresql
> > fi
> >
> > do_ctl_all start "$1" "Starting PostgreSQL $1 database server"
> > }
> >
> > We create the directory, what specifically doesn't work for you?
>
> I've been able to reproduce this as well, and have found the root
> cause. Presently, the init script itself
> (/etc/init.d/postgresql) does this:
>
> start|stop|restart|reload)
> if [ -z "`pg_lsclusters -h`" ]; then
> log_warning_msg 'No PostgreSQL clusters exist; see "man pg_createcluster"'
> exit 0
> fi
>
> pg_lsclusters is a Perl application that gets most of its logic
> from PgCommon.pm. Its entry point into PgCommon.pm (the one we
> care about, anyway) is the subroutine &cluster_info.
> &cluster_info, at line 575, calls &get_cluster_socketdir. This
> subroutine, at line 360, gets the 'unix_socket_directories' value
> from the configuration file. If the value is defined in the
> configuration file (the default), it immediately returns that
> value to &cluster_info. This is important. If the value is
> *not* defined, then &get_cluster_socketdir defaults it to
> '/var/run/postgresql' and attempts to stat() this directory. As
> '/var/run' is now a tmpfs in Debian, this fails, because no
> attempt has been made to create that directory before the stat()
> runs.
>
> There are actually two bugs here. The first is attempting to
> stat() a directory that the code has not ensured exists, which is
> what this bug is about. The second, more general, is that the
> code makes no attempts at creating or even verifying the
> existence of any unix_socket_{directory,directories} before
> returning. I've attached a patch to the init script that ensures
> /var/run/postgresql actually exists before the code tries to
> stat() it.
>
> The reproduction steps for this bug are as follows:
>
> 1. install fresh Debian wheezy machine
> 2. Add official Postgres mirror to apt, installing the keyring
> for same
> 3. Install the Postgres server
> 4. Remove any unix_socket_{directory,directories} directive from
> /etc/postgresql/9.?/main/postgresql.conf
> 5. Reboot, observe Postgres startup failure
>
> I've been able to reproduce this 100% of the time with the above
> steps.
>
> --
> Chris Nehren

--
Chris Nehren