Discussion:
Bug#830888: icinga2-ido-pgsql: dbconfig configuration fails with unix sockets
Christoph Anton Mitterer
2016-07-12 16:37:52 UTC
Permalink
Package: icinga2-ido-pgsql
Version: 2.4.10-1~bpo8+1
Severity: important


Hi.

On jessie with the backports package,... when selecting to use unix sockets
during dbconfig, it apparently still tries to create the DB/etc. using
TCP and an error appears:

An error occurred while installing the database:
psql: could not connect to server: Connection refused Is the server running on host "localhost" (::1) and accepting TCP/IP connections on port 5432? could not connect to server:
Connection refused Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5432

Of course there is nothing to connect there as I've configured my postgresql
server to only listen on sockets.

Also tried it with dbconfig-common 2.0.4~bpo8+1, same problem.
Perhaps it's also a bug there, if so please reassign :)

Thanks,
Chris.


-- System Information:
Debian Release: stretch/sid
APT prefers unstable-debug
APT policy: (500, 'unstable-debug'), (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.6.0-1-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_DE.UTF-8, LC_CTYPE=en_DE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
Christoph Anton Mitterer
2016-07-12 17:04:25 UTC
Permalink
Even if one enables TCP, it doesn't work:
2016-07-12 18:45:55 CEST [8172-2] ***@template1 LOG:  connection authorized: user=icinga database=template1 SSL enabled (protocol=TLSv1.2, cipher=ECDHE-RSA-AES256-GCM-SHA384, compression=off)
2016-07-12 18:45:55 CEST [8202-1] [unknown]@[unknown] LOG:  connection received: host=::1 port=39691
2016-07-12 18:45:55 CEST [8202-2] ***@template1 LOG:  connection authorized: user=icinga database=template1 SSL enabled (protocol=TLSv1.2, cipher=ECDHE-RSA-AES256-GCM-SHA384, compression=off)
2016-07-12 18:45:55 CEST [8202-3] ***@template1 ERROR:  permission denied
2016-07-12 18:45:55 CEST [8202-4] ***@template1 STATEMENT:  ALTER USER "icinga"
2016-07-12 18:45:55 CEST [8246-1] [unknown]@[unknown] LOG:  connection received: host=::1 port=39692
2016-07-12 18:45:55 CEST [8246-2] ***@template1 LOG:  connection authorized: user=icinga database=template1 SSL enabled (protocol=TLSv1.2, cipher=ECDHE-RSA-AES256-GCM-SHA384, compression=off)
2016-07-12 18:45:55 CEST [8246-3] ***@template1 ERROR:  permission denied to create role
2016-07-12 18:45:55 CEST [8246-4] ***@template1 STATEMENT:  CREATE USER "icinga" WITH PASSWORD 'VfCrhGSrkbDW'



It tries to create roles as icinga (even though I've given postgres as
db admin user)... and even though I've left the password empty, it
tries to apply one, which is not ideal either.


Cheers,
Chris.
Christoph Anton Mitterer
2016-07-18 23:35:13 UTC
Permalink
Control: reassign -1 dbconfig-common
Control: retitle -1 dbconfig configuration fails with unix sockets

I've just checked the whole package again, and nothing in it seems to
so contain anything about tcp or UNIX sockets... so it rather seems to
me that this might be a problem in dbconfig, thus reassigning and
kindly asking its maintainers to have a look.

Cheers.
Paul Gevers
2016-08-13 20:19:56 UTC
Permalink
Control: tags -1 moreinfo

Hi Christoph,

Sorry for taking a while before responding. Holiday season.

On Tue, 19 Jul 2016 01:35:13 +0200 Christoph Anton Mitterer
Post by Christoph Anton Mitterer
I've just checked the whole package again, and nothing in it seems to
so contain anything about tcp or UNIX sockets... so it rather seems to
me that this might be a problem in dbconfig, thus reassigning and
kindly asking its maintainers to have a look.
I agree with you that the issue you experience is more likely in
dbconfig than in icinga.

I have multiple question:

For your initial question: I couldn't quickly find under which user the
program icinga is running, but are Unix sockets appropriate? I.e. I
found a hint it may be running under the nagios user, then Unix sockets
only work if the database user is called the same obviously. (The
maintainers of icinga believe that the default authentication scheme
should be password (thus TCP) as can be seen in the config file of
icinga2-ido-pgsql. One of my packages (cacti) doesn't even know itself
how to run via an Unix socket, so the dbconfig question is misleading
there (but luckily for the package ignored anyways), maybe icinga has a
similar "issue".

If you leave the password for the icinga user blank, dbconfig will
create one for you, as if you are using the password scheme, you need a
password obviously. The debconf question should have told you. Now, when
I try to install icinga2-ido-pgsql (with normal priority), the package
installs fine. So can you please be more verbose on what you did and
answered in the TCP case, such that I can reproduce?

Please also run "debconf-show icinga2-ido-pgsql" and show us the output
(in the situation where you have the issue).

Also, if you can reproduce the issue, running the installation with
export dbc_debug=true
is also very helpful (mind you, you may have to hide your passwords
before you send the response).

Paul
Christoph Anton Mitterer
2016-08-13 20:45:59 UTC
Permalink
Post by Paul Gevers
Sorry for taking a while before responding. Holiday season.
No worries..
Post by Paul Gevers
I couldn't quickly find under which user
the
program icinga is running
Mutliple things come here together:
- the user/groups under which the icinga daemon itself runs
  => per default user: nagios
- the user/groups under which several services, sitting on top of
  icinga, run... for example the webfrontends, which access the DB then
  => here it depends a bit on the frontend (the legacy one from nagios,
     which runs as CGI vs. the "new" ones from icinga itself, which run
     as PHP). The legacy one however, doesn't access the DB.
  => for CGI, it depends again,... is suexec?
  => for PHP, it depends again on the SAPI (mod_php, CGI?, FCGI?)

In the end, every user is basically free to choose, I for example run
all frontends under their own separated user.
So the legacy CGI get's su'ed to e.g. cgi-icinga-classic, the new ones
run via PHP-CGI and get's su'ed to e.g. cgi-icinga-web...
Post by Paul Gevers
but are Unix sockets appropriate?
This is also the most appropriate way. Security-wise it's plain stu***
to run all CGI programs under the same user (e.g. cgi-suexec), and it's
even worse to run PHP programs with mod_php, which place them all under
the webserver's user context (typically and per default www-data user
in Debian).
Post by Paul Gevers
I.e. I
found a hint it may be running under the nagios user, then Unix sockets
only work if the database user is called the same obviously.
Which is the hint?
Anyway, e.g. postgresql (which I use) allows for a user mapping (https:
//www.postgresql.org/docs/current/static/auth-username-maps.html) when
using the socket based "peer" auth method.
That basically means I can tell postgresql, if system user "foo"
connects, treat him as postgresql user "bar".
So there is no need for the user names being the same.

This works quite well, partially even under db-config (for some of the
connections it makes)... its just that it still tries to make TCP
connections for whatever reason.
Post by Paul Gevers
(The
maintainers of icinga believe that the default authentication scheme
should be password (thus TCP) as can be seen in the config file of
icinga2-ido-pgsql.
Well not sure what they actually mean or not.
Icinga1/2 both support connecting via UNIX sockets, they just don't
show examples for this in their installation guide.
Post by Paul Gevers
One of my packages (cacti) doesn't even know itself
how to run via an Unix socket
Sure? Normally most postgresql driver interfaces support both (the only
exception I'd know are the Java drivers),.. usually they just configure
it awkwardly via the host/port config options shared with TCP.
Post by Paul Gevers
If you leave the password for the icinga user blank, dbconfig will
create one for you, as if you are using the password scheme, you need a
password obviously.
Well I've seen that,...

I generally think that's a bad thing, btw,... if the user says don't
set a password, then no password should be set, assuming the user does
security somehow else (e.g. via sockets, which is way more secure than
doing local auth with some password, that can probably be easily brute-
forced).
*If* the user takes care on security already somehow else, the
additional password auth just costs performance on each connection.
On UNIX socket connections, no passwords are examined anyway.
Any on TCP the user may have far more superior stuff in place (some
tunneling via a secure network/VPN/SSH?).
Post by Paul Gevers
Now, when
I try to install icinga2-ido-pgsql (with normal priority), the
package
installs fine. So can you please be more verbose on what you did and
answered in the TCP case, such that I can reproduce?
Well as I've said I did choose UNIX sockets, and I think I've renamed
the DB from icinga2 to just icinga, but that shouldn't matter.
Also my debconf prio was low.

The error that came was as given in the initial bug report, i.e.
despite me choosing unix sockets, it still tried TCP connections (which
of course didn't work, as I've disabled it in postgresql).
Post by Paul Gevers
Please also run "debconf-show icinga2-ido-pgsql" and show us the output
(in the situation where you have the issue).
# debconf-show icinga2-ido-pgsql
  icinga2-ido-pgsql/pgsql/app-pass: (password omitted)
  icinga2-ido-pgsql/password-confirm: (password omitted)
  icinga2-ido-pgsql/app-password-confirm: (password omitted)
  icinga2-ido-pgsql/pgsql/admin-pass: (password omitted)
* icinga2-ido-pgsql/db/app-user: ***@localhost
  icinga2-ido-pgsql/install-error: abort
  icinga2-ido-pgsql/internal/reconfiguring: false
* icinga2-ido-pgsql/pgsql/authmethod-admin: ident
  icinga2-ido-pgsql/missing-db-package-error: abort
* icinga2-ido-pgsql/pgsql/admin-user: postgres
  icinga2-ido-pgsql/remote/newhost:
  icinga2-ido-pgsql/upgrade-error: abort
  icinga2-ido-pgsql/pgsql/no-empty-passwords:
* icinga2-ido-pgsql/dbconfig-reinstall: false
* icinga2-ido-pgsql/db/dbname: icinga
  icinga2-ido-pgsql/remove-error: abort
  icinga2-ido-pgsql/dbconfig-remove: true
  icinga2-ido-pgsql/pgsql/manualconf:
  icinga2-ido-pgsql/pgsql/changeconf: false
  icinga2-ido-pgsql/passwords-do-not-match:
* icinga2-ido-pgsql/pgsql/method: Unix socket
  icinga2-ido-pgsql/remote/port:
  icinga2-ido-pgsql/purge: false
  icinga2-ido-pgsql/dbconfig-upgrade: true
  icinga2-ido-pgsql/database-type: pgsql
  icinga2-ido-pgsql/remote/host: localhost
  icinga2-ido-pgsql/upgrade-backup: true
* icinga2-ido-pgsql/enable: true
* icinga2-ido-pgsql/dbconfig-install: true
* icinga2-ido-pgsql/pgsql/authmethod-user: ident
  icinga2-ido-pgsql/internal/skip-preseed: false

I should note here that "ident" is the same as "peer" for postgresql
when doing UNIX sockets.
Post by Paul Gevers
Also, if you can reproduce the issue
Sure it happens every time.
Post by Paul Gevers
, running the installation with
export dbc_debug=true
is also very helpful (mind you, you may have to hide your passwords
before you send the response).
I can do this but it's a bit work-intensive now that the server
basically runs already in production.
Was the information above already enough for you?
If not I can stop the services, make some temp DB for a test and run it
with dbc_debug=true


Cheers,
Chris.
Paul Gevers
2016-08-13 20:56:39 UTC
Permalink
Hi Christoph,

I am going to bed, now, so a short answer.
Post by Christoph Anton Mitterer
Was the information above already enough for you?
If not I can stop the services, make some temp DB for a test and run it
with dbc_debug=true
I will try to reproduce with the information you provided, but having
the output of dbc_debug=true will be extremely valuable and making my
life in debugging a lot easier, as I then can follow what YOU did
instead of trying to reproduce what steps you did exactly (I for one
don't know yet how to disable the TCP in postgresql etc).

Thanks for your report.

Paul
Christoph Anton Mitterer
2016-08-13 21:22:16 UTC
Permalink
Hey.

See below... went actually much faster than I'd have expected.
populating database via sql...  warning: ident method specified but local account doesn't exist.
warning: ident method specified but local account doesn't exist.
It thinks the account doesn't exist (which is correct, but doesn't
really matter) and I'd blindly assume now it falls back to TCP for this
reason?


One further thing I've just notied, there is no way to configure the
socket dir and "port", i.g.:
/var/run/postgresql/.s.PGSQL.5432
per default.
But this may be different and for postgresql there may be even multiple
daemons running all with their own "ports" e.g. .s.PGSQL.5433 ... this
is all perfectly supported out of the box by the debian packages with
the pg_* tools.

I'll open separate tickets for these in a minute.

Cheers,
Chris.
but having
the output of dbc_debug=true will be extremely valuable and making my
life in debugging a lot easier
# dpkg-reconfigure icinga2-ido-pgsql 
(prerm) dbc_go() icinga2-ido-pgsql upgrade 2.4.10-1~bpo8+1.
dbc_config() icinga2-ido-pgsql upgrade 2.4.10-1~bpo8+1.
dbc_set_dbtype_defaults() .
dbc_read_package_config() .
dbc_set_dbtype_defaults() pgsql.
(config) dbc_go() icinga2-ido-pgsql reconfigure 2.4.10-1~bpo8+1.
dbc_config() icinga2-ido-pgsql reconfigure 2.4.10-1~bpo8+1.
dbc_set_dbtype_defaults() .
dbc_register_debconf() .
dbc_read_package_config() .
dbc_preseed_package_debconf() .
dbc_forget_app_password() .
dbc_get_app_pass() .
(postinst) dbc_go() icinga2-ido-pgsql configure 2.4.10-1~bpo8+1.
dbc_config() icinga2-ido-pgsql configure 2.4.10-1~bpo8+1.
dbc_set_dbtype_defaults() .
dbc_read_package_debconf() .
dbc_set_dbtype_defaults() pgsql.
settings determined from dbc_read_package_debconf:.
dbc_install=true.
dbc_upgrade=true.
dbc_remove=true.
dbc_dbtype=pgsql.
dbc_dbuser=icinga.
dbc_dbpass=t3QJ3BzYp2om.
dbc_dballow=.
dbc_dbadmin=postgres.
dbc_dbadmpass=.
dbc_dbserver=localhost.
dbc_dbport=.
dbc_dbname=icinga.
dbc_authmethod_admin=ident.
dbc_authmethod_user=ident.
dbc_ssl=.
dbc_write_package_config() .
dbconfig-common: writing config to /etc/dbconfig-common/icinga2-ido-pgsql.conf
dbc_read_package_config() .
dbc_detect_installed_dbtype() pgsql.
_dbc_detect_installed_dbtype() pgsql.
su -s /bin/sh postgres -c "env HOME='/tmp/dbconfig-common.psql_home.0wG8JU' PGPASSFILE='/tmp/dbconfig-common.psql_home.0wG8JU/.pgpass' PGSSLMODE='prefer' psql --set \"ON_ERROR_STOP=1\" -q -U 'postgres' template1" 2>&1.
creating postgres user icinga:  su -s /bin/sh postgres -c "env HOME='/tmp/dbconfig-common.psql_home.sCUNV3' PGPASSFILE='/tmp/dbconfig-common.psql_home.sCUNV3/.pgpass' PGSSLMODE='prefer' psql --set \"ON_ERROR_STOP=1\" -q -U 'postgres' template1" 2>&1.
su -s /bin/sh postgres -c "env HOME='/tmp/dbconfig-common.psql_home.7D4bvy' PGPASSFILE='/tmp/dbconfig-common.psql_home.7D4bvy/.pgpass' PGSSLMODE='prefer' psql --set \"ON_ERROR_STOP=1\" -q -U 'postgres' -U 'postgres' template1" 2>&1.
success.
verifying creation of user: su -s /bin/sh postgres -c "env HOME='/tmp/dbconfig-common.psql_home.xFZUCY' PGPASSFILE='/tmp/dbconfig-common.psql_home.xFZUCY/.pgpass' PGSSLMODE='prefer' psql --set \"ON_ERROR_STOP=1\" -q -U 'postgres' template1" 2>&1.
success.
dbconfig-common: dumping pgsql database icinga to /var/tmp/icinga2-ido-pgsql.icinga.2016-08-13-23.09.pgsql.4IRwbV.
su -s /bin/sh postgres -c "env HOME='/tmp/dbconfig-common.psql_home.NU1oJg' PGPASSFILE='/tmp/dbconfig-common.psql_home.NU1oJg/.pgpass' PGSSLMODE='prefer' psql --set \"ON_ERROR_STOP=1\" -q -U 'postgres' template1" 2>&1.
database does not exist.
dbconfig-common: dropping old pgsql database icinga.
su -s /bin/sh postgres -c "env HOME='/tmp/dbconfig-common.psql_home.UsQZjg' PGPASSFILE='/tmp/dbconfig-common.psql_home.UsQZjg/.pgpass' PGSSLMODE='prefer' psql --set \"ON_ERROR_STOP=1\" -q -U 'postgres' template1" 2>&1.
dropping database icinga: database does not exist.
_dbc_detect_installed_dbtype() psql.
su -s /bin/sh postgres -c "env HOME='/tmp/dbconfig-common.psql_home.FOmBye' PGPASSFILE='/tmp/dbconfig-common.psql_home.FOmBye/.pgpass' PGSSLMODE='prefer' psql --set \"ON_ERROR_STOP=1\" -q -U 'postgres' template1" 2>&1.
creating database icinga: su -s /bin/sh postgres -c "env HOME='/tmp/dbconfig-common.psql_home.Ln487E' PGPASSFILE='/tmp/dbconfig-common.psql_home.Ln487E/.pgpass' PGSSLMODE='prefer' psql --set \"ON_ERROR_STOP=1\" -q -U 'postgres' -U 'postgres'" 2>&1.
success.
verifying database icinga exists: success.
populating database via sql...  warning: ident method specified but local account doesn't exist.
warning: ident method specified but local account doesn't exist.
su -s /bin/sh root -c "env HOME='/tmp/dbconfig-common.psql_home.RIlqrF' PGPASSFILE='/tmp/dbconfig-common.psql_home.RIlqrF/.pgpass' PGSSLMODE='prefer' psql --set \"ON_ERROR_STOP=1\" -q -h 'localhost' -U 'icinga' icinga" 2>&1.
error encountered populating database:
psql: could not connect to server: Connection refused Is the server running on host "localhost" (::1) and accepting TCP/IP connections on port 5432? could not connect to server: Connection refused Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5432?
dbconfig-common: icinga2-ido-pgsql configure: aborted.
dbc_postinst_cleanup() .
dbc_forget_dbadmin_password() .
dbconfig-common: flushing administrative password
dbc_forget_app_password() .
Paul Gevers
2016-08-14 09:30:46 UTC
Permalink
Control: clone -1 -2 -3
Control: retitle -1 psql ident without existing sys-user unclear error
Control: tags -1 -moreinfo
Control: retitle -2 psql incorrectly falls back to TCP in ident case
Control: tags -1 -moreinfo
Control: retitle -3 psql alters credentials with dbuser iso dbadmin

Hi Christoph,
Post by Christoph Anton Mitterer
See below... went actually much faster than I'd have expected.
Thanks, that was helpful (for the ident/peer case).

And just to be sure, I assume everything we discuss here is done with
the backported version of dbconfig. There are too many changes since
Jessie to investigate that now (I may check once we identified all issues).
Post by Christoph Anton Mitterer
populating database via sql... warning: ident method specified but local account doesn't exist.
warning: ident method specified but local account doesn't exist.
It thinks the account doesn't exist (which is correct, but doesn't
really matter) and I'd blindly assume now it falls back to TCP for this
reason?
Yes, it does that by design and it does really matter. However, as seen
in this bug, I guess it should just fail instead, right? We fill the
database with the database user credentials to avoid the need for the
administrator credentials (requiring the administrator credentials for
anything that isn't strictly needing it is a bug¹ that I intent to close
soon), so using the administrator credentials to solve this issue is a
no-go. I wasn't aware of the possibility to have system-user to
postgresql-user mapping, so I am afraid that we need an additional
debconf question to ask for the system-user name that will be used for
the connection in case of ident/peer.

After writing the above, I found where the fallback to TCP happens: the
logic of _dbc_psql_cmd_args checks for the password instead of the
method. Should be easy to fix, but alone it will change the fallback
from TCP/$dbc_dbuser to ident/root and still fail (I think).
Post by Christoph Anton Mitterer
populating database via sql... warning: ident method specified but local account doesn't exist.
warning: ident method specified but local account doesn't exist.
su -s /bin/sh root -c "env HOME='/tmp/dbconfig-common.psql_home.RIlqrF' PGPASSFILE='/tmp/dbconfig-common.psql_home.RIlqrF/.pgpass' PGSSLMODE='prefer' psql --set \"ON_ERROR_STOP=1\" -q -h 'localhost' -U 'icinga' icinga" 2>&1.
^^^^ should have read icinga

The user to use is here is determined by the _dbc_psql_local_username
function in /usr/share/dbconfig-common/internal/pgsql.

Could you also create the debugging info for the TCP issue you reported?
I cloned this bug to separate out that issue and leave the moreinfo tag
standing for that issue.

Paul

¹ https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=475829
Christoph Anton Mitterer
2016-08-14 22:02:51 UTC
Permalink
Hey Paul.
Post by Paul Gevers
Control: clone -1 -2 -3
ouch... I didn't want to cause you so much work ;)
Post by Paul Gevers
And just to be sure, I assume everything we discuss here is done with
the backported version of dbconfig. There are too many changes since
Jessie to investigate that now (I may check once we identified all issues).
Yes,...
# dpkg -l | grep dbconfig
ii  dbconfig-common                      2.0.4~bpo8+1                all          framework that helps packages to manage databases
ii  dbconfig-pgsql                       2.0.4~bpo8+1                all          dbconfig-common PostgreSQL support
Post by Paul Gevers
Post by Christoph Anton Mitterer
It thinks the account doesn't exist (which is correct, but doesn't
really matter) and I'd blindly assume now it falls back to TCP for this
reason?
Yes, it does that by design and it does really matter.
Well but as I've laid out before, when one uses pg_ident, than there's
no need that a system user of the same name exists...
Post by Paul Gevers
However, as seen
in this bug, I guess it should just fail instead, right?
Well depends a bit... in the end it should just try with the user name
I specified and that via UNIX sockets.

Or perhaps it could warn the user like "hey, the username you gave
doesn't exist as a system user name, so perhaps this may fail,.. still
wanna continue?"?
If the admin wasn't then careful enough to set up some mapping with
e.g. pg_ident than it's his/her fault.


In general though, I think that any auto-magical fall-back from a user-
specified "use sockets" to TCP (or vice versa) is not something that
should be even tried, regardless of which condition may arise or not.

Consider the example where a user runs two different DB servers, one
only listening on sockets, the other only on TCP,... both using the
same port/socket-number.
If dbconfig then falls back to the other, it may even destroy data,
because the user intended that it would only connect to the other.
Sure, this is a rather stupid setup, but if the user chose "use
sockets" or "use TCP", than IMO only this should be done.
Post by Paul Gevers
We fill the
database with the database user credentials to avoid the need for the
administrator credentials (requiring the administrator credentials for
anything that isn't strictly needing it is a bug¹ that I intent to close
soon), so using the administrator credentials to solve this issue is a
no-go.
Let me see whether I understand:
"Normally" (or rather what I considered normal ;) ), I'd have expected
that, when a tool like dbconfig creates DBs/DB-users respectively
populates/upgrades schemas it would either:
- do everything as DB-admin-user (which for Postgres is (per default)
  the DB-username "postgresl", which happens to be granted (per
  default) for the system user "posgresql")...
  BUT with owners set to the respective DB-(non-admin)-user
OR:
- create only those objects as DB-admin-user, that need to be (e.g.
  other DB-users, the (empty) DBs, and so n)... and everything else
  (including populating the DBs) as the DB-(non-admin)-user

AFAIU, you do the 2nd?
For security reasons I'd also prefer the 2nd, though there is probably
not that much different from a security PoV, at least unless we assume
that any debian package ships SQL code that may contain evil stuff.


Anyway, for both I don't think there should be the need to test whether
a corresponding system-user-name exists; that is at least for that
types of databases where we know they have a mapping technique as
postgresql does.
Post by Paul Gevers
I wasn't aware of the possibility to have system-user to
postgresql-user mapping, so I am afraid that we need an additional
debconf question to ask for the system-user name that will be used
for the connection in case of ident/peer.
I don't understand why this should be needed.
What do you mean by " the system-user name that will be used for the
connection in case of ident/peer"?
Do you mean that you connect *as* this system-user, i.e. doing
something like: # sudo -u mySystemUser psql -f package-schemas.sql
Or do you mean connecting with that DB-admin-user, i.e.
(as root): # psql --username=myDBadminUser -f package-schemas.sql
?

AFAIU, dbconfig should never need to use the actual (UNIX) system users
(perhaps with the exception of the system user "postgres"), but only
always work with the DB-users, and expect that the user has a
pg_hba.conf (or similar for other DBs), that grants dbconfig with the
necessary access permissions.
In the postgresql case we probably would need to use the UNIX system
user "postgresql", as in the default pg_hba.conf "root" has no "DB-
admin" permissions.

But even in that case:
Why would we need another question? AFAIU the following should work:
- In order to create DBs/DB-usernames and stuff for which DB-admin-
  rights are needed:
  # sudo -u postgres psql doTheAdminStuff.sql
- In order to create everything else for:
  ...
  okay here I see the problem... it couldn't just run
  doTheNormalStuff.sql as root... well it could but no package using
  dbconfig expects this right now, so no access permissions would have
  been set up.
  So either you do everything as DB-user/system-user postgres or
  something like:
  # sudo -u normalUser psql doTheNormalStuff.sql
  is needed, which then
requires the normal user (unless there is
  mapping).

hmm... well
a) I think this could be a hint towards a design problem:
   Neither dbconfig, nor packages should expect that system-users
   matching the DB-user names exist (there's no reason for such
   restriction).
   psql should have rather always been run *as* either UNIX user "root"
   or perhaps "postgres"... AND connections should have been made as
   DB-admin-user "postgres" respectively "<normal-user(s)>" and pg_hba
   entries being expected to be in place (or perhaps added
   automatically, which is however dangerous IMO)

Okay, now I also kinda understand why you want to ask for the system user name that should be used for connecting.
In order to use the existing mapping rules to execute the doTheNormalStuff.sql.

Well one kinda could do this as system-user-postgresql as in
# sudo -u postgres psql --username=normalUser doTheAdminStuff.sql
which you want to avoid for security reasons...
Post by Paul Gevers
After writing the above, I found where the fallback to TCP happens: the
logic of _dbc_psql_cmd_args checks for the password instead of the
method. Should be easy to fix, but alone it will change the fallback
from TCP/$dbc_dbuser to ident/root
So, you generally drop the fall back? I think that would be good, as
outlaid above.
Post by Paul Gevers
and still fail (I think).
Unless you connect as "postgresql" or the user would have added some
pg_hba rule allowing system user root to do the stuff, probably yes.
Post by Paul Gevers
Post by Christoph Anton Mitterer
populating database via sql...  warning: ident method specified but
local account doesn't exist.
warning: ident method specified but local account doesn't exist.
su -s /bin/sh root -c "env HOME='/tmp/dbconfig-
common.psql_home.RIlqrF' PGPASSFILE='/tmp/dbconfig-
common.psql_home.RIlqrF/.pgpass' PGSSLMODE='prefer' psql --set
\"ON_ERROR_STOP=1\" -q -h 'localhost' -U 'icinga' icinga" 2>&1.
                ^^^^ should have read icinga
=> well not really,... cause as in my case there is no (system)user
icinga.
That's just the thing you found out above (and which I needed so long
to understand ^^)... either we ask the user for the user to connect as,
or we always use one for which we know it exists and granted access
(which would only be postgresql).

=> also it shouldn't to -h 'localhost", but rather something like:
   -h "/var/run/postgresql" -p 5342...

I'm tempted to say:
- if unix-sockets are used AND the DB-normal-user doesn't exist as
  system user
  => ask for a connecting user for the "su".
     => in that case (cause the system user doesn't exist) we could
        either:
        - provide "postgres" as default, wich should-work™
        - don't provide a default for that question at all, so the user
          would really need to find out.
        - do the later but mention in the documentation, that
          "postgres" should just work, but might have security
          implications
OR
- if not, do as it's done now

but:
- the mapping system of postgresql not only works for UNIX sockets, but
  also TCP (e.g. in the case of GSSAPI)
  => does this affect us, i.e. do we somehow need to take care for the
     TCP case as well??
- and I wonder whether there can be situations in which the system user
  *does* exist, but pg_hab could result into a different DB-user based
  on the configured mappings.
  => probably, as long as you always use -U normalUser to psql, we're
  fine.
Post by Paul Gevers
Could you also create the debugging info for the TCP issue you
reported?
Which TCP issue exactly do you mean now?


Cheers,
Chris.
Paul Gevers
2016-08-15 20:11:13 UTC
Permalink
Hi
Post by Christoph Anton Mitterer
In general though, I think that any auto-magical fall-back from a user-
specified "use sockets" to TCP (or vice versa) is not something that
should be even tried, regardless of which condition may arise or not.
Fully ack. Fall-back to TCP was by accident, not by design.
Post by Christoph Anton Mitterer
Post by Paul Gevers
We fill the
database with the database user credentials to avoid the need for the
administrator credentials (requiring the administrator credentials for
anything that isn't strictly needing it is a bug¹ that I intent to close
soon), so using the administrator credentials to solve this issue is a
no-go.
"Normally" (or rather what I considered normal ;) ), I'd have expected
that, when a tool like dbconfig creates DBs/DB-users respectively
- do everything as DB-admin-user (which for Postgres is (per default)
the DB-username "postgresl", which happens to be granted (per
default) for the system user "posgresql")...
BUT with owners set to the respective DB-(non-admin)-user
- create only those objects as DB-admin-user, that need to be (e.g.
other DB-users, the (empty) DBs, and so n)... and everything else
(including populating the DBs) as the DB-(non-admin)-user
AFAIU, you do the 2nd?
I want to (upload pending). We're doing it already mostly.
Post by Christoph Anton Mitterer
For security reasons I'd also prefer the 2nd, though there is probably
not that much different from a security PoV, at least unless we assume
that any debian package ships SQL code that may contain evil stuff.
It does make a difference. I have had multiple request to enable package
setup to remote db servers where the system administrator of the system
installing the package doesn't know (and doesn't get) the administrator
credentials of the database. Typically in such situations the database
and the database user are already created and known to the system
administrator where the package is installing. That should be sufficient
(and this is a security PoV as the installing system administrator has
no need to now the database administrator credentials.
Post by Christoph Anton Mitterer
Anyway, for both I don't think there should be the need to test whether
a corresponding system-user-name exists; that is at least for that
types of databases where we know they have a mapping technique as
postgresql does.
So which system user should be used than? For sure it needs to be
something and I thing dbconfig doesn't have a choice, only the one used
in the mapping works. I think you agree on this later in the mail.
Post by Christoph Anton Mitterer
Post by Paul Gevers
I wasn't aware of the possibility to have system-user to
postgresql-user mapping, so I am afraid that we need an additional
debconf question to ask for the system-user name that will be used
for the connection in case of ident/peer.
I don't understand why this should be needed.
What do you mean by " the system-user name that will be used for the
connection in case of ident/peer"?
Do you mean that you connect *as* this system-user, i.e. doing
something like: # sudo -u mySystemUser psql -f package-schemas.sql
This. and....
Post by Christoph Anton Mitterer
Or do you mean connecting with that DB-admin-user, i.e.
(as root): # psql --username=myDBadminUser -f package-schemas.sql
this. We do the combination already.
Post by Christoph Anton Mitterer
AFAIU, dbconfig should never need to use the actual (UNIX) system users
(perhaps with the exception of the system user "postgres"), but only
always work with the DB-users, and expect that the user has a
pg_hba.conf (or similar for other DBs), that grants dbconfig with the
necessary access permissions.
I'll read up on how PostgreSQL actually does this, but I would expect
that the system user starting the Unix connection MUST be the right user
for the mapping. Again, I read you write something similar later in your
mail.
Post by Christoph Anton Mitterer
In the postgresql case we probably would need to use the UNIX system
user "postgresql", as in the default pg_hba.conf "root" has no "DB-
admin" permissions.
You can't rely on having the credentials of the postgres user as you may
be working on a remote DB server.
Post by Christoph Anton Mitterer
- In order to create DBs/DB-usernames and stuff for which DB-admin-
# sudo -u postgres psql doTheAdminStuff.sql
True for local DB servers, but not for remote hosts.
Post by Christoph Anton Mitterer
...
okay here I see the problem... it couldn't just run
doTheNormalStuff.sql as root... well it could but no package using
dbconfig expects this right now, so no access permissions would have
been set up.
So either you do everything as DB-user/system-user postgres or
# sudo -u normalUser psql doTheNormalStuff.sql
is needed, which then
requires the normal user (unless there is
mapping).
Right.
Post by Christoph Anton Mitterer
Okay, now I also kinda understand why you want to ask for the system user name that should be used for connecting.
In order to use the existing mapping rules to execute the doTheNormalStuff.sql.
Indeed.
Post by Christoph Anton Mitterer
Post by Paul Gevers
After writing the above, I found where the fallback to TCP happens: the
logic of _dbc_psql_cmd_args checks for the password instead of the
method. Should be easy to fix, but alone it will change the fallback
from TCP/$dbc_dbuser to ident/root
So, you generally drop the fall back? I think that would be good, as
outlaid above.
If you ask I want to remove the fall back, then, yes.
Post by Christoph Anton Mitterer
Post by Paul Gevers
and still fail (I think).
Unless you connect as "postgresql" or the user would have added some
pg_hba rule allowing system user root to do the stuff, probably yes.
Right.
Post by Christoph Anton Mitterer
Post by Paul Gevers
Post by Christoph Anton Mitterer
\"ON_ERROR_STOP=1\" -q -h 'localhost' -U 'icinga' icinga" 2>&1.
^^^^ should have read icinga
=> well not really,... cause as in my case there is no (system)user
icinga.
That's just the thing you found out above (and which I needed so long
to understand ^^)... either we ask the user for the user to connect as,
or we always use one for which we know it exists and granted access
(which would only be postgresql).
Again, not for the remote system.
Post by Christoph Anton Mitterer
-h "/var/run/postgresql" -p 5342...
This is due to the wrong "fall back", it isn't even fall-backing, but
doing it always wrong.
Post by Christoph Anton Mitterer
- if unix-sockets are used AND the DB-normal-user doesn't exist as
system user
=> ask for a connecting user for the "su".
=> in that case (cause the system user doesn't exist) we could
- provide "postgres" as default, wich should-work™
- don't provide a default for that question at all, so the user
would really need to find out.
- do the later but mention in the documentation, that
"postgres" should just work, but might have security
implications
OR
- if not, do as it's done now
We can choose (we do mention something like that in the FIRST question
asked by dbconfig on behave of the package) to not implement this, and
leave the system administrator to manual set up the configuration for
the package in this case. On the other hand, it doesn't look really hard
to support this. Albeit a new question in the case of Sockets.
Interestingly you are the first to report on this for soooooo long...
Post by Christoph Anton Mitterer
- the mapping system of postgresql not only works for UNIX sockets, but
also TCP (e.g. in the case of GSSAPI)
huh? Can you elaborate a bit?
Post by Christoph Anton Mitterer
=> does this affect us, i.e. do we somehow need to take care for the
TCP case as well??
- and I wonder whether there can be situations in which the system user
*does* exist, but pg_hab could result into a different DB-user based
on the configured mappings.
=> probably, as long as you always use -U normalUser to psql, we're
fine.
Good to know.
Post by Christoph Anton Mitterer
Post by Paul Gevers
Could you also create the debugging info for the TCP issue you reported?
Which TCP issue exactly do you mean now?
Your second e-mail to this bug contained some information where you
tried to enable TCP login and it still failed for you because the
credentials were tried to be created with the $dbuser login. As TCP is
the default answer, I am even more curious to know. Did this by any
chance happen during error handling? I may have found in the past few
months that there are situations where the $_dbc_asuser variable isn't
properly restored.

Paul
Christoph Anton Mitterer
2016-08-15 21:05:44 UTC
Permalink
Post by Paul Gevers
Post by Christoph Anton Mitterer
- create only those objects as DB-admin-user, that need to be (e.g.
  other DB-users, the (empty) DBs, and so n)... and everything else
  (including populating the DBs) as the DB-(non-admin)-user
AFAIU, you do the 2nd?
I want to (upload pending). We're doing it already mostly.
:-)
Post by Paul Gevers
Post by Christoph Anton Mitterer
For security reasons I'd also prefer the 2nd, though there is
probably
not that much different from a security PoV, at least unless we assume
that any debian package ships SQL code that may contain evil stuff.
It does make a difference. I have had multiple request to enable package
setup to remote db servers where the system administrator of the system
installing the package doesn't know (and doesn't get) the
administrator
credentials of the database. Typically in such situations the
database
and the database user are already created and known to the system
administrator where the package is installing. That should be
sufficient
(and this is a security PoV as the installing system administrator has
no need to now the database administrator credentials.
But in these specific cases (i.e. installation to a remote system), the
UNIX user under which we run psql/etc. locally shouldn't matter at al
(that is unless the remote DB server would e.g. use identd based auth
(with the local users), or similar things like GSSAPI).
Apart from that, the remote system couldn't know which local UNIX user
we use and should do auth based on password, IP, etc.

Anyway, we both agree the 2nd way is the better one =)
Post by Paul Gevers
Post by Christoph Anton Mitterer
Anyway, for both I don't think there should be the need to test whether
a corresponding system-user-name exists; that is at least for that
types of databases where we know they have a mapping technique as
postgresql does.
So which system user should be used than? For sure it needs to be
something and I thing dbconfig doesn't have a choice, only the one used
in the mapping works. I think you agree on this later in the mail.
hmm... IMO the best thing would have been either:
(1) There is a dedicated dbconfig user, this is the user under which
    any SQL, DB-clients, etc. are run, regardless of whether TCP or
    UNIX sockets, regardless of whether locally or remotely
(2) This user is expected to be allowed to do all necessary things,
    like: creating users, databases, languages, populating schemas,
    upgrading schemas, etc. .... either by being that user (then using
    things like peer-auth of pgsql) or by plain password, in which case
    the dbconfig user is in principle moot, but it's better to use such
    user instead of root or postgresql to run these things, for the
    sake of privilege separation.
(3) For certain operations (user creation, DB creation and similar),
    i.e. all the things a remote DB admin wouldn't typically grant
    normal users but supply them with,... dbconfig should:
    first check if that already exists (perhaps try to use it and see
    if that works - if possible) and if it does, not creating it again
    but simply continuing with the DB-non-admin-user stuff (i.e.
    creating tables, upgrading schemas, etc.)
    Perhaps a warning should be given, if the user/db already exists,
    asking whether one wants to (try to) continue (after all, the admin
    could have accidentally given you access to something).
(4) For those auth methods, that aren't based on which UNIX user we run
    under, things would of course require some credentials (e.g.
    password) just as it's the case already now.
    And in general (i.e. also for the auth methods that are based on
    the UNIX user under which dbconfig runs its command:
    Appropriate pg_hba/pg_ident (and for other DBs similar) rules are
    expected to be in place.
    So while right now, in postgres, we depend e.g. on a line like
    this:
    local   all             postgres                               peer
    for creating users/DBs we would need one like this:
    local   all             dbconfig                               peer

The above schema could be modified to:
- use UNIX user AND DB-user "dbconfig" for everything (creation &
  population), expecting that it's allowed to do so by DB config.
- use UNIX user AND DB-user "postgres" for creation
  but UNIX user "dbconfig" WITH the respective DB-user (like icinga)
  for population, expecting appropriate mappings in place
- use UNIX user "dbconfig" WITH DB-user "postgres" for everything
 
(creation & population), expecting that dbconfig maps to postgres
 
=>bad, not even sure if this works...


Pros:
- we don't run any commands (locally) under UNIX users "postgres" or
  "root", which are not "our" (i.e. dbconfig's) domain and thus it's
  good for privilege separation.
  we don't run any commands under arbitrary other users like "icinga"
  or "phpbb" either, which are again not "our" users and we shouldn't
  su to them unless really necessary.
- shouldn't make a difference whether remote DB or local, unless the
  remote user name is somehow transported to the remote side (GSSAPI,
  identd etc).

Cons:
- appropriate auth rules need to be in place on the DB server side,
  that allow the dbconfig user to do its business
  => could be provided by the packages in theory...
- if another user (like postgres or dbconfig) is used to populate the
  databses with tables/etc., then the schemas must be carefully written
  to make everything properly owned not by postgres/dbconfig, but the
  actually intended DB user

Perhaps even better then the above, would be hybrid between the above
and the current way:
1) Allow the user to choose what is used for DB-admin-stuff:
   a) which UNIX user to use
   b) which DB user to use
   c) whether to expect that things (DB-users, databases) are already
      created, and that this shall be checked&verified and used.
2) Allow the user to choose what is used for DB-non-admin-stuff:
   a) which UNIX user to use
   b) which DB user to use

And we could even provide sensible defaults depending on which users we find, or whether it was detected that a remote DB server was given (of course this is just heuristics).

So in a remote server case, the user could choose:
1c
2a=dbconfig (in case the local UNIX user doesn't matter, because we
   don't do SSH socket forwarding or identd or similar).
3a=icinga

In the local server case, one could have:
1a=postgres
1b=postgres
2a=icinga
2b=icinga


Well... not sure if all the above really works out... it's basically
just some start of brainstorming ;-)
Perhaps another user category would be needed, e.g. "the DB user which
shall own created objects", if that differs from 2b (if this is even
possible or makes sense).
Post by Paul Gevers
I'll read up on how PostgreSQL actually does this, but I would expect
that the system user starting the Unix connection MUST be the right user
for the mapping.
Yes this is the case.
Post by Paul Gevers
Post by Christoph Anton Mitterer
In the postgresql case we probably would need to use the UNIX system
user "postgresql", as in the default pg_hba.conf "root" has no "DB-
admin" permissions.
You can't rely on having the credentials of the postgres user as you may
be working on a remote DB server.
Sure... but that's a general problem isn't it? For those cases where we
have no DB-admin rights, the remote server would need to be required to
provide everything for us already (i.e. users, databases, and similar
things)... and we just populate/upgrade the databases.
Under which UNIX user we run this locally shouldn't matter much for the
remote server (unless an auth method like postgresl's TCP-ident-with-
ident*d* is used, or the local user has some other special rights to
connect to remote server, which other users wouldn't have (perhaps a
tunneled socket via SSH).
Post by Paul Gevers
Post by Christoph Anton Mitterer
- In order to create DBs/DB-usernames and stuff for which DB-admin-
  # sudo -u postgres psql doTheAdminStuff.sql
True for local DB servers, but not for remote hosts.
Same as above... if the remote (or local) server doesn't give you DB-
amin-rights... we never can do anything about it... just check whether
the databases/DB-users we want to use already exist and move on if they
do.

I don't see what difference it makes under which local UNIX user we
run.
Post by Paul Gevers
If you ask I want to remove the fall back, then, yes.
+1
Post by Paul Gevers
Post by Christoph Anton Mitterer
Post by Paul Gevers
Post by Christoph Anton Mitterer
\"ON_ERROR_STOP=1\" -q -h 'localhost' -U 'icinga' icinga" 2>&1.
                ^^^^ should have read icinga
=> well not really,... cause as in my case there is no (system)user
icinga.
That's just the thing you found out above (and which I needed so long
to understand ^^)... either we ask the user for the user to connect as,
or we always use one for which we know it exists and granted access
(which would only be postgresql).
Again, not for the remote system.
But in case of the remote system, we never get the remote-SYSTEM-user
context ever... we only connect via some networking, and do some
authentication, right?
Post by Paul Gevers
Post by Christoph Anton Mitterer
- if unix-sockets are used AND the DB-normal-user doesn't exist as
  system user
  => ask for a connecting user for the "su".
     => in that case (cause the system user doesn't exist) we could
        - provide "postgres" as default, wich should-work™
        - don't provide a default for that question at all, so the
user
          would really need to find out.
        - do the later but mention in the documentation, that
          "postgres" should just work, but might have security
          implications
OR
- if not, do as it's done now
We can choose (we do mention something like that in the FIRST
question
asked by dbconfig on behave of the package) to not implement this, and
leave the system administrator to manual set up the configuration for
the package in this case. On the other hand, it doesn't look really hard
to support this. Albeit a new question in the case of Sockets.
Interestingly you are the first to report on this for soooooo long...
;-)
Post by Paul Gevers
Post by Christoph Anton Mitterer
- the mapping system of postgresql not only works for UNIX sockets, but
  also TCP (e.g. in the case of GSSAPI)
huh? Can you elaborate a bit?
I don't really know that much about this either,.. I just read it in
the postgres docs and GSSAPI and identd are mentioned.
So I'd blindly assume that these two are used to map the local user
into the remote user namespace.
Post by Paul Gevers
Your second e-mail to this bug contained some information where you
tried to enable TCP login and it still failed for you because the
credentials were tried to be created with the $dbuser login. As TCP is
the default answer, I am even more curious to know. Did this by any
chance happen during error handling? I may have found in the past few
months that there are situations where the $_dbc_asuser variable isn't
properly restored.
Ah I see... I'll try to reproduce that tomorrow... maybe I've just
messed up some things :D


Cheers,
Chris.
Paul Gevers
2016-08-16 19:20:31 UTC
Permalink
Hi
Post by Christoph Anton Mitterer
But in these specific cases (i.e. installation to a remote system), the
UNIX user under which we run psql/etc. locally shouldn't matter at al
(that is unless the remote DB server would e.g. use identd based auth
(with the local users), or similar things like GSSAPI).
The intent is that dbconfig indeed support identd based authentication.
But indeed, normally we run under root (which IS the common domain
during installation/configuration of packages in the Debian.)
Post by Christoph Anton Mitterer
(1) There is a dedicated dbconfig user, this is the user under which
any SQL, DB-clients, etc. are run, regardless of whether TCP or
UNIX sockets, regardless of whether locally or remotely
This is putting even higher requirements on the (remote) database. I
really don't like this approach as it also makes stuff more complicated
(to check for).
Post by Christoph Anton Mitterer
(3) For certain operations (user creation, DB creation and similar),
i.e. all the things a remote DB admin wouldn't typically grant
first check if that already exists (perhaps try to use it and see
if that works - if possible) and if it does, not creating it again
but simply continuing with the DB-non-admin-user stuff
That is exactly what my pending upload is going to do.
Post by Christoph Anton Mitterer
Perhaps a warning should be given, if the user/db already exists,
asking whether one wants to (try to) continue (after all, the admin
could have accidentally given you access to something).
What do you have in mind here exactly? I guess this is more of an issue
for UNIX sockets than for password access, because if the admin tells me
the right password, it must be expecting that I am going to do something.
Post by Christoph Anton Mitterer
- we don't run any commands (locally) under UNIX users "postgres" or
"root", which are not "our" (i.e. dbconfig's) domain and thus it's
good for privilege separation.
We already run under root and that is quite normal. On localhost MariaDB
I even need to be root to have the UNIX socket (like "postgres" for
PostgreSQL).
Post by Christoph Anton Mitterer
we don't run any commands under arbitrary other users like "icinga"
or "phpbb" either, which are again not "our" users and we shouldn't
su to them unless really necessary.
Well, here we may disagree. I think the package icinga is using dbconfig
to do the DB setup, but it is the package icinga that is running the
install.
Post by Christoph Anton Mitterer
- shouldn't make a difference whether remote DB or local, unless the
remote user name is somehow transported to the remote side (GSSAPI,
identd etc).
Which is in principle supported.
Post by Christoph Anton Mitterer
- appropriate auth rules need to be in place on the DB server side,
that allow the dbconfig user to do its business
=> could be provided by the packages in theory...
Again, not for remote hosts. Biggest argument against this (more
complex) scheme.
Post by Christoph Anton Mitterer
And we could even provide sensible defaults depending on which users
we find, or whether it was detected that a remote DB server was given
(of course this is just heuristics).
I believe the heuristics already greatly improved between jessie and
jessie-backports/strech, so maybe some of your ideas are already in
place in a way.
Post by Christoph Anton Mitterer
Post by Paul Gevers
True for local DB servers, but not for remote hosts.
Same as above... if the remote (or local) server doesn't give you DB-
amin-rights... we never can do anything about it... just check whether
the databases/DB-users we want to use already exist and move on if they
do.
Ok, so here is something were some of my answered may be biased by: the
(current) design and implementation of dbconfig. I think some of your
ideas may be good/valid, but need extensive rewrite (for not enough?
gain). Remember (or note, not sure how aware you are) that the core of
dbconfig doesn't really care what DB you are talking to, MySQL/MariaDB,
PostgreSQL, SQLite/SQLite3 and in the near future maybe MongoDB. So the
logic in the main part should not be geared TOO much against PostgreSQL.

Paul
Paul Gevers
2016-08-19 07:50:21 UTC
Permalink
Hi Christoph,

Would you agree with me, i.e. do you know the following to be true, that
peer authentication requires Unix socket (localhost) and that Unix
socket requires peer identification for PostgreSQL?

I tried the other day to have password authentication via the Unix
socket, but that failed:
***@sid:/# psql -U icinga234 -W
Password for user icinga234:
psql: FATAL: Peer authentication failed for user "icinga234"

If this is true, I think it warrants some improved logic during config.

By the way, I see that PostgreSQL has a lot more authentication
possibilities than when Sean invented dbconfig. I don't think I am going
to support this on the short/mid term, but it may warrant improved
messages here and there.

Paul
PS: I believe this works different for MySQL/MariaDB, so the answer is
not trivial.
Christoph Anton Mitterer
2016-08-20 01:27:39 UTC
Permalink
Hey.
Post by Paul Gevers
Would you agree with me, i.e. do you know the following to be true, that
peer authentication requires Unix socket (localhost)
This, I think, is the case:
https://www.postgresql.org/docs/current/static/auth-pg-hba-conf.html
peer says: "This is only available for local connections."
Post by Paul Gevers
and that Unix
socket requires peer identification for PostgreSQL?
This is, I think, definitely not the case... at least "trust" and
"reject" would work there as well, notice also that the socket is
created as e.g.:
srwxrwxrwx  1 postgres postgres    0 Aug 20 02:49 .s.PGSQL.5432
i.e. everyone can write/read.

I haven't checked whether the other postgres-protocol-level auth
methods work with sockets, but I would imagine that things like "md5",
"password", probably "pam"... etc. would.
Post by Paul Gevers
I tried the other day to have password authentication via the Unix
psql: FATAL:  Peer authentication failed for user "icinga234"
hmm what was your pg_hba.conf looking like?
You should notice that postgres will IIRC always take the first
possible matching rule in pg_hba.conf, so if you have:
local   all             all                                     peer
local   all             all                                     md5
it
would always try peer.
Post by Paul Gevers
By the way, I see that PostgreSQL has a lot more authentication
possibilities than when Sean invented dbconfig. I don't think I am going
to support this on the short/mid term, but it may warrant improved
messages here and there.
Well the problem is that supporting all these is probably quite
difficult...OTOH, most of the others would be (IMO) much more
beneficial than md5/password ;-)


Cheers,
Chris.
Christoph Berg
2016-08-23 08:55:57 UTC
Permalink
Hi,

I was on vacation last week and couldn't answer earlier here.
Post by Paul Gevers
Would you agree with me, i.e. do you know the following to be true, that
peer authentication requires Unix socket (localhost)
This is true.
Post by Paul Gevers
and that Unix socket requires peer identification for PostgreSQL?
This is true in the default config (pg_createcluster), but can be set
to various other authentication mechanisms, the most widely used next
to "peer" being "md5", but it could also be "pam", "ldap", and others.
Post by Paul Gevers
I tried the other day to have password authentication via the Unix
psql: FATAL: Peer authentication failed for user "icinga234"
You can't force the use of password authentication, the server will
always use the authentication method configured in pg_hba.conf, based
on the client connection parameters (unix vs tcp/ip, user name,
database name, IP address). The psql -W parameter is mostly useless in
practise.
Post by Paul Gevers
If this is true, I think it warrants some improved logic during config.
By the way, I see that PostgreSQL has a lot more authentication
possibilities than when Sean invented dbconfig. I don't think I am going
to support this on the short/mid term, but it may warrant improved
messages here and there.
I don't think you should be trying to implement too much logic on the
dbconfig-common side here. I'd suggest this:

simple mode: assume a default config cluster, i.e. OS user "postgres",
database user "postgres", unix socket in /var/run/postgresql/.

advanced mode: Ask the user for
* database hostname, or blank for default "/var/run/postgresql" or a
full path name for different UNIX socket
* database (super)user (default postgres)
* database password (default blank which suits peer authentication)
* whatever other parameters dbconfig-common wants (DB to create, DB
user to create if not existing)

You can prompt the user which way they want; simple mode should be the
default.

If you unconditionally run the database commands as OS user
"postgres", you don't even need to prompt the user for that variable.
(The case of non-postgres clusters using peer UNIX socket
authentication would be left unsupported, but that's rare, and the
user can easily provide a password that works via TCP/IP on
localhost.)

Note that in the above scheme you don't need to know which
authentication method the server is using, or if the connection is
TCP/IP or a UNIX socket, everything should Just Work.

Christoph
Paul Gevers
2016-08-23 19:28:58 UTC
Permalink
Hi Christoph(s),
Post by Christoph Berg
I was on vacation last week and couldn't answer earlier here.
No problem (of course, I hope you enjoyed it).
Post by Christoph Berg
Post by Paul Gevers
By the way, I see that PostgreSQL has a lot more authentication
possibilities than when Sean invented dbconfig. I don't think I am going
to support this on the short/mid term, but it may warrant improved
messages here and there.
I don't think you should be trying to implement too much logic on the
I agree.
Post by Christoph Berg
simple mode: assume a default config cluster, i.e. OS user "postgres",
database user "postgres", unix socket in /var/run/postgresql/.
Sure, this works for setting up things and is what is being done, but
most programs will not be the postgres user when connecting (and mapping
to postgres also sounds like a bad idea). So the simple mode has to
still set things up correctly for the final (system) user. Currently it
uses TCP/IP and password authentication as default because (quoting the
template): "typically the system username doesn't match the database
username". It seems this default works OK(ish). I haven't seen complains
on this front yet.
Post by Christoph Berg
advanced mode: Ask the user for
* database hostname, or blank for default "/var/run/postgresql" or a
full path name for different UNIX socket
* database (super)user (default postgres)
* database password (default blank which suits peer authentication)
* whatever other parameters dbconfig-common wants (DB to create, DB
user to create if not existing)
You can prompt the user which way they want; simple mode should be the
default.
The current implementation of dbconfig is pretty close to your proposal.
As mentioned, it needs to provide the right database user and it
defaults to password authentication as that is easier, i.e. it doesn't
require the installing package to tell me what the system user will
(probably) be. I think I need to update the template for the hostname to
include some hint that an UNIX socket can also be entered (for PostgreSQL).
Post by Christoph Berg
If you unconditionally run the database commands as OS user
"postgres", you don't even need to prompt the user for that variable.
On local systems, postgres it detected and used (by default). However, I
need the prompt because the database may be on a different host. Or do
you mean to say that even for remote connections, I could just make the
connection as postgres user. How about (yes unsafe) ident authentication?
Post by Christoph Berg
Note that in the above scheme you don't need to know which
authentication method the server is using, or if the connection is
TCP/IP or a UNIX socket, everything should Just Work.
Yes, this was the reason of my e-mail, it seems like there are redundant
questions in the flow. Now I need to figure out if this is generic, or
only for the PostgreSQL case.

Thanks for you answers.

Paul
Christoph Berg
2016-08-24 08:15:01 UTC
Permalink
Post by Paul Gevers
Hi Christoph(s),
:)
Post by Paul Gevers
Post by Christoph Berg
simple mode: assume a default config cluster, i.e. OS user "postgres",
database user "postgres", unix socket in /var/run/postgresql/.
Sure, this works for setting up things and is what is being done, but
most programs will not be the postgres user when connecting (and mapping
to postgres also sounds like a bad idea). So the simple mode has to
still set things up correctly for the final (system) user. Currently it
uses TCP/IP and password authentication as default because (quoting the
template): "typically the system username doesn't match the database
username". It seems this default works OK(ish). I haven't seen complains
on this front yet.
Oh right, I was only thinking about the initial connection for
creating that user. (Though I guess if you create that user yourself,
you don't need to prompt for anything in the "simple and fast" mode.)
Post by Paul Gevers
Post by Christoph Berg
If you unconditionally run the database commands as OS user
"postgres", you don't even need to prompt the user for that variable.
On local systems, postgres it detected and used (by default). However, I
need the prompt because the database may be on a different host. Or do
you mean to say that even for remote connections, I could just make the
connection as postgres user. How about (yes unsafe) ident authentication?
IMHO forget about "ident" please, I haven't seen anyone use that since
the 90ies (and that was for IRC back then). The local OS user
shouldn't matter for remote connections.

Christoph

Christoph Anton Mitterer
2016-08-20 01:38:55 UTC
Permalink
Post by Paul Gevers
The intent is that dbconfig indeed support identd based
authentication.
I see.. hmm well wouldn't be my first choice for security reasons,...
but I think it should be possible to get "full" support even with
supporting identd as well.
Post by Paul Gevers
But indeed, normally we run under root (which IS the common domain
during installation/configuration of packages in the Debian.)
Well as I said, it might be, that the UNIX user that dbconfig runs as
already affects the actual authentication... so perhaps it's worth to
generally allow people to specify which UNIX-user to run under in
addition.
But once should probably also put some warnings in place, that this
really requires people to know what they're doing.
Post by Paul Gevers
Post by Christoph Anton Mitterer
(3) For certain operations (user creation, DB creation and
similar),
    i.e. all the things a remote DB admin wouldn't typically grant
    first check if that already exists (perhaps try to use it and
see
    if that works - if possible) and if it does, not creating it
again
    but simply continuing with the DB-non-admin-user stuff
That is exactly what my pending upload is going to do.
:)
Post by Paul Gevers
Post by Christoph Anton Mitterer
    Perhaps a warning should be given, if the user/db already
exists,
    asking whether one wants to (try to) continue (after all, the
admin
    could have accidentally given you access to something).
What do you have in mind here exactly? I guess this is more of an issue
for UNIX sockets than for password access, because if the admin tells me
the right password, it must be expecting that I am going to do
something.
Well... it's based on... let's call it personal experience...
I work for a university and we run a Tier-2 for the LHC Computing Grid
using a massive storage management software called dCache, which every
now and then changes its DB schemas (uses Postgres as well ;) )...
Their policy is usually also not to complain loudly/fail when things
are already there (e.g. when a site already created an index or so),
which over time resulted in many sites having different schemas even
though running the same version.
IMO this creates rather more problems than sites solve by having some
freedom of flexibility.

Therefore, I'd say it's better to warn one time too often than not.

And keep in mind that just by using TCP doesn't mean that the DB is
actually remote,... this is e.g. the case in dCache which is written in
Java (which doesn't support UNIX sockets for DB connections AFAIK).
Post by Paul Gevers
Post by Christoph Anton Mitterer
  we don't run any commands under arbitrary other users like "icinga"
  or "phpbb" either, which are again not "our" users and we shouldn't
  su to them unless really necessary.
Well, here we may disagree. I think the package icinga is using dbconfig
to do the DB setup, but it is the package icinga that is running the
install.
Hmm this is in fact a valid point...
Post by Paul Gevers
I believe the heuristics already greatly improved between jessie and
jessie-backports/strech, so maybe some of your ideas are already in
place in a way.
Sure :)
Post by Paul Gevers
Post by Christoph Anton Mitterer
Same as above... if the remote (or local) server doesn't give you DB-
amin-rights... we never can do anything about it... just check whether
the databases/DB-users we want to use already exist and move on if they
do.
Ok, so here is something were some of my answered may be biased by: the
(current) design and implementation of dbconfig. I think some of your
ideas may be good/valid, but need extensive rewrite (for not enough?
gain).
Well as I've said at some point... most of this should be firstly
considered brainstorming... it doesn't mean per se that it needs or
should be implemented :)
Post by Paul Gevers
So
the
logic in the main part should not be geared TOO much against
PostgreSQL.
Sure... :)


Cheers,
Chris.
Christoph Anton Mitterer
2016-08-16 15:49:42 UTC
Permalink
Post by Paul Gevers
Your second e-mail to this bug contained some information where you
tried to enable TCP login and it still failed for you because the
credentials were tried to be created with the $dbuser login. As TCP is
the default answer, I am even more curious to know. Did this by any
chance happen during error handling? I may have found in the past few
months that there are situations where the $_dbc_asuser variable isn't
properly restored.
I tried it again with the current backports version, but couldn't
reproduce it... so either that was with the native jessie version... or
I simply had messed up something (or just didn't find out again, how to
reproduce it ;) ).

I'd say let's call that a non-issue, until it would pop up again.


Cheers,
Chris.
Loading...