Discussion:
[Puppet Users] Puppet Certificate Issues
Rohit
2018-10-18 16:52:30 UTC
Permalink
Hello, we currently have a puppet docker container setup and are
experiencing certificate issues. Basically, in our docker setup (on our
main server) I had generated and signed new certificates, but the puppet_db
container keeps restarting. Here are logs from the puppet_db container:

‘Error: Could not retrieve catalog from remote server: SSL_connect
returned=1 errno=0 state=error: certificate verify failed: [unable to get
local issuer certificate for /CN=our.puppet.domain]
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0
state=error: certificate verify failed: [unable to get local issuer
certificate for /CN=our.puppet.domain]’

I have tried series of steps to solve this problem as it looks like Puppet
is not functioning correctly as our servers are not properly listening to
the host server. Any idea what I can do to solve this problem? For
reference, we are running Puppet_DB version 4.2 and Puppet Server version
2.7.2, all of which is set up on a docker container environment on one
server.
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/66479e42-5d70-41b0-a0d9-0774e273fdab%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Morgan Rhodes
2018-10-18 21:14:54 UTC
Permalink
Hi Rohit,

Is the hostname from `/CN=our.puppet.domain` showing up in your
puppetserver's certificate? You can verify that with `puppet cert list
--all` on the puppetserver container. This looks like a DNS issue.
Post by Rohit
Hello, we currently have a puppet docker container setup and are
experiencing certificate issues. Basically, in our docker setup (on our
main server) I had generated and signed new certificates, but the puppet_db
‘Error: Could not retrieve catalog from remote server: SSL_connect
returned=1 errno=0 state=error: certificate verify failed: [unable to get
local issuer certificate for /CN=our.puppet.domain]
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0
state=error: certificate verify failed: [unable to get local issuer
certificate for /CN=our.puppet.domain]’
I have tried series of steps to solve this problem as it looks like Puppet
is not functioning correctly as our servers are not properly listening to
the host server. Any idea what I can do to solve this problem? For
reference, we are running Puppet_DB version 4.2 and Puppet Server version
2.7.2, all of which is set up on a docker container environment on one
server.
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/1a315e30-fd7f-4da8-ba52-52da56756311%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Rohit
2018-10-19 16:02:47 UTC
Permalink
Hello Morgan,

If you are refferring to the cert being in the conf/ssl/certs folder, then
yes, our.puppet.domain.pem is in the folder. When running the 'puppet cert
list --all' I see three certificates (in the SHA256 format):

- computername.our.puppet.domain
- our.puppet.domain
- servername.our.puppet.domain

If it is a DNS issue, do I have to likely change something from the
docker-compose side?
Post by Morgan Rhodes
Hi Rohit,
Is the hostname from `/CN=our.puppet.domain` showing up in your
puppetserver's certificate? You can verify that with `puppet cert list
--all` on the puppetserver container. This looks like a DNS issue.
Post by Rohit
Hello, we currently have a puppet docker container setup and are
experiencing certificate issues. Basically, in our docker setup (on our
main server) I had generated and signed new certificates, but the puppet_db
‘Error: Could not retrieve catalog from remote server: SSL_connect
returned=1 errno=0 state=error: certificate verify failed: [unable to get
local issuer certificate for /CN=our.puppet.domain]
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0
state=error: certificate verify failed: [unable to get local issuer
certificate for /CN=our.puppet.domain]’
I have tried series of steps to solve this problem as it looks like
Puppet is not functioning correctly as our servers are not properly
listening to the host server. Any idea what I can do to solve this problem?
For reference, we are running Puppet_DB version 4.2 and Puppet Server
version 2.7.2, all of which is set up on a docker container environment on
one server.
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Morgan Rhodes
2018-10-19 17:09:33 UTC
Permalink
A few things to verify:

1) what hostname is your puppetdb container trying to connect to
puppetserver at?
a) This should be in your docker-entrypoint.sh script in the puppetdb
container. Likely either 'puppet' or '$PUPPETSERVER_HOSTNAME' depending on
what variables you have set in your compose file and what version of the
puppetdb container you have.

2) Is the hostname your puppetdb container is trying to connect to listed
as one of the certificate names for your puppet server's cert?
a) For example, in my puppetserver container when I run `puppet cert
list --all` I see:

+ "testserver" (SHA256)
F0:31:6D:1D:03:82:C0:84:0D:FA:2B:28:5B:52:CB:18:88:87:61:5F:5A:F5:7E:AB:A2:73:29:44:BC:57:D0:99
(alt names: "DNS:testserver", "DNS:foo")

if my puppetdb container tries to connect to that host over any names
other than 'testserver' or 'foo' I get a certificate verify failed error.
Post by Rohit
Hello Morgan,
If you are refferring to the cert being in the conf/ssl/certs folder, then
yes, our.puppet.domain.pem is in the folder. When running the 'puppet cert
- computername.our.puppet.domain
- our.puppet.domain
- servername.our.puppet.domain
If it is a DNS issue, do I have to likely change something from the
docker-compose side?
Post by Morgan Rhodes
Hi Rohit,
Is the hostname from `/CN=our.puppet.domain` showing up in your
puppetserver's certificate? You can verify that with `puppet cert list
--all` on the puppetserver container. This looks like a DNS issue.
Post by Rohit
Hello, we currently have a puppet docker container setup and are
experiencing certificate issues. Basically, in our docker setup (on our
main server) I had generated and signed new certificates, but the puppet_db
‘Error: Could not retrieve catalog from remote server: SSL_connect
returned=1 errno=0 state=error: certificate verify failed: [unable to get
local issuer certificate for /CN=our.puppet.domain]
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0
state=error: certificate verify failed: [unable to get local issuer
certificate for /CN=our.puppet.domain]’
I have tried series of steps to solve this problem as it looks like
Puppet is not functioning correctly as our servers are not properly
listening to the host server. Any idea what I can do to solve this problem?
For reference, we are running Puppet_DB version 4.2 and Puppet Server
version 2.7.2, all of which is set up on a docker container environment on
one server.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
***@puppet.com
Release Engineer
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CA%2BFnDv15nGSpEuX7otQa%2B%3Dc9_FKW7YLB_FQzWWhJgRSvEHP%2BxA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Rohit
2018-10-19 18:35:04 UTC
Permalink
1. puppet_db is trying to connect our.puppet.domain, there is no
docker-entrypoint.sh script that I was able to find.
1. For reference, this is the docker-compose.yml:
2.

puppetdb:

container_name: puppet_db

hostname: puppetdb.peninsula.wednet.edu

dns:

- 10.0.0.7

image: puppet/puppetdb:latest

ports:

- 8087:8080

- 8088:8081

depends_on:

- puppet

links:

- puppet:puppet4.psd401.net

- puppetdbpostgres:postgres

volumes:

- ./puppet-client.conf:/etc/puppetlabs/puppet/puppet.conf

- ./puppetdb_conf:/etc/puppetlabs/puppetdb/conf.d

- ./puppetdb_ssl:/etc/puppetlabs/puppet/ssl/

networks:

puppet:

ipv4_address: 172.19.0.4

restart: always


2. The hostname that the puppetdb container is trying to connect to
is indeed the one listed on the certificate name on the puppet servers cert.
Post by Morgan Rhodes
1) what hostname is your puppetdb container trying to connect to
puppetserver at?
a) This should be in your docker-entrypoint.sh script in the puppetdb
container. Likely either 'puppet' or '$PUPPETSERVER_HOSTNAME' depending on
what variables you have set in your compose file and what version of the
puppetdb container you have.
2) Is the hostname your puppetdb container is trying to connect to listed
as one of the certificate names for your puppet server's cert?
a) For example, in my puppetserver container when I run `puppet cert
+ "testserver" (SHA256)
F0:31:6D:1D:03:82:C0:84:0D:FA:2B:28:5B:52:CB:18:88:87:61:5F:5A:F5:7E:AB:A2:73:29:44:BC:57:D0:99
(alt names: "DNS:testserver", "DNS:foo")
if my puppetdb container tries to connect to that host over any names
other than 'testserver' or 'foo' I get a certificate verify failed error.
Post by Rohit
Hello Morgan,
If you are refferring to the cert being in the conf/ssl/certs folder,
then yes, our.puppet.domain.pem is in the folder. When running the 'puppet
- computername.our.puppet.domain
- our.puppet.domain
- servername.our.puppet.domain
If it is a DNS issue, do I have to likely change something from the
docker-compose side?
Post by Morgan Rhodes
Hi Rohit,
Is the hostname from `/CN=our.puppet.domain` showing up in your
puppetserver's certificate? You can verify that with `puppet cert list
--all` on the puppetserver container. This looks like a DNS issue.
Post by Rohit
Hello, we currently have a puppet docker container setup and are
experiencing certificate issues. Basically, in our docker setup (on our
main server) I had generated and signed new certificates, but the puppet_db
‘Error: Could not retrieve catalog from remote server: SSL_connect
returned=1 errno=0 state=error: certificate verify failed: [unable to get
local issuer certificate for /CN=our.puppet.domain]
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0
state=error: certificate verify failed: [unable to get local issuer
certificate for /CN=our.puppet.domain]’
I have tried series of steps to solve this problem as it looks like
Puppet is not functioning correctly as our servers are not properly
listening to the host server. Any idea what I can do to solve this problem?
For reference, we are running Puppet_DB version 4.2 and Puppet Server
version 2.7.2, all of which is set up on a docker container environment on
one server.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
Release Engineer
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/fcf0c6da-82dd-4970-ab81-a60131b291f1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Morgan Rhodes
2018-10-19 23:37:47 UTC
Permalink
When you look at the output of `puppet cert list all` does the certificate
for your puppetmaster also include the alt name 'puppet'? (Something like
'alt names: "DNS:puppet", "DNS:testpuppet"'). If not, I'm guessing that's
your problem.

You mentioned in your earlier email that you were using puppetdb 4.2.0. I'm
assuming you're running the puppet/puppetdb:4.2.0 container. To get the
container entrypoint, I start the container manually with a custom
entrypoint so I can look around, there should be a file
'docker-entrypoint.sh' in the root directory of the container.

$ docker run --rm -it --entrypoint /bin/bash puppet/puppetdb:4.2.0
***@e09f677618d7:/# ls
Dockerfile bin boot dev docker-entrypoint.sh etc home lib lib64
media mnt opt proc root run sbin srv sys tmp usr var
***@e09f677618d7:/# cat docker-entrypoint.sh
#!/bin/bash

if [ ! -d "/etc/puppetlabs/puppetdb/ssl" ]; then
while ! nc -z puppet 8140; do
sleep 1
done
set -e
/opt/puppetlabs/bin/puppet agent --verbose --onetime --no-daemonize
--waitforcert 120
/opt/puppetlabs/server/bin/puppetdb ssl-setup -f
fi

exec /opt/puppetlabs/server/bin/puppetdb "$@"
***@e09f677618d7:/#

The docker-entrypoint.sh script in that version of the container doesn't
have any logic for a puppetserver with a non-default name, which means when
it runs `puppet agent --verbose --onetime --no-daemonize --waitforcert 120`
it will connect to the host named 'puppet'. From the link you have set up
in your docker-compose.yml, I'm assuming your puppetserver container name
is 'puppet' with the hostname 'puppet4.psd401.net'. Since the container
name is 'puppet', the puppetdb container is able to resolve 'puppet' as
'puppet4....', so when it runs puppet agent -t it can connect to the host,
but certificate validation will fail if puppet isn't listed as one of the
valid altnames for the puppet container.
Post by Rohit
1. puppet_db is trying to connect our.puppet.domain, there is no
docker-entrypoint.sh script that I was able to find.
2.
container_name: puppet_db
hostname: puppetdb.peninsula.wednet.edu
- 10.0.0.7
image: puppet/puppetdb:latest
- 8087:8080
- 8088:8081
- puppet
- puppet:puppet4.psd401.net
- puppetdbpostgres:postgres
- ./puppet-client.conf:/etc/puppetlabs/puppet/puppet.conf
- ./puppetdb_conf:/etc/puppetlabs/puppetdb/conf.d
- ./puppetdb_ssl:/etc/puppetlabs/puppet/ssl/
ipv4_address: 172.19.0.4
restart: always
2. The hostname that the puppetdb container is trying to connect to
is indeed the one listed on the certificate name on the puppet servers cert.
Post by Morgan Rhodes
1) what hostname is your puppetdb container trying to connect to
puppetserver at?
a) This should be in your docker-entrypoint.sh script in the puppetdb
container. Likely either 'puppet' or '$PUPPETSERVER_HOSTNAME' depending on
what variables you have set in your compose file and what version of the
puppetdb container you have.
2) Is the hostname your puppetdb container is trying to connect to listed
as one of the certificate names for your puppet server's cert?
a) For example, in my puppetserver container when I run `puppet cert
+ "testserver" (SHA256)
F0:31:6D:1D:03:82:C0:84:0D:FA:2B:28:5B:52:CB:18:88:87:61:5F:5A:F5:7E:AB:A2:73:29:44:BC:57:D0:99
(alt names: "DNS:testserver", "DNS:foo")
if my puppetdb container tries to connect to that host over any names
other than 'testserver' or 'foo' I get a certificate verify failed error.
Hello Morgan,
Post by Morgan Rhodes
Post by Rohit
If you are refferring to the cert being in the conf/ssl/certs folder,
then yes, our.puppet.domain.pem is in the folder. When running the 'puppet
- computername.our.puppet.domain
- our.puppet.domain
- servername.our.puppet.domain
If it is a DNS issue, do I have to likely change something from the
docker-compose side?
Post by Morgan Rhodes
Hi Rohit,
Is the hostname from `/CN=our.puppet.domain` showing up in your
puppetserver's certificate? You can verify that with `puppet cert list
--all` on the puppetserver container. This looks like a DNS issue.
Post by Rohit
Hello, we currently have a puppet docker container setup and are
experiencing certificate issues. Basically, in our docker setup (on our
main server) I had generated and signed new certificates, but the puppet_db
‘Error: Could not retrieve catalog from remote server: SSL_connect
returned=1 errno=0 state=error: certificate verify failed: [unable to get
local issuer certificate for /CN=our.puppet.domain]
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0
state=error: certificate verify failed: [unable to get local issuer
certificate for /CN=our.puppet.domain]’
I have tried series of steps to solve this problem as it looks like
Puppet is not functioning correctly as our servers are not properly
listening to the host server. Any idea what I can do to solve this problem?
For reference, we are running Puppet_DB version 4.2 and Puppet Server
version 2.7.2, all of which is set up on a docker container environment on
one server.
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
Post by Rohit
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
Release Engineer
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/fcf0c6da-82dd-4970-ab81-a60131b291f1%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/fcf0c6da-82dd-4970-ab81-a60131b291f1%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
***@puppet.com
Release Engineer
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CA%2BFnDv0VNCXsT_JnR6YeMw-vdmtcNg7Jg1F%2Bnt%3D3eKkTZZgovw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Rohit
2018-11-13 19:23:46 UTC
Permalink
Hello Morgan,

Apologies for the late response here, some of our Puppet services had
started working but it looks like the same issue has arised and I am not
entirely sure why. I did check the docker-entrypoint.sh file and indeed see
the very exact response as you posted. However my question is for the
"altname" that you suggested, would I change this in the docker-compose.yml
file? I also realize the full docker-compose.yml did not show up in my
previous post but have attached it again in a separate file.
Post by Morgan Rhodes
When you look at the output of `puppet cert list all` does the certificate
for your puppetmaster also include the alt name 'puppet'? (Something like
'alt names: "DNS:puppet", "DNS:testpuppet"'). If not, I'm guessing that's
your problem.
You mentioned in your earlier email that you were using puppetdb 4.2.0.
I'm assuming you're running the puppet/puppetdb:4.2.0 container. To get the
container entrypoint, I start the container manually with a custom
entrypoint so I can look around, there should be a file
'docker-entrypoint.sh' in the root directory of the container.
$ docker run --rm -it --entrypoint /bin/bash puppet/puppetdb:4.2.0
Dockerfile bin boot dev docker-entrypoint.sh etc home lib lib64
media mnt opt proc root run sbin srv sys tmp usr var
#!/bin/bash
if [ ! -d "/etc/puppetlabs/puppetdb/ssl" ]; then
while ! nc -z puppet 8140; do
sleep 1
done
set -e
/opt/puppetlabs/bin/puppet agent --verbose --onetime --no-daemonize
--waitforcert 120
/opt/puppetlabs/server/bin/puppetdb ssl-setup -f
fi
The docker-entrypoint.sh script in that version of the container doesn't
have any logic for a puppetserver with a non-default name, which means when
it runs `puppet agent --verbose --onetime --no-daemonize --waitforcert 120`
it will connect to the host named 'puppet'. From the link you have set up
in your docker-compose.yml, I'm assuming your puppetserver container name
is 'puppet' with the hostname 'puppet4.psd401.net'. Since the container
name is 'puppet', the puppetdb container is able to resolve 'puppet' as
'puppet4....', so when it runs puppet agent -t it can connect to the host,
but certificate validation will fail if puppet isn't listed as one of the
valid altnames for the puppet container.
Post by Rohit
1. puppet_db is trying to connect our.puppet.domain, there is no
docker-entrypoint.sh script that I was able to find.
2.
container_name: puppet_db
hostname: puppetdb.peninsula.wednet.edu
- 10.0.0.7
image: puppet/puppetdb:latest
- 8087:8080
- 8088:8081
- puppet
- puppet:puppet4.psd401.net
- puppetdbpostgres:postgres
- ./puppet-client.conf:/etc/puppetlabs/puppet/puppet.conf
- ./puppetdb_conf:/etc/puppetlabs/puppetdb/conf.d
- ./puppetdb_ssl:/etc/puppetlabs/puppet/ssl/
ipv4_address: 172.19.0.4
restart: always
2. The hostname that the puppetdb container is trying to connect
to is indeed the one listed on the certificate name on the puppet servers
cert.
Post by Morgan Rhodes
1) what hostname is your puppetdb container trying to connect to
puppetserver at?
a) This should be in your docker-entrypoint.sh script in the
puppetdb container. Likely either 'puppet' or '$PUPPETSERVER_HOSTNAME'
depending on what variables you have set in your compose file and what
version of the puppetdb container you have.
2) Is the hostname your puppetdb container is trying to connect to
listed as one of the certificate names for your puppet server's cert?
a) For example, in my puppetserver container when I run `puppet cert
+ "testserver" (SHA256)
F0:31:6D:1D:03:82:C0:84:0D:FA:2B:28:5B:52:CB:18:88:87:61:5F:5A:F5:7E:AB:A2:73:29:44:BC:57:D0:99
(alt names: "DNS:testserver", "DNS:foo")
if my puppetdb container tries to connect to that host over any names
other than 'testserver' or 'foo' I get a certificate verify failed error.
Hello Morgan,
Post by Morgan Rhodes
Post by Rohit
If you are refferring to the cert being in the conf/ssl/certs folder,
then yes, our.puppet.domain.pem is in the folder. When running the 'puppet
- computername.our.puppet.domain
- our.puppet.domain
- servername.our.puppet.domain
If it is a DNS issue, do I have to likely change something from the
docker-compose side?
Post by Morgan Rhodes
Hi Rohit,
Is the hostname from `/CN=our.puppet.domain` showing up in your
puppetserver's certificate? You can verify that with `puppet cert list
--all` on the puppetserver container. This looks like a DNS issue.
Post by Rohit
Hello, we currently have a puppet docker container setup and are
experiencing certificate issues. Basically, in our docker setup (on our
main server) I had generated and signed new certificates, but the puppet_db
[unable to get local issuer certificate for /CN=our.puppet.domain]
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0
state=error: certificate verify failed: [unable to get local issuer
certificate for /CN=our.puppet.domain]’
I have tried series of steps to solve this problem as it looks like
Puppet is not functioning correctly as our servers are not properly
listening to the host server. Any idea what I can do to solve this problem?
For reference, we are running Puppet_DB version 4.2 and Puppet Server
version 2.7.2, all of which is set up on a docker container environment on
one server.
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send
Post by Rohit
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
Release Engineer
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/fcf0c6da-82dd-4970-ab81-a60131b291f1%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/fcf0c6da-82dd-4970-ab81-a60131b291f1%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
Release Engineer
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/1015311c-0e2f-44f0-a096-6c5015d00d98%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Morgan Rhodes
2018-11-15 21:33:17 UTC
Permalink
Hi Rohit,

No, unfortunately, it's not just a change in your docker-compose.yml. When
you're generating the certs for your puppetserver, you'll want to make sure
you're passing the `--dns_alt_names=<altnames>`, so it would be something
like:
puppet cert generate puppet4.psd401.net --dns_alt_names=puppet,
puppet.psd401.net

Afterwards, you can confirm that your certificate has all of the altnames
with `puppet cert list --all`, you should see something like:
$ puppet cert list --all
+ "puppet4.psd401.net" (SHA256) <fingerprint> (alt names: "DNS:puppet",
"DNS:puppet4.psd401.net")
Post by Rohit
Hello Morgan,
Apologies for the late response here, some of our Puppet services had
started working but it looks like the same issue has arised and I am not
entirely sure why. I did check the docker-entrypoint.sh file and indeed see
the very exact response as you posted. However my question is for the
"altname" that you suggested, would I change this in the docker-compose.yml
file? I also realize the full docker-compose.yml did not show up in my
previous post but have attached it again in a separate file.
Post by Morgan Rhodes
When you look at the output of `puppet cert list all` does the
certificate for your puppetmaster also include the alt name 'puppet'?
(Something like 'alt names: "DNS:puppet", "DNS:testpuppet"'). If not, I'm
guessing that's your problem.
You mentioned in your earlier email that you were using puppetdb 4.2.0.
I'm assuming you're running the puppet/puppetdb:4.2.0 container. To get the
container entrypoint, I start the container manually with a custom
entrypoint so I can look around, there should be a file
'docker-entrypoint.sh' in the root directory of the container.
$ docker run --rm -it --entrypoint /bin/bash puppet/puppetdb:4.2.0
Dockerfile bin boot dev docker-entrypoint.sh etc home lib lib64
media mnt opt proc root run sbin srv sys tmp usr var
#!/bin/bash
if [ ! -d "/etc/puppetlabs/puppetdb/ssl" ]; then
while ! nc -z puppet 8140; do
sleep 1
done
set -e
/opt/puppetlabs/bin/puppet agent --verbose --onetime --no-daemonize
--waitforcert 120
/opt/puppetlabs/server/bin/puppetdb ssl-setup -f
fi
The docker-entrypoint.sh script in that version of the container doesn't
have any logic for a puppetserver with a non-default name, which means when
it runs `puppet agent --verbose --onetime --no-daemonize --waitforcert 120`
it will connect to the host named 'puppet'. From the link you have set up
in your docker-compose.yml, I'm assuming your puppetserver container name
is 'puppet' with the hostname 'puppet4.psd401.net'. Since the container
name is 'puppet', the puppetdb container is able to resolve 'puppet' as
'puppet4....', so when it runs puppet agent -t it can connect to the host,
but certificate validation will fail if puppet isn't listed as one of the
valid altnames for the puppet container.
Post by Rohit
1. puppet_db is trying to connect our.puppet.domain, there is no
docker-entrypoint.sh script that I was able to find.
2.
container_name: puppet_db
hostname: puppetdb.peninsula.wednet.edu
- 10.0.0.7
image: puppet/puppetdb:latest
- 8087:8080
- 8088:8081
- puppet
- puppet:puppet4.psd401.net
- puppetdbpostgres:postgres
- ./puppet-client.conf:/etc/puppetlabs/puppet/puppet.conf
- ./puppetdb_conf:/etc/puppetlabs/puppetdb/conf.d
- ./puppetdb_ssl:/etc/puppetlabs/puppet/ssl/
ipv4_address: 172.19.0.4
restart: always
2. The hostname that the puppetdb container is trying to connect
to is indeed the one listed on the certificate name on the puppet servers
cert.
Post by Morgan Rhodes
1) what hostname is your puppetdb container trying to connect to
puppetserver at?
a) This should be in your docker-entrypoint.sh script in the
puppetdb container. Likely either 'puppet' or '$PUPPETSERVER_HOSTNAME'
depending on what variables you have set in your compose file and what
version of the puppetdb container you have.
2) Is the hostname your puppetdb container is trying to connect to
listed as one of the certificate names for your puppet server's cert?
a) For example, in my puppetserver container when I run `puppet
+ "testserver" (SHA256)
F0:31:6D:1D:03:82:C0:84:0D:FA:2B:28:5B:52:CB:18:88:87:61:5F:5A:F5:7E:AB:A2:73:29:44:BC:57:D0:99
(alt names: "DNS:testserver", "DNS:foo")
if my puppetdb container tries to connect to that host over any names
other than 'testserver' or 'foo' I get a certificate verify failed error.
Hello Morgan,
Post by Morgan Rhodes
Post by Rohit
If you are refferring to the cert being in the conf/ssl/certs folder,
then yes, our.puppet.domain.pem is in the folder. When running the 'puppet
- computername.our.puppet.domain
- our.puppet.domain
- servername.our.puppet.domain
If it is a DNS issue, do I have to likely change something from the
docker-compose side?
Post by Morgan Rhodes
Hi Rohit,
Is the hostname from `/CN=our.puppet.domain` showing up in your
puppetserver's certificate? You can verify that with `puppet cert list
--all` on the puppetserver container. This looks like a DNS issue.
Post by Rohit
Hello, we currently have a puppet docker container setup and are
experiencing certificate issues. Basically, in our docker setup (on our
main server) I had generated and signed new certificates, but the puppet_db
[unable to get local issuer certificate for /CN=our.puppet.domain]
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0
state=error: certificate verify failed: [unable to get local issuer
certificate for /CN=our.puppet.domain]’
I have tried series of steps to solve this problem as it looks like
Puppet is not functioning correctly as our servers are not properly
listening to the host server. Any idea what I can do to solve this problem?
For reference, we are running Puppet_DB version 4.2 and Puppet Server
version 2.7.2, all of which is set up on a docker container environment on
one server.
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send
Post by Rohit
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
Release Engineer
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send
To view this discussion on the web visit
Post by Rohit
https://groups.google.com/d/msgid/puppet-users/fcf0c6da-82dd-4970-ab81-a60131b291f1%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/fcf0c6da-82dd-4970-ab81-a60131b291f1%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
Release Engineer
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/1015311c-0e2f-44f0-a096-6c5015d00d98%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/1015311c-0e2f-44f0-a096-6c5015d00d98%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
***@puppet.com
Release Engineer
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CA%2BFnDv195nhn9XJ%3DnbGza%3DqxfRcxgVhhMnR1ahkho08Jg6VPMA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Rohit
2018-11-16 23:08:47 UTC
Permalink
Hello Morgan

I was able to generate a new certificate with the alt name, and when doing
a 'puppet cert list --all' I see the following:

+ "puppet4.psd401.net" (SHA256) 1D:16:67:30:0D:62:CE:6C:2A:80:11:7E:C7:79:BA
:4F:25:C6:0E:E6:90:9D:4D:9F:86:4B:5C:42:A1:6D:09:96 (alt names: "DNS:puppet"
, "DNS:puppet4.psd401.net")

But when doing a docker logs on puppet_db, it still says:

Warning: Unable to fetch my node definition, but the agent run will
continue:
Warning: SSL_connect returned=1 errno=0 state=error: certificate verify
failed: [unable to get local issuer certificate for /CN=puppet4.psd401.net]
Info: Retrieving pluginfacts
Error: /File[/opt/puppetlabs/puppet/cache/facts.d]: Failed to generate
additional resources using 'eval_generate': SSL_connect returned=1 errno=0
state=error: certificate verify failed: [unable to get local issuer
certificate for /CN=puppet4.psd401.net]
Error: /File[/opt/puppetlabs/puppet/cache/facts.d]: Could not evaluate:
Could not retrieve file metadata for puppet:///pluginfacts: SSL_connect
returned=1 errno=0 state=error: certificate verify failed: [unable to get
local issuer certificate for /CN=puppet4.psd401.net]
Info: Retrieving plugin
Error: /File[/opt/puppetlabs/puppet/cache/lib]: Failed to generate
additional resources using 'eval_generate': SSL_connect returned=1 errno=0
state=error: certificate verify failed: [unable to get local issuer
certificate for /CN=puppet4.psd401.net]
Error: /File[/opt/puppetlabs/puppet/cache/lib]: Could not evaluate: Could
not retrieve file metadata for puppet:///plugins: SSL_connect returned=1
errno=0 state=error: certificate verify failed: [unable to get local issuer
certificate for /CN=puppet4.psd401.net]
Error: Could not retrieve catalog from remote server: SSL_connect
returned=1 errno=0 state=error: certificate verify failed: [unable to get
local issuer certificate for /CN=puppet4.psd401.net]
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0 state=error:
certificate verify failed: [unable to get local issuer certificate for
/CN=puppet4.psd401.net]

Not entirely sure what the problem here still can be... I did clean the
certs as well. Realizing this is a pretty old version of Puppet, would it
perhaps be better to do a clean install of Puppet in a non-docker
environment?
Post by Morgan Rhodes
Hi Rohit,
No, unfortunately, it's not just a change in your docker-compose.yml. When
you're generating the certs for your puppetserver, you'll want to make sure
you're passing the `--dns_alt_names=<altnames>`, so it would be something
puppet cert generate puppet4.psd401.net --dns_alt_names=puppet,
puppet.psd401.net
Afterwards, you can confirm that your certificate has all of the altnames
$ puppet cert list --all
+ "puppet4.psd401.net" (SHA256) <fingerprint> (alt names: "DNS:puppet",
"DNS:puppet4.psd401.net")
Post by Rohit
Hello Morgan,
Apologies for the late response here, some of our Puppet services had
started working but it looks like the same issue has arised and I am not
entirely sure why. I did check the docker-entrypoint.sh file and indeed see
the very exact response as you posted. However my question is for the
"altname" that you suggested, would I change this in the docker-compose.yml
file? I also realize the full docker-compose.yml did not show up in my
previous post but have attached it again in a separate file.
Post by Morgan Rhodes
When you look at the output of `puppet cert list all` does the
certificate for your puppetmaster also include the alt name 'puppet'?
(Something like 'alt names: "DNS:puppet", "DNS:testpuppet"'). If not, I'm
guessing that's your problem.
You mentioned in your earlier email that you were using puppetdb 4.2.0.
I'm assuming you're running the puppet/puppetdb:4.2.0 container. To get the
container entrypoint, I start the container manually with a custom
entrypoint so I can look around, there should be a file
'docker-entrypoint.sh' in the root directory of the container.
$ docker run --rm -it --entrypoint /bin/bash puppet/puppetdb:4.2.0
Dockerfile bin boot dev docker-entrypoint.sh etc home lib lib64
media mnt opt proc root run sbin srv sys tmp usr var
#!/bin/bash
if [ ! -d "/etc/puppetlabs/puppetdb/ssl" ]; then
while ! nc -z puppet 8140; do
sleep 1
done
set -e
/opt/puppetlabs/bin/puppet agent --verbose --onetime --no-daemonize
--waitforcert 120
/opt/puppetlabs/server/bin/puppetdb ssl-setup -f
fi
The docker-entrypoint.sh script in that version of the container doesn't
have any logic for a puppetserver with a non-default name, which means when
it runs `puppet agent --verbose --onetime --no-daemonize --waitforcert 120`
it will connect to the host named 'puppet'. From the link you have set up
in your docker-compose.yml, I'm assuming your puppetserver container name
is 'puppet' with the hostname 'puppet4.psd401.net'. Since the container
name is 'puppet', the puppetdb container is able to resolve 'puppet' as
'puppet4....', so when it runs puppet agent -t it can connect to the host,
but certificate validation will fail if puppet isn't listed as one of the
valid altnames for the puppet container.
Post by Rohit
1. puppet_db is trying to connect our.puppet.domain, there is no
docker-entrypoint.sh script that I was able to find.
2.
container_name: puppet_db
hostname: puppetdb.peninsula.wednet.edu
- 10.0.0.7
image: puppet/puppetdb:latest
- 8087:8080
- 8088:8081
- puppet
- puppet:puppet4.psd401.net
- puppetdbpostgres:postgres
- ./puppet-client.conf:/etc/puppetlabs/puppet/puppet.conf
- ./puppetdb_conf:/etc/puppetlabs/puppetdb/conf.d
- ./puppetdb_ssl:/etc/puppetlabs/puppet/ssl/
ipv4_address: 172.19.0.4
restart: always
2. The hostname that the puppetdb container is trying to connect
to is indeed the one listed on the certificate name on the puppet servers
cert.
Post by Morgan Rhodes
1) what hostname is your puppetdb container trying to connect to
puppetserver at?
a) This should be in your docker-entrypoint.sh script in the
puppetdb container. Likely either 'puppet' or '$PUPPETSERVER_HOSTNAME'
depending on what variables you have set in your compose file and what
version of the puppetdb container you have.
2) Is the hostname your puppetdb container is trying to connect to
listed as one of the certificate names for your puppet server's cert?
a) For example, in my puppetserver container when I run `puppet
+ "testserver" (SHA256)
F0:31:6D:1D:03:82:C0:84:0D:FA:2B:28:5B:52:CB:18:88:87:61:5F:5A:F5:7E:AB:A2:73:29:44:BC:57:D0:99
(alt names: "DNS:testserver", "DNS:foo")
if my puppetdb container tries to connect to that host over any
names other than 'testserver' or 'foo' I get a certificate verify failed
error.
Hello Morgan,
Post by Morgan Rhodes
Post by Rohit
If you are refferring to the cert being in the conf/ssl/certs folder,
then yes, our.puppet.domain.pem is in the folder. When running the 'puppet
- computername.our.puppet.domain
- our.puppet.domain
- servername.our.puppet.domain
If it is a DNS issue, do I have to likely change something from the
docker-compose side?
Post by Morgan Rhodes
Hi Rohit,
Is the hostname from `/CN=our.puppet.domain` showing up in your
puppetserver's certificate? You can verify that with `puppet cert list
--all` on the puppetserver container. This looks like a DNS issue.
Post by Rohit
Hello, we currently have a puppet docker container setup and are
experiencing certificate issues. Basically, in our docker setup (on our
main server) I had generated and signed new certificates, but the puppet_db
[unable to get local issuer certificate for /CN=our.puppet.domain]
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0
state=error: certificate verify failed: [unable to get local issuer
certificate for /CN=our.puppet.domain]’
I have tried series of steps to solve this problem as it looks like
Puppet is not functioning correctly as our servers are not properly
listening to the host server. Any idea what I can do to solve this problem?
For reference, we are running Puppet_DB version 4.2 and Puppet Server
version 2.7.2, all of which is set up on a docker container environment on
one server.
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send
Post by Rohit
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
Release Engineer
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send
To view this discussion on the web visit
Post by Rohit
https://groups.google.com/d/msgid/puppet-users/fcf0c6da-82dd-4970-ab81-a60131b291f1%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/fcf0c6da-82dd-4970-ab81-a60131b291f1%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
Release Engineer
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/1015311c-0e2f-44f0-a096-6c5015d00d98%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/1015311c-0e2f-44f0-a096-6c5015d00d98%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
Release Engineer
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/ddf39b7c-d1b0-4e84-b660-207383ad022c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
John Gelnaw
2018-11-21 16:43:15 UTC
Permalink
I had difficulties with the stock puppetdb entrypoint script. I wound up
changing it thusly:

#!/bin/bash

if [ ! -d "/etc/puppetlabs/puppetdb/ssl" ]; then
set -e
/opt/puppetlabs/bin/puppet config set certname ${HOSTNAME}
if [ ! -f "/etc/puppetlabs/puppet/ssl/certs/ca.pem" ]; then
while ! nc -z puppet 8140; do
sleep 1
done
/opt/puppetlabs/bin/puppet agent --verbose --onetime --no-daemonize
--waitforcert 120
fi
/opt/puppetlabs/server/bin/puppetdb ssl-setup -f
fi

exec /opt/puppetlabs/server/bin/puppetdb "$@"

And in case it helps, here's the docker-compose stanza for puppetdb:

puppetdb:
hostname: puppetdb
# image: puppet/puppetdb:4.4.0
build: builds/puppetdb
ports:
- 8080
- 8081
volumes:
- ./puppetdb/ssl:/etc/puppetlabs/puppet/ssl/

Note that I'm using a local build (I did the same for puppet itself, but
that's because we have a number of local customizations) instead of an
official image.

And the Dockerfile I used to build puppetdb:

FROM puppet/puppetdb:4.4.0

EXPOSE 8080
EXPOSE 8081

COPY docker-entrypoint.sh /

VOLUME /etc/puppetlabs/puppet/ssl
VOLUME /etc/puppetlabs/puppetdb

ENTRYPOINT ["/docker-entrypoint.sh", "foreground"]

So basically, I'm using the official image, but I'm overwriting the
docker-entrypoint.sh with my own version.

The important part is definitely the puppet config line to set the hostname
to match the container.

The filetest for ca.pem was something I put in to prevent a certain
condition that may have been unique to my environment-- apparently it was
possible to have a local certificate already, but not a (persistent)
puppetdb ssl configuration.
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/9f0bddd4-8a61-4fe6-aa74-2a11bc5bd0ae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Rohit
2018-11-21 18:59:09 UTC
Permalink
Thanks for the response, I did try those changes to see if it helps but
unfortunately the issue still exists
Post by John Gelnaw
I had difficulties with the stock puppetdb entrypoint script. I wound up
#!/bin/bash
if [ ! -d "/etc/puppetlabs/puppetdb/ssl" ]; then
set -e
/opt/puppetlabs/bin/puppet config set certname ${HOSTNAME}
if [ ! -f "/etc/puppetlabs/puppet/ssl/certs/ca.pem" ]; then
while ! nc -z puppet 8140; do
sleep 1
done
/opt/puppetlabs/bin/puppet agent --verbose --onetime --no-daemonize
--waitforcert 120
fi
/opt/puppetlabs/server/bin/puppetdb ssl-setup -f
fi
hostname: puppetdb
# image: puppet/puppetdb:4.4.0
build: builds/puppetdb
- 8080
- 8081
- ./puppetdb/ssl:/etc/puppetlabs/puppet/ssl/
Note that I'm using a local build (I did the same for puppet itself, but
that's because we have a number of local customizations) instead of an
official image.
FROM puppet/puppetdb:4.4.0
EXPOSE 8080
EXPOSE 8081
COPY docker-entrypoint.sh /
VOLUME /etc/puppetlabs/puppet/ssl
VOLUME /etc/puppetlabs/puppetdb
ENTRYPOINT ["/docker-entrypoint.sh", "foreground"]
So basically, I'm using the official image, but I'm overwriting the
docker-entrypoint.sh with my own version.
The important part is definitely the puppet config line to set the
hostname to match the container.
The filetest for ca.pem was something I put in to prevent a certain
condition that may have been unique to my environment-- apparently it was
possible to have a local certificate already, but not a (persistent)
puppetdb ssl configuration.
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/16bed3bf-e065-44f0-920b-00c7a96885a8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Rohit
2018-11-28 17:20:26 UTC
Permalink
Any idea if there are other steps I can consider? If not, should I simply
rebuild the system? If I do go this route, is there a way to backup all the
Puppet configurations set for servers and services that can be reimported
in a fresh install? Would it also be suggested to go a non-Docker route due
to stability?
Post by Morgan Rhodes
Hi Rohit,
No, unfortunately, it's not just a change in your docker-compose.yml. When
you're generating the certs for your puppetserver, you'll want to make sure
you're passing the `--dns_alt_names=<altnames>`, so it would be something
puppet cert generate puppet4.psd401.net --dns_alt_names=puppet,
puppet.psd401.net
Afterwards, you can confirm that your certificate has all of the altnames
$ puppet cert list --all
+ "puppet4.psd401.net" (SHA256) <fingerprint> (alt names: "DNS:puppet",
"DNS:puppet4.psd401.net")
Post by Rohit
Hello Morgan,
Apologies for the late response here, some of our Puppet services had
started working but it looks like the same issue has arised and I am not
entirely sure why. I did check the docker-entrypoint.sh file and indeed see
the very exact response as you posted. However my question is for the
"altname" that you suggested, would I change this in the docker-compose.yml
file? I also realize the full docker-compose.yml did not show up in my
previous post but have attached it again in a separate file.
Post by Morgan Rhodes
When you look at the output of `puppet cert list all` does the
certificate for your puppetmaster also include the alt name 'puppet'?
(Something like 'alt names: "DNS:puppet", "DNS:testpuppet"'). If not, I'm
guessing that's your problem.
You mentioned in your earlier email that you were using puppetdb 4.2.0.
I'm assuming you're running the puppet/puppetdb:4.2.0 container. To get the
container entrypoint, I start the container manually with a custom
entrypoint so I can look around, there should be a file
'docker-entrypoint.sh' in the root directory of the container.
$ docker run --rm -it --entrypoint /bin/bash puppet/puppetdb:4.2.0
Dockerfile bin boot dev docker-entrypoint.sh etc home lib lib64
media mnt opt proc root run sbin srv sys tmp usr var
#!/bin/bash
if [ ! -d "/etc/puppetlabs/puppetdb/ssl" ]; then
while ! nc -z puppet 8140; do
sleep 1
done
set -e
/opt/puppetlabs/bin/puppet agent --verbose --onetime --no-daemonize
--waitforcert 120
/opt/puppetlabs/server/bin/puppetdb ssl-setup -f
fi
The docker-entrypoint.sh script in that version of the container doesn't
have any logic for a puppetserver with a non-default name, which means when
it runs `puppet agent --verbose --onetime --no-daemonize --waitforcert 120`
it will connect to the host named 'puppet'. From the link you have set up
in your docker-compose.yml, I'm assuming your puppetserver container name
is 'puppet' with the hostname 'puppet4.psd401.net'. Since the container
name is 'puppet', the puppetdb container is able to resolve 'puppet' as
'puppet4....', so when it runs puppet agent -t it can connect to the host,
but certificate validation will fail if puppet isn't listed as one of the
valid altnames for the puppet container.
Post by Rohit
1. puppet_db is trying to connect our.puppet.domain, there is no
docker-entrypoint.sh script that I was able to find.
2.
container_name: puppet_db
hostname: puppetdb.peninsula.wednet.edu
- 10.0.0.7
image: puppet/puppetdb:latest
- 8087:8080
- 8088:8081
- puppet
- puppet:puppet4.psd401.net
- puppetdbpostgres:postgres
- ./puppet-client.conf:/etc/puppetlabs/puppet/puppet.conf
- ./puppetdb_conf:/etc/puppetlabs/puppetdb/conf.d
- ./puppetdb_ssl:/etc/puppetlabs/puppet/ssl/
ipv4_address: 172.19.0.4
restart: always
2. The hostname that the puppetdb container is trying to connect
to is indeed the one listed on the certificate name on the puppet servers
cert.
Post by Morgan Rhodes
1) what hostname is your puppetdb container trying to connect to
puppetserver at?
a) This should be in your docker-entrypoint.sh script in the
puppetdb container. Likely either 'puppet' or '$PUPPETSERVER_HOSTNAME'
depending on what variables you have set in your compose file and what
version of the puppetdb container you have.
2) Is the hostname your puppetdb container is trying to connect to
listed as one of the certificate names for your puppet server's cert?
a) For example, in my puppetserver container when I run `puppet
+ "testserver" (SHA256)
F0:31:6D:1D:03:82:C0:84:0D:FA:2B:28:5B:52:CB:18:88:87:61:5F:5A:F5:7E:AB:A2:73:29:44:BC:57:D0:99
(alt names: "DNS:testserver", "DNS:foo")
if my puppetdb container tries to connect to that host over any
names other than 'testserver' or 'foo' I get a certificate verify failed
error.
Hello Morgan,
Post by Morgan Rhodes
Post by Rohit
If you are refferring to the cert being in the conf/ssl/certs folder,
then yes, our.puppet.domain.pem is in the folder. When running the 'puppet
- computername.our.puppet.domain
- our.puppet.domain
- servername.our.puppet.domain
If it is a DNS issue, do I have to likely change something from the
docker-compose side?
Post by Morgan Rhodes
Hi Rohit,
Is the hostname from `/CN=our.puppet.domain` showing up in your
puppetserver's certificate? You can verify that with `puppet cert list
--all` on the puppetserver container. This looks like a DNS issue.
Post by Rohit
Hello, we currently have a puppet docker container setup and are
experiencing certificate issues. Basically, in our docker setup (on our
main server) I had generated and signed new certificates, but the puppet_db
[unable to get local issuer certificate for /CN=our.puppet.domain]
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0
state=error: certificate verify failed: [unable to get local issuer
certificate for /CN=our.puppet.domain]’
I have tried series of steps to solve this problem as it looks like
Puppet is not functioning correctly as our servers are not properly
listening to the host server. Any idea what I can do to solve this problem?
For reference, we are running Puppet_DB version 4.2 and Puppet Server
version 2.7.2, all of which is set up on a docker container environment on
one server.
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send
Post by Rohit
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
Release Engineer
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send
To view this discussion on the web visit
Post by Rohit
https://groups.google.com/d/msgid/puppet-users/fcf0c6da-82dd-4970-ab81-a60131b291f1%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/fcf0c6da-82dd-4970-ab81-a60131b291f1%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
Release Engineer
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/1015311c-0e2f-44f0-a096-6c5015d00d98%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/1015311c-0e2f-44f0-a096-6c5015d00d98%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
Release Engineer
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/9d319859-8f13-4e6e-97c9-c2366152a4e3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Morgan Rhodes
2018-11-30 16:48:00 UTC
Permalink
Hi Rohit,

I don't have great ideas about what's going on in your environment. Are you
using custom built containers or the puppet namespaced containers from
hub.docker.com. You could try applying this patch (
https://github.com/puppetlabs/puppetdb/commit/a1ab2f50598f12ac51acb21f256232143891dbc1)
and setting PUPPETSERVER_HOSTNAME in your docker-compose.yml to
puppet4.psd401.net.

In one of your earlier emails you mentioned you were using puppetserver
2.7, but it looks like in the compose file you attached you're using the
puppet/puppetserver:latest, which is puppetserver 6.x, just want to make
sure I know what versions of things you're running with here.

The containers for puppetserver 2.7 / puppetdb 4.2 are definitely old, and
with recent efforts we have made a number of changes and improvements to
the containers and the compose stack (
https://github.com/puppetlabs/pupperware), but that work has been for
puppetserver and puppetdb 5+.

If you run `puppet agent -t` on your puppetserver container does it
succeed? (docker-compose exec puppet puppet agent -t)
Post by Rohit
Any idea if there are other steps I can consider? If not, should I simply
rebuild the system? If I do go this route, is there a way to backup all the
Puppet configurations set for servers and services that can be reimported
in a fresh install? Would it also be suggested to go a non-Docker route due
to stability?
Post by Morgan Rhodes
Hi Rohit,
No, unfortunately, it's not just a change in your docker-compose.yml.
When you're generating the certs for your puppetserver, you'll want to make
sure you're passing the `--dns_alt_names=<altnames>`, so it would be
puppet cert generate puppet4.psd401.net --dns_alt_names=puppet,
puppet.psd401.net
Afterwards, you can confirm that your certificate has all of the altnames
$ puppet cert list --all
+ "puppet4.psd401.net" (SHA256) <fingerprint> (alt names: "DNS:puppet",
"DNS:puppet4.psd401.net")
Hello Morgan,
Post by Morgan Rhodes
Post by Rohit
Apologies for the late response here, some of our Puppet services had
started working but it looks like the same issue has arised and I am not
entirely sure why. I did check the docker-entrypoint.sh file and indeed see
the very exact response as you posted. However my question is for the
"altname" that you suggested, would I change this in the docker-compose.yml
file? I also realize the full docker-compose.yml did not show up in my
previous post but have attached it again in a separate file.
Post by Morgan Rhodes
When you look at the output of `puppet cert list all` does the
certificate for your puppetmaster also include the alt name 'puppet'?
(Something like 'alt names: "DNS:puppet", "DNS:testpuppet"'). If not, I'm
guessing that's your problem.
You mentioned in your earlier email that you were using puppetdb 4.2.0.
I'm assuming you're running the puppet/puppetdb:4.2.0 container. To get the
container entrypoint, I start the container manually with a custom
entrypoint so I can look around, there should be a file
'docker-entrypoint.sh' in the root directory of the container.
$ docker run --rm -it --entrypoint /bin/bash puppet/puppetdb:4.2.0
Dockerfile bin boot dev docker-entrypoint.sh etc home lib
lib64 media mnt opt proc root run sbin srv sys tmp usr var
#!/bin/bash
if [ ! -d "/etc/puppetlabs/puppetdb/ssl" ]; then
while ! nc -z puppet 8140; do
sleep 1
done
set -e
/opt/puppetlabs/bin/puppet agent --verbose --onetime --no-daemonize
--waitforcert 120
/opt/puppetlabs/server/bin/puppetdb ssl-setup -f
fi
The docker-entrypoint.sh script in that version of the container
doesn't have any logic for a puppetserver with a non-default name, which
means when it runs `puppet agent --verbose --onetime --no-daemonize
--waitforcert 120` it will connect to the host named 'puppet'. From the
link you have set up in your docker-compose.yml, I'm assuming your
puppetserver container name is 'puppet' with the hostname '
puppet4.psd401.net'. Since the container name is 'puppet', the
puppetdb container is able to resolve 'puppet' as 'puppet4....', so when it
runs puppet agent -t it can connect to the host, but certificate validation
will fail if puppet isn't listed as one of the valid altnames for the
puppet container.
Post by Rohit
1. puppet_db is trying to connect our.puppet.domain, there is no
docker-entrypoint.sh script that I was able to find.
2.
container_name: puppet_db
hostname: puppetdb.peninsula.wednet.edu
- 10.0.0.7
image: puppet/puppetdb:latest
- 8087:8080
- 8088:8081
- puppet
- puppet:puppet4.psd401.net
- puppetdbpostgres:postgres
- ./puppet-client.conf:/etc/puppetlabs/puppet/puppet.conf
- ./puppetdb_conf:/etc/puppetlabs/puppetdb/conf.d
- ./puppetdb_ssl:/etc/puppetlabs/puppet/ssl/
ipv4_address: 172.19.0.4
restart: always
2. The hostname that the puppetdb container is trying to
connect to is indeed the one listed on the certificate name on the puppet
servers cert.
Post by Morgan Rhodes
1) what hostname is your puppetdb container trying to connect to
puppetserver at?
a) This should be in your docker-entrypoint.sh script in the
puppetdb container. Likely either 'puppet' or '$PUPPETSERVER_HOSTNAME'
depending on what variables you have set in your compose file and what
version of the puppetdb container you have.
2) Is the hostname your puppetdb container is trying to connect to
listed as one of the certificate names for your puppet server's cert?
a) For example, in my puppetserver container when I run `puppet
+ "testserver" (SHA256)
F0:31:6D:1D:03:82:C0:84:0D:FA:2B:28:5B:52:CB:18:88:87:61:5F:5A:F5:7E:AB:A2:73:29:44:BC:57:D0:99
(alt names: "DNS:testserver", "DNS:foo")
if my puppetdb container tries to connect to that host over any
names other than 'testserver' or 'foo' I get a certificate verify failed
error.
Hello Morgan,
Post by Morgan Rhodes
Post by Rohit
If you are refferring to the cert being in the conf/ssl/certs
folder, then yes, our.puppet.domain.pem is in the folder. When running the
- computername.our.puppet.domain
- our.puppet.domain
- servername.our.puppet.domain
If it is a DNS issue, do I have to likely change something from the
docker-compose side?
Post by Morgan Rhodes
Hi Rohit,
Is the hostname from `/CN=our.puppet.domain` showing up in your
puppetserver's certificate? You can verify that with `puppet cert list
--all` on the puppetserver container. This looks like a DNS issue.
Post by Rohit
Hello, we currently have a puppet docker container setup and are
experiencing certificate issues. Basically, in our docker setup (on our
main server) I had generated and signed new certificates, but the puppet_db
[unable to get local issuer certificate for /CN=our.puppet.domain]
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0
state=error: certificate verify failed: [unable to get local issuer
certificate for /CN=our.puppet.domain]’
I have tried series of steps to solve this problem as it looks
like Puppet is not functioning correctly as our servers are not properly
listening to the host server. Any idea what I can do to solve this problem?
For reference, we are running Puppet_DB version 4.2 and Puppet Server
version 2.7.2, all of which is set up on a docker container environment on
one server.
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it,
Post by Rohit
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/899d2bf7-ceed-4d9e-bd24-c4ba2cc93928%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
Release Engineer
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send
To view this discussion on the web visit
Post by Rohit
https://groups.google.com/d/msgid/puppet-users/fcf0c6da-82dd-4970-ab81-a60131b291f1%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/fcf0c6da-82dd-4970-ab81-a60131b291f1%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
Release Engineer
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send
To view this discussion on the web visit
Post by Rohit
https://groups.google.com/d/msgid/puppet-users/1015311c-0e2f-44f0-a096-6c5015d00d98%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/1015311c-0e2f-44f0-a096-6c5015d00d98%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
Release Engineer
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/9d319859-8f13-4e6e-97c9-c2366152a4e3%40googlegroups.com
<https://groups.google.com/d/msgid/puppet-users/9d319859-8f13-4e6e-97c9-c2366152a4e3%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
Morgan Rhodes
***@puppet.com
Release Engineer
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+***@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CA%2BFnDv2pbbH5pQJdgkZEYnHJd475mWX2QDX%3D_4CCtVwHohtqkg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Continue reading on narkive:
Loading...