[Puppet Users] create an array of hashes with reduce

[Arnau]

Hi all,

I'm using puppet 5.3.

I'd like to build proxy_pass array of hashes using the *reduce *puppet
built-in function <https://puppet.com/docs/puppet/5.5/function.html#reduce>and
picking the values from a nested hash in hiera.

I first played a little bit with the reduce function:

$dirs = ['static','media','photos']
$proxy = $dirs.reduce([ { 'path' => '/', 'url' => "http://localhost:
${port}/" } ]) |$memo, $value| { $memo + [ { 'path' => "/$value/", 'url' =>
'!' } ] }
notify { "MemoOut: $proxy" : ;}

If I puppet apply the above code the output looks like:

Notice: MemoOut: [{path => /, url => http://localhost:189/}, {path =>
/static/, url => !}, {path => /media/, url => !}, {path => /photos/, url =>
!}]
Notice: /Stage[main]/Main/Node[default]/Notify[MemoOut: [{path => /, url =>
http://localhost:189/}, {path => /static/, url => !}, {path => /media/, url
=> !}, {path => /photos/, url => !}]]/message: defined 'message' as 'MemoOut:
[{path => /, url => http://localhost:189/}, {path => /static/, url => !},
{path => /media/, url => !}, {path => /photos/, url => !}]'

*To my eyes this is an array of hashes, and that's what I need to pass to
the apache vhost define.*

So, now I want to pick the array (list of directories) from a nested hash
in hiera that look like:

application:
apache:
enable: true
servername: application
static_served:
- static

I have a define that picks the above hash.... The array with the list of
directories becomes "${apache['static_served']}" inside teh define.
I can print it using notice ${apache['static_served']}":

Notice: static_served: application [static]

This looks like an array but *is_array *says that this is *not *an array
anymore.
And this is my first question, why it is not an array anymore?


I make this an array using any2array (array un puppet 5) so I can still
call the reduce function::

$var=(Array("${apache['static_served']}"))

so the *reduce *line now looks like:

$_proxy_pass = $var.reduce([ { 'path' => '/', 'url' => "http://localhost:
${port}/" } ]) |$memo, $value| { $memo + [ { 'path' => "/$value/", 'url' =>
'!' } ] }

And here is where everythiong starts to make even less sense:

If I print the output (using notice $_proxy_pass and ) and I get a weird
output:

Notice: /Stage[main]/.../Python::Virtualenv[application]/Notify[PROXY_PASS:
[{path => /, url => http://local
host:11080/}, {path => /[/, url => !}, {path => /s/, url => !}, {path => /t/,
url => !}, {path => /a/, url => !}, {path => /t/, url => !}, {path => /i/,
url => !},
{path => /c/, url => !}, {path => /]/, url => !}]]/message: defined
'message' as 'PROXY_PASS: [{path => /, url => http://localhost:11080/},
{path => /[/, url => !
}, {path => /s/, url => !}, {path => /t/, url => !}, {path => /a/, url =>
!}, {path => /t/, url => !}, {path => /i/, url => !}, {path => /c/, url =>
!}, {path => /
]/, url => !}]'

reduce is taking all letters from "static" instead of 'static" as the first
(an uniqe) element from the array....

Later, in my code, I configure the proxy_pass in the apache:vhost like:

apache::vhost { "${apache['servername']}":
[...]
proxy_pass => "$_proxy_pass",
[...]

But, in the apache file, the proxy pass parameters looks like

ProxyPass path url
ProxyPassReverse path url
ProxyPass path url
ProxyPassReverse path url
[...]

So it's picking the keys from the hashes in the above array of hashes and
not the values.

I don't understand what is going on. Any help will be appreciated.

Best,
Arnau

[Henrik Lindberg]

Post by Arnau
Hi all,
I'm using puppet 5.3.
I'd like to build proxy_pass array of hashes using the /reduce
/puppetbuilt-in function
<https://puppet.com/docs/puppet/5.5/function.html#reduce>and picking the
values from a nested hash in hiera.
$dirs = ['static','media','photos']
$proxy = $dirs.reduce([ { 'path' => '/', 'url' =>
"http://localhost:${port}/" } ]) |$memo, $value| { $memo + [ { 'path' =>
"/$value/", 'url' => '!' } ] }
notify { "MemoOut: $proxy" : ;}

Avoid having notify with a title containing [ ] - it can get very
confusing. Use a title like "testing", and set the message instead.

You give the reduce an array with a hash in it, and you then append to
that array with an array containing a hash. The result will be an
array where each element is a hash.

Post by Arnau
Notice: MemoOut: [{path => /, url => http://localhost:189/}, {path =>
/static/, url => !}, {path => /media/, url => !}, {path => /photos/, url
=> !}]
Notice: /Stage[main]/Main/Node[default]/Notify[MemoOut: [{path => /, url
=> http://localhost:189/}, {path => /static/, url => !}, {path =>
/media/, url => !}, {path => /photos/, url => !}]]/message: defined
'message' as 'MemoOut: [{path => /, url => http://localhost:189/}, {path
=> /static/, url => !}, {path => /media/, url => !}, {path => /photos/,
url => !}]'
/To my eyes this is an array of hashes, and that's what I need to pass
to the apache vhost define./
So, now I want to pick the array (list of directories) from a nested
enable: true
      servername: application
        - static
I have a define that picks the above hash.... The array with the list of
directories becomes "${apache['static_served']}" inside teh define.
Notice: static_served: application [static]
This looks like an array but /*is_array */says that this is *not *an
array anymore.
And this is my first question, why it is not an array anymore?
I make this an array using any2array (array un puppet 5) so I can still
$var=(Array("${apache['static_served']}"))

You are asking Array to create an array out of a string. It will
construct that as an array where each element is a single character string.

You probably wanted

$var = Array($apache['static_served'])

Which by the way is a no-op since $apache['static_served'] already is an
array (as per the hiera data you supplied).

Post by Arnau
$_proxy_pass = $var.reduce([ { 'path' => '/', 'url' =>
"http://localhost:${port}/" } ]) |$memo, $value| { $memo + [ { 'path' =>
"/$value/", 'url' => '!' } ] }

You are now performing a reduce based on a sequence of single character
strings. This is indeed confusing as you have already gone off the road.

Hope this helps you get back on track.

- henrik

Post by Arnau
If I print the output (using notice $_proxy_pass and )  and I get a
[{path => /, url => http://local
host:11080/}, {path => /[/, url => !}, {path => /s/, url => !}, {path =>
/t/, url => !}, {path => /a/, url => !}, {path => /t/, url => !}, {path
=> /i/, url => !},
 {path => /c/, url => !}, {path => /]/, url => !}]]/message: defined
'message' as 'PROXY_PASS: [{path => /, url => http://localhost:11080/},
{path => /[/, url => !
}, {path => /s/, url => !}, {path => /t/, url => !}, {path => /a/, url
=> !}, {path => /t/, url => !}, {path => /i/, url => !}, {path => /c/,
url => !}, {path => /
]/, url => !}]'
reduce is taking all letters from "static" instead of 'static" as the
first (an uniqe) element from the array....
[...]
proxy_pass     => "$_proxy_pass",
[...]
But, in the apache file, the proxy pass parameters  looks like
ProxyPass path url
 ProxyPassReverse path url
ProxyPass path url
 ProxyPassReverse path url
[...]
So it's picking the keys from the hashes in the above array of hashes
and not the values.
I don't understand what is going on. Any help will be appreciated.
Best,
Arnau
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CAM69jx-wKXmP%3DhfPLsMsrYDeOS7TCTQaQ9XyhUaoWJ9uq%3DeZEg%40mail.gmail.com
<https://groups.google.com/d/msgid/puppet-users/CAM69jx-wKXmP%3DhfPLsMsrYDeOS7TCTQaQ9XyhUaoWJ9uq%3DeZEg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

[Arnau]

Hi Henrik,

first of all, thanks for your answer.

El mar., 6 nov. 2018 a las 20:12, Henrik Lindberg (<
***@puppet.com>) escribió:
[...]

Post by Henrik Lindberg

Post by Arnau
$dirs = ['static','media','photos']
$proxy = $dirs.reduce([ { 'path' => '/', 'url' =>
"http://localhost:${port}/" } ]) |$memo, $value| { $memo + [ { 'path'

=>

Post by Arnau
"/$value/", 'url' => '!' } ] }
notify { "MemoOut: $proxy" : ;}

Avoid having notify with a title containing [ ] - it can get very
confusing. Use a title like "testing", and set the message instead.

Ok, thanks. Then:
notify {"test $name" :
message => "static_served: ${apache[static_served]}",
}
produces:
defined 'message' as 'static_served: [static]'

*I need to enclose the hash in curly braces otherwise it shows the whole
hash like:*
defined 'message' as 'Message {enable => true, servername => application,
static_served => [static]}[static_served]'


You give the reduce an array with a hash in it, and you then append to

Post by Henrik Lindberg
that array with an array containing a hash. The result will be an
array where each element is a hash.

Yes, exaclty. That's what I need (if I have not missunderstood it) to pass
tot he proxy_pass parameter in the apache forge module:

proxy_pass => [
{ 'path' => '/a', 'url' => 'http://backend-a/' },
{ 'path' => '/b', 'url' => 'http://backend-b/' },
]

[...]

Post by Henrik Lindberg

Post by Arnau
I make this an array using any2array (array un puppet 5) so I can still
$var=(Array("${apache['static_served']}"))

You are asking Array to create an array out of a string. It will
construct that as an array where each element is a single character string.

Ah, ok, cause I'm passing the first element of the array and not the array
to the Array function? and this is becasue of the quoting, right?

You probably wanted

Post by Henrik Lindberg
$var = Array($apache['static_served'])

Post by Arnau
$_proxy_pass = $var.reduce([ { 'path' => '/', 'url' =>
"http://localhost:${port}/" } ]) |$memo, $value| { $memo + [ { 'path'

=>

Post by Arnau
"/$value/", 'url' => '!' } ] }

You are now performing a reduce based on a sequence of single character
strings. This is indeed confusing as you have already gone off the road.

So, at the end it was as simple as calling the reduce function like:
$apache['static_served'].reduce

(I as using a more larger hiera data and, in the very first application,
the static_serverd array was empty, so reduce was failling all the time
cause the array was empty... Damn!!

now the code looks like:

if $apache['static_served'] {
$_proxy_pass = $apache['static_served'].reduce([ { 'path' => '/', 'url'
=> "http://localhost:${port}/" } ]) |$memo, $value| { $memo + [ { 'path' =>
"/$value/", 'url' => '!' } ] }
}else{
$_proxy_pass = [ { 'path' => '/', 'url' => "http://localhost:${port}/"
} ]
}
notify { "PROXY: $name" :
message => $_proxy_pass,
}

But the reduce is now not working:

Notice: {"path"=>"/", "url"=>"http://localhost:56902/"}
Notify[PROXY: application]/message: defined 'message' as {
'path' => '/',
'url' => 'http://localhost:56902/'
}


what am I doing wrong now?



BTW, I find the hash notation/manipulation a little confusing... According
to https://puppet.com/docs/puppet/5.4/lang_data_hash.html it must be as
simple as:

$key[$value]

And quotes should be used in "keys" when they are strings, but in the above
example:

$var = Array($apache['static_served'])

you are quoting the $value when it's a array and not a string...

what is the generic rule for quoting in hashes?


Hope this helps you get back on track.

Post by Henrik Lindberg
- henrik

Thanks again!
Arnau

[Henrik Lindberg]

Post by Arnau
Hi Henrik,
first of all, thanks for your answer.
El mar., 6 nov. 2018 a las 20:12, Henrik Lindberg
[...]

Post by Arnau
$dirs = ['static','media','photos']
$proxy = $dirs.reduce([ { 'path' => '/', 'url' =>
"http://localhost:${port}/" } ]) |$memo, $value| { $memo + [ {

'path' =>

Post by Arnau
"/$value/", 'url' => '!' } ] }
notify { "MemoOut: $proxy" : ;}

Avoid having  notify with a title containing [ ] - it can get very
confusing. Use a title like "testing", and set the message instead.
    message => "static_served: ${apache[static_served]}",
 }
 defined 'message' as 'static_served: [static]'
/I need to enclose the hash in curly braces otherwise it shows the whole
hash like:/
defined 'message' as 'Message {enable => true, servername =>
application, static_served => [static]}[static_served]'
You give the reduce an array with a hash in it, and you then append to
that array with an array containing a hash. The result will be an
array where each element is a hash.
Yes, exaclty. That's what I need (if I have not missunderstood it) to
proxy_pass => [
{ 'path' => '/a', 'url' => 'http://backend-a/' },
{ 'path' => '/b', 'url' => 'http://backend-b/' }, ]
[...]

Post by Arnau
I make this an array using any2array (array un puppet 5) so I can

still

Post by Arnau
$var=(Array("${apache['static_served']}"))

You are asking Array to create an array out of a string. It will
construct that as an array where each element is a single character string.
Ah, ok, cause I'm passing the first element of the array and not the
array to the Array function? and this is becasue of the quoting, right?
You probably wanted
$var = Array($apache['static_served'])

Post by Arnau
$_proxy_pass = $var.reduce([ { 'path' => '/', 'url' =>
"http://localhost:${port}/" } ]) |$memo, $value| { $memo + [ {

'path' =>

Post by Arnau
"/$value/", 'url' => '!' } ] }

You are now performing a reduce based on a sequence of single character
strings. This is indeed confusing as you have already gone off the road.
$apache['static_served'].reduce
(I as using a more larger hiera data and, in the very first application,
the static_serverd array was empty, so reduce was failling all the time
cause the array was empty... Damn!!
if $apache['static_served'] {
    $_proxy_pass =  $apache['static_served'].reduce([ { 'path' => '/',
'url' => "http://localhost:${port}/" } ]) |$memo, $value| { $memo + [ {
'path' => "/$value/", 'url' => '!' } ] }
  }else{
   $_proxy_pass =  [ { 'path' => '/', 'url' =>
"http://localhost:${port}/" } ]
  }
    message => $_proxy_pass,
  }
Notice: {"path"=>"/", "url"=>"http://localhost:56902/"}
Notify[PROXY: application]/message: defined 'message' as {
'path' => '/',
'url' => 'http://localhost:56902/'
}
what am I doing wrong now?

Not sure - but I simplified your code, and this works:

test.pp
------
$apache = { 'static_served' => [ 'test1', 'test2'] }
$port = 8888

$start_value = [{
'path' => '/',
'url' => "http://localhost:${port}/"
}]
$_proxy_pass = if $apache['static_served'] =~ Array {
$apache['static_served'].reduce($start_value) |$memo, $value| {
$memo + [{'path' => "/${value}/", 'url' => '!' }]
}
} else {
$start_value
}
notice($_proxy_pass)

Post by Arnau

Post by Arnau
puppet apply test.pp

Notice: Scope(Class[main]): [{path => /, url => http://localhost:8888/},
{path => /test1/, url => !}, {path => /test2/, url => !}]


If you change the last notice to do this:

notice String($_proxy_pass)

You will get better looking output with quotes around strings:

Notice: Scope(Class[main]): [{'path' => '/', 'url' =>
'http://localhost:8888/'}, {'path' => '/test1/', 'url' => '!'}, {'path'
=> '/test2/', 'url' => '!'}]

Which to me looks like what you intended.

The result is the same in both cases, but the output you get by using
String to convert the result before the notice gives you sane
formatting. If just doing a notice or setting the value as a "message"
you will get a backwards compatible transformation to string that does
not show quotes etc. You may not want the quotes in the actual result
you are going to use, but it is valuable when testing/debugging.

Hope this helps you.

Best,
- henrik

Post by Arnau
BTW, I find the hash notation/manipulation a little confusing...
According to https://puppet.com/docs/puppet/5.4/lang_data_hash.html it
$key[$value]
And quotes should be used in "keys" when they are strings, but in the
$var = Array($apache['static_served'])
 you are quoting the $value when it's a array and not a string...
what is the generic rule for quoting in hashes?

There is something you must have misunderstood. You need to quote values
when you need to make sure it is a string. Puppet allows using what is
known as bare words - i.e. some characters that make out a word, when
that bare word does not mean anything in particular to puppet.

For example:

notice(hello)

Which works because the word hello is not special to puppet. A bare word
is just a string. This is exactly the same thing:

notice("hello")

Which is the same as:

notice('hello')

The documentation says that you should quote string keys in hashes -
i.e. it is recommended to write:

{"hello" => "world"}

instead of

{ hello => world }

since you never know if bare words will have no meaning to puppet for
ever - by quoting you are telling puppet "hey, this is really a string".

For example - in this example - you *must* quote:

{ "if" => "is the question"}

If you remove the quotes around "if" you will get a syntax error.

Next thing - the difference between $x and "$x" or "${x}" - the $x means
the value of the variable named x. The "$x" means a string where the
part $x is replaced by the value of the variable x, and "${x}" is the
same thing as "$x", but allows more elaborate expression inside the
braces. Always use the "${ }" form.

Post by Arnau
Hope this helps you get back on track.
- henrik
Thanks again!
Arnau
--
You received this message because you are subscribed to the Google
Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CAM69jx9evODEdpT%2B-aCoHtf6iZc1j1pxguA7%2B6onaBAb31daBA%40mail.gmail.com
<https://groups.google.com/d/msgid/puppet-users/CAM69jx9evODEdpT%2B-aCoHtf6iZc1j1pxguA7%2B6onaBAb31daBA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

[Arnau]

Hi Henrik,

I did not answer before cause I was playing a little more with the code....
thanks a lot for your asnwers and your explanation.

El mié., 7 nov. 2018 a las 11:37, Henrik Lindberg (<

Post by Henrik Lindberg

Post by Arnau
what am I doing wrong now?

My code is working as well. But, for soe reason, I have to move the notify
below in my code ....

Post by Henrik Lindberg

Post by Arnau
BTW, I find the hash notation/manipulation a little confusing...
According to https://puppet.com/docs/puppet/5.4/lang_data_hash.html it
$key[$value]
And quotes should be used in "keys" when they are strings, but in the
$var = Array($apache['static_served'])
you are quoting the $value when it's a array and not a string...
what is the generic rule for quoting in hashes?

There is something you must have misunderstood. You need to quote values
Thanks for this explanatin!! very clarifying!

Best,
Arnau