[Puppet Users] Hiera doesn't work with JSON

Greetings,

I am having an issue with Hiera again. Bangin my head on it for a few hours
and not getting anywhere. Since this group was able to help last time, I
have my fingers crossed you can help me again. :-D

I am trying to use JSON with Hiera this time.

First, fully updated dev box.
$ cat /etc/redhat-release
Scientific Linux release 6.7 (Carbon)

$ sudo yum update
Loaded plugins: refresh-packagekit, security
Setting up Update Process
No Packages marked for Update

$ rpm -qa | egrep '(puppet|fact|hiera|ruby)'
facter-2.4.4-1.el6.x86_64
puppet-3.8.2-1.el6.noarch
hiera-1.3.4-1.el6.noarch
puppet-server-3.8.2-1.el6.noarch
ruby-libs-1.8.7.374-4.el6_6.x86_64
ruby-rdoc-1.8.7.374-4.el6_6.x86_64
ruby-shadow-2.2.0-2.el6.x86_64
ruby-1.8.7.374-4.el6_6.x86_64
rubygems-1.3.7-5.el6.noarch
ruby-augeas-0.4.1-3.el6.x86_64
rubygem-deep_merge-1.0.0-2.el6.noarch
libselinux-ruby-2.0.94-5.8.el6.x86_64
rubygem-json-1.5.5-3.el6.x86_64
ruby-irb-1.8.7.374-4.el6_6.x86_64


Let's use the puppet documentation to get started, shall we?
https://docs.puppetlabs.com/hiera/1/data_sources.html

$ cat /etc/puppet/hiera.yaml
---
:backends:
    - json
:hierarchy:
    - common
:merge_behavior: "deeper"
:json:
    :datadir: /etc/puppet/hieradata

BTW: the doc's actual JSON example is broken as it is missing the [ ]
around it. I included it. below and if you don't you fail the parser and
Hiera just ignores everything instead of giving an error.

$ cat /etc/puppet/hieradata/common.json
[
{
    "apache-packages" : [
    "apache2",
    "apache2-common",
    "apache2-utils"
    ],

     "hosts_entry" : "sandbox.%{fqdn}",

     "sshd_settings" : {
                        "root_allowed" : "no",
                        "password_allowed" : "no"
                      }
}
]

Verify valid JSON
$ cat /etc/puppet/hieradata/common.json | jq '.[0]'
{
   "sshd_settings": {
     "password_allowed": "no",
     "root_allowed": "no"
   },
   "hosts_entry": "sandbox.%{fqdn}",
   "apache-packages": [
     "apache2",
     "apache2-common",
     "apache2-utils"
   ]
}

Awesome! Lets validate in Hiera:
$ hiera --version
1.3.4
$ hiera -c /etc/puppet/hiera.yaml hosts_entry
/usr/lib/ruby/site_ruby/1.8/hiera/filecache.rb:56:in `read_file': Data
retrieved from /etc/puppet/hieradata/common.json is String not Hash
(TypeError)
from /usr/lib/ruby/site_ruby/1.8/hiera/backend/json_backend.rb:24:in
`lookup'
from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:76:in `datasources'
from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:74:in `map'
from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:74:in `datasources'
from /usr/lib/ruby/site_ruby/1.8/hiera/backend/json_backend.rb:17:in
`lookup'
from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:206:in `lookup'
from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:203:in `each'
from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:203:in `lookup'
from /usr/lib/ruby/site_ruby/1.8/hiera.rb:60:in `lookup'
from /usr/bin/hiera:225



And here is where things went wrong. The worst part is that it is such a
generic error I can't find anything of real value in my online searching.
Even worse, the only directly relevant hit I found basically said to stop
using JSON and use YAML instead. Not really a solution is it?

But whatever. It is probably good to verify that YAML is at least working
with Heira in the dev environment. Using the puppet documentation again for
guidance.
$ mv /etc/puppet/hieradata/common.json /etc/puppet/hieradata/common.yaml
$ vim /etc/puppet/hieradata/common.yaml
$ cat /etc/puppet/hieradata/common.yaml
---
# array
apache-packages:
     - apache2
     - apache2-common
     - apache2-utils

# string
apache-service: apache2

# interpolated facter variable
hosts_entry: "sandbox.%{fqdn}"

# hash
sshd_settings:
     root_allowed: "no"
     password_allowed: "yes"

# alternate hash notation
sshd_settings: {root_allowed: "no", password_allowed: "yes"}

# to return "true" or "false"
sshd_settings: {root_allowed: no, password_allowed: yes}


$ vim /etc/puppet/hiera.yaml
$ cat /etc/puppet/hiera.yaml
---
:backends:
     - yaml
:hierarchy:
     - common
:merge_behavior: "deeper"
:yaml:
     :datadir: /etc/puppet/hieradata


$ hiera -c /etc/puppet/hiera.yaml hosts_entry
sandbox.

Awesome! So YAML works.

Back to the JSON version....

$ hiera -c /etc/puppet/hiera.yaml hosts_entry
/usr/lib/ruby/site_ruby/1.8/hiera/filecache.rb:56:in `read_file': Data
retrieved from /etc/puppet/hieradata/common.json is String not Hash
(TypeError)
from /usr/lib/ruby/site_ruby/1.8/hiera/backend/json_backend.rb:24:in
`lookup'
from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:76:in `datasources'
from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:74:in `map'
from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:74:in `datasources'
from /usr/lib/ruby/site_ruby/1.8/hiera/backend/json_backend.rb:17:in
`lookup'
from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:206:in `lookup'
from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:203:in `each'
from /usr/lib/ruby/site_ruby/1.8/hiera/backend.rb:203:in `lookup'
from /usr/lib/ruby/site_ruby/1.8/hiera.rb:60:in `lookup'
from /usr/bin/hiera:225


Dah!

No matter how simple I make it or what tweaks I do to the common.json file,
I always get that exact error. If I run puppet agent, I get a very similar
error too!

"So just use YAML like the other post said"
1) That doesn't fix JSON and it avoids the problem ("Hey doc, my legs are
broken!" "Have you tried walking on your hands instead? Why not just do
that?" :-P)

2) The fancy new tool that my security team is using to manage and monitor
system wide variables reads/writes JSON but doesn't do YAML. Meaning we
either have to keep doing the manual checks we are doing, or stick a parser
in between. Why? Puppet and Heira are supposed to work with JSON, it
read/writes JSON, this should theoretically just work.

Does anyone know why hiera isn't working with JSON? I really feel like this
is probably a simple solution that I am just not seeing. I am hoping
someone else might be able to recognize the answer.

Thanks!
~Stack~