FAQ
Hi!

I am having problems with state.highstate calls via API that are not being
executed.

I am using the REST API via rest_cherrypy with Salt 2014.7.4.

After setting up a new host, I do the following:

- Wait for the minion id to appear in minions_pre.
- If that happens, I accept the minion. That always works.
- After that, I verify the minion by issuing "grains.items" calls until I
get a non-empty result (IMHO, this should not be necessary, but the first
one or two attempts I just get an empty result.)
- Then I issue the "state.highstate" call via /minions, and get a jobid
- I then periodically check for the job result. I always get something like

{
     "info": [{
         "Function": "state.highstate",
         "jid": "20150410153131993258",
         "Target": "myminion",
         "Target-type": "glob",
         "User": "testuser",
         "StartTime": "2015, Apr 10 15:31:31.993258",
         "Arguments": [],
         "Minions": ["myminion"],
         "Result": {

         }
     }],
     "return": [{

     }]
}

- At this point, nothing happens, even after 30 minutes. No states are
being executed on the minion, the job status does not change.

The master clearly receives the job request:

2015-04-10 16:17:33,235 [salt.master
][INFO ] User cdsdev Published command state.highstate with jid
20150410161733234805
2015-04-10 16:17:33,236 [salt.master
][DEBUG ] Published command details {'tgt_type': 'glob', 'jid':
'20150410161733234805', 'tgt': 'myminion', 'ret': '', 'user': 'testuser',
'arg': [], 'fun': 'state.highstate'}
2015-04-10 16:17:33,271 [cherrypy.access.43528400 ][INFO
] 10.14.6.194 - - [10/Apr/2015:16:17:33] "POST /minions HTTP/1.1" 202 163
"" "Apache-HttpClient/4.4 (Java 1.5 minimum; Java/1.8.0_40)"

and does some stuff (ext_node and ext_pillars are called...), but nothing
happens on the minion.

How can I decide whether the state.highstate job failed (which for me
includes "finished with no result") or is still running? If I could decide
whether the state.highstate job is actually running, I could at least retry
it until it succeeds...

If I later run state.highstate from the master command line, it usually
works.

Thanks

Martin

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Colton Myers at Apr 21, 2015 at 9:25 pm
    Have you considered using the jobs.active runner to see what jobs are still
    running?

    --
    Colton Myers
    Platform Engineer, SaltStack
    @basepi on Twitter/Github/IRC
    On Fri, Apr 10, 2015 at 8:59 AM, Martin Emrich wrote:

    Hi!

    I am having problems with state.highstate calls via API that are not being
    executed.

    I am using the REST API via rest_cherrypy with Salt 2014.7.4.

    After setting up a new host, I do the following:

    - Wait for the minion id to appear in minions_pre.
    - If that happens, I accept the minion. That always works.
    - After that, I verify the minion by issuing "grains.items" calls until I
    get a non-empty result (IMHO, this should not be necessary, but the first
    one or two attempts I just get an empty result.)
    - Then I issue the "state.highstate" call via /minions, and get a jobid
    - I then periodically check for the job result. I always get something like

    {
    "info": [{
    "Function": "state.highstate",
    "jid": "20150410153131993258",
    "Target": "myminion",
    "Target-type": "glob",
    "User": "testuser",
    "StartTime": "2015, Apr 10 15:31:31.993258",
    "Arguments": [],
    "Minions": ["myminion"],
    "Result": {

    }
    }],
    "return": [{

    }]
    }

    - At this point, nothing happens, even after 30 minutes. No states are
    being executed on the minion, the job status does not change.

    The master clearly receives the job request:

    2015-04-10 16:17:33,235 [salt.master
    ][INFO ] User cdsdev Published command state.highstate with jid
    20150410161733234805
    2015-04-10 16:17:33,236 [salt.master
    ][DEBUG ] Published command details {'tgt_type': 'glob', 'jid':
    '20150410161733234805', 'tgt': 'myminion', 'ret': '', 'user': 'testuser',
    'arg': [], 'fun': 'state.highstate'}
    2015-04-10 16:17:33,271 [cherrypy.access.43528400
    ][INFO ] 10.14.6.194 - - [10/Apr/2015:16:17:33] "POST /minions HTTP/1.1"
    202 163 "" "Apache-HttpClient/4.4 (Java 1.5 minimum; Java/1.8.0_40)"

    and does some stuff (ext_node and ext_pillars are called...), but nothing
    happens on the minion.

    How can I decide whether the state.highstate job failed (which for me
    includes "finished with no result") or is still running? If I could decide
    whether the state.highstate job is actually running, I could at least retry
    it until it succeeds...

    If I later run state.highstate from the master command line, it usually
    works.

    Thanks

    Martin

    --
    You received this message because you are subscribed to the Google Groups
    "Salt-users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to salt-users+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "Salt-users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Martin Emrich at May 6, 2015 at 9:56 am
    Hi!

    I actually considered it, but I think it would be more complicated
    (filtering out the job I am interested in from possibly many active jobs).

    In the meantime, I "fixed" it by running state.highstate twice: first with
    a very short timeout (15s), this triggered "something". Then I issued a
    second highstate call (with my intended long timeout) which actually got
    executed.
    This worked reliably for about a week.

    Then the problem solved itself; I now can call state.highstate once and it
    runs mostly correct (I still have another issue, but I'll open a bug for
    it).

    Strange though, and it did not fix itself with 2014.7.5 (which I installed
    only today).

    Ciao

    Martin

    Am Dienstag, 21. April 2015 23:25:57 UTC+2 schrieb basepi:
    Have you considered using the jobs.active runner to see what jobs are
    still running?
    --
    You received this message because you are subscribed to the Google Groups "Salt-users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Colton Myers at May 12, 2015 at 5:27 pm
    This is definitely a strange one. You might consider trying the new
    2015.5.0 release, and if the problem doesn't go away we should probably
    open an issue on Github to continue to debug and follow up.

    --
    Colton Myers
    Platform Engineer, SaltStack
    @basepi on Twitter/Github/IRC
    On Wed, May 6, 2015 at 3:56 AM, Martin Emrich wrote:

    Hi!

    I actually considered it, but I think it would be more complicated
    (filtering out the job I am interested in from possibly many active jobs).

    In the meantime, I "fixed" it by running state.highstate twice: first with
    a very short timeout (15s), this triggered "something". Then I issued a
    second highstate call (with my intended long timeout) which actually got
    executed.
    This worked reliably for about a week.

    Then the problem solved itself; I now can call state.highstate once and it
    runs mostly correct (I still have another issue, but I'll open a bug for
    it).

    Strange though, and it did not fix itself with 2014.7.5 (which I installed
    only today).

    Ciao

    Martin

    Am Dienstag, 21. April 2015 23:25:57 UTC+2 schrieb basepi:
    Have you considered using the jobs.active runner to see what jobs are
    still running?

    --
    You received this message because you are subscribed to the Google Groups
    "Salt-users" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to salt-users+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "Salt-users" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupsalt-users @
postedApr 10, '15 at 2:59p
activeMay 12, '15 at 5:27p
posts4
users2

2 users in discussion

Martin Emrich: 2 posts Colton Myers: 2 posts

People

Translate

site design / logo © 2022 Grokbase