[nodejs] Batch jobs in Node

On Fri, Oct 5, 2012 at 11:08 AM, Guilherme Pim wrote:

Hi guys,

Having the need to synchronize some data with external sources, I have
written some tasks on Grunt to do so. I wonder if Grunt is actually the
tool to write batch jobs in, or there actually is some other alternative.

I have come across Node.io, but I rather stick to Grunt in this case. Do
you guys know about some alternatives, please?

Thanks!
Guilherme

--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
nodejs+[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Can you share some more details on what you are trying to accomplish?

You have talked about tools (both of which I am unaware of) and said
"batch", but apart from that there wasn't much more that could help anyone
make an informed suggestion.

Let me know, I'll try to help :)
Nuno
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
nodejs+[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Okay, my bad!

I am crawling some web pages to populate a database, and currently doing
that with Grunt (http://gruntjs.com/), which is a task-based 'build tool'
(that gets me confused, is it supposed to solely build?).

So, I have created some scripts in the following manner:

========== Task file ==========

module.exports = function (grunt) {

grunt.registerTask('courses/update-info', 'Updates all courses info on
the database', ... )

}

============================

So then I may have a somewhat controlled access to these jobs from the
terminal, with ability to list all existing jobs and their descriptions.
Also, I may pass some parameters to each task, although it's not very
readable because they can't be associated with a name:

grunt courses/update-info:4:5:2 (these will be passed as the tasks's
function arguments)

I guess I could go about creating node scripts and using a parameters
library along with console colors and stuff, but I would rather find a
framework, if such exists.

Did I make myself clearer at all? haha

Thanks for you time,
Guilherme

2012/10/5 Nuno Job <[email protected] <javascript:>>

On Oct 5, 2012, at 4:22 AM, Guilherme Pim wrote:

Okay, my bad!

I am crawling some web pages to populate a database, and currently doing that with Grunt (http://gruntjs.com/), which is a task-based 'build tool' (that gets me confused, is it supposed to solely build?).

So, I have created some scripts in the following manner:

========== Task file ==========

module.exports = function (grunt) {

grunt.registerTask('courses/update-info', 'Updates all courses info on the database', ... )

}

============================

So then I may have a somewhat controlled access to these jobs from the terminal, with ability to list all existing jobs and their descriptions. Also, I may pass some parameters to each task, although it's not very readable because they can't be associated with a name:

grunt courses/update-info:4:5:2 (these will be passed as the tasks's function arguments)

I guess I could go about creating node scripts and using a parameters library along with console colors and stuff, but I would rather find a framework, if such exists.

Did I make myself clearer at all? haha

Thanks for you time,
Guilherme

2012/10/5 Nuno Job <[email protected]>
Can you share some more details on what you are trying to accomplish?

You have talked about tools (both of which I am unaware of) and said "batch", but apart from that there wasn't much more that could help anyone make an informed suggestion.

Let me know, I'll try to help :)
Nuno

On Fri, Oct 5, 2012 at 11:08 AM, Guilherme Pim wrote:
Hi guys,

Having the need to synchronize some data with external sources, I have written some tasks on Grunt to do so. I wonder if Grunt is actually the tool to write batch jobs in, or there actually is some other alternative.

I have come across Node.io, but I rather stick to Grunt in this case. Do you guys know about some alternatives, please?

Thanks!
Guilherme

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
nodejs+[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en


--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
nodejs+[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en


--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
nodejs+[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

On Oct 11, 2012, at 1:52 PM, Ted Young wrote:

Sounds like you just want a node service that does the scraping, and provides an http api for adding tasks and querying what tasks are currently running. Then have a separate command-line app that can talk to the web service. And maybe also an express app that can talk to it via a browser. I use that pattern a lot, but haven't yet written an abstraction for it. An abstraction for easily banging out command-line apps with tab-complete, etc that just talk to web services would be a useful thingie though.

I'm not sure if there's a generic node-based job-processor out there, but you might like https://github.com/defunkt/resque. It's ruby, but each job could just be a system call to start a node process.

Ted

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
nodejs+[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en