Hi guys,

I use Nokogiri to fetch some HTML pages and scrape it in the background.
So far it works fine. But now I have to login to view some other pages
in the background. How can I login in the background to fetch some HTML?
In other words how can I login in the background using Rails?

Note that I'm not talking about some login form or something.

--
Posted via http://www.ruby-forum.com/.

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To post to this group, send email to rubyonrails-talk@googlegroups.com.
To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.

Search Discussions

  • Everaldo Gomes at Nov 14, 2011 at 4:07 pm
    Hi!

    I don't know if I understood you well, but take a lookt at mechanize:

    http://railscasts.com/episodes/191-mechanize

    Regards,
    Everaldo
    On Mon, Nov 14, 2011 at 2:01 PM, Jeroen van Ingen wrote:

    Hi guys,

    I use Nokogiri to fetch some HTML pages and scrape it in the background.
    So far it works fine. But now I have to login to view some other pages
    in the background. How can I login in the background to fetch some HTML?
    In other words how can I login in the background using Rails?

    Note that I'm not talking about some login form or something.

    --
    Posted via http://www.ruby-forum.com/.

    --
    You received this message because you are subscribed to the Google Groups
    "Ruby on Rails: Talk" group.
    To post to this group, send email to rubyonrails-talk@googlegroups.com.
    To unsubscribe from this group, send email to
    rubyonrails-talk+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/rubyonrails-talk?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
    To post to this group, send email to rubyonrails-talk@googlegroups.com.
    To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
  • Jeroen van Ingen at Nov 14, 2011 at 4:34 pm
    Mechanize looks good, I'll definitely give it a try.

    Where I'm basically searching for is a gem or library that can store a
    session / cookie, so Rails can login on a website in the background.

    Anyone who know if there are other options besides mechanize?

    --
    Posted via http://www.ruby-forum.com/.

    --
    You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
    To post to this group, send email to rubyonrails-talk@googlegroups.com.
    To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
  • Fearless Fool at Nov 15, 2011 at 1:26 pm

    Jeroen van Ingen wrote in post #1031839:
    Mechanize looks good, I'll definitely give it a try.

    Where I'm basically searching for is a gem or library that can store a
    session / cookie, so Rails can login on a website in the background.

    Anyone who know if there are other options besides mechanize?
    Hi Jeroen:

    It seems you're talking about two separate things:
    (a) a mechanism to log in and scrape pages
    (b) a mechanism to run a task in the background

    For (a) I use Mechanize for logging in and page scraping -- that's what
    it's designed for. For (b), I use DelayedJob. I've had no problem
    running a Mechanize task in the background using DelayedJob.

    - ff

    --
    Posted via http://www.ruby-forum.com/.

    --
    You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
    To post to this group, send email to rubyonrails-talk@googlegroups.com.
    To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouprubyonrails-talk @
categoriesrubyonrails
postedNov 14, '11 at 4:01p
activeNov 15, '11 at 1:26p
posts4
users2
websiterubyonrails.org
irc#RubyOnRails

2 users in discussion

Fearless Fool: 3 posts Everaldo Gomes: 1 post

People

Translate

site design / logo © 2021 Grokbase