FAQ
Dear Group,
I am trying to construct a web based crawler with Python and for that
I am using the URLLIB module, and by doing
import urllib and then trying with urllib.urlopen("url).
Am I going fine?
If some one can kindly highlight if I am doing any mistake.
Best Regards,
Subhabrata Banerjee.

Search Discussions

  • Santoso Wijaya at Mar 15, 2011 at 6:33 am
    Well, have you run into any problem, expected or otherwise, yet?

    ~/santa

    On Mon, Mar 14, 2011 at 11:10 PM, joy99 wrote:

    Dear Group,
    I am trying to construct a web based crawler with Python and for that
    I am using the URLLIB module, and by doing
    import urllib and then trying with urllib.urlopen("url).
    Am I going fine?
    If some one can kindly highlight if I am doing any mistake.
    Best Regards,
    Subhabrata Banerjee.
    --
    http://mail.python.org/mailman/listinfo/python-list
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://mail.python.org/pipermail/python-list/attachments/20110314/87469b63/attachment-0001.html>
  • David Marek at Mar 15, 2011 at 8:49 am
    You are doing fine so far :-)

    However, urllib is quite low level module. Do you know mechanize
    http://wwwsearch.sourceforge.net/mechanize/ ? I have used it in my
    crawler. If you want to crawl a specific site for known data then
    Scrapy http://scrapy.org/ could be useful too.

    David Marek
    davidm at atrey.karlin.mff.cuni.cz
    http://davidmarek.cz
    On Tue, Mar 15, 2011 at 7:10 AM, joy99 wrote:
    Dear Group,
    I am trying to construct a web based crawler with Python and for that
    I am using the URLLIB module, and ?by doing
    import urllib and then trying with urllib.urlopen("url).
    Am I going fine?
    If some one can kindly highlight if I am doing any mistake.
    Best Regards,
    Subhabrata Banerjee.
    --
    http://mail.python.org/mailman/listinfo/python-list
  • Joy99 at Mar 15, 2011 at 9:31 am

    On Mar 15, 1:49?pm, David Marek wrote:
    You are doing fine so far :-)

    However, urllib is quite low level module. Do you know mechanizehttp://wwwsearch.sourceforge.net/mechanize/? I have used it in my
    crawler. If you want to crawl a specific site for known data then
    Scrapyhttp://scrapy.org/could be useful too.

    David Marek
    dav... at atrey.karlin.mff.cuni.czhttp://davidmarek.cz
    On Tue, Mar 15, 2011 at 7:10 AM, joy99 wrote:
    Dear Group,
    I am trying to construct a web based crawler with Python and for that
    I am using the URLLIB module, and ?by doing
    import urllib and then trying with urllib.urlopen("url).
    Am I going fine?
    If some one can kindly highlight if I am doing any mistake.
    Best Regards,
    Subhabrata Banerjee.
    --
    http://mail.python.org/mailman/listinfo/python-list
    Thanks David,I'll surely try to have a check.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedMar 15, '11 at 6:10a
activeMar 15, '11 at 9:31a
posts4
users3
websitepython.org

People

Translate

site design / logo © 2022 Grokbase