FAQ
Hi,

Here is my situation:
I'm using the command line, as in, I'm not starting gnome or kde (I'm on
linux.)
I have a string of text attached to a variable,. So I need to use one of
the browsers on linux, that run under the command line, eg. lynx,
elinks, links, links2 and do the following.
1. Open a certain web page and find the first text box on the page, and
put this text into the form.
2. Press the submit button, and wait for the result page to load.
3. Click on the 15th link down the page.

So, how do I do this, can anyone point me to some docs or modules that
may help out here?

Thank you,

Search Discussions

  • Diez B. Roggisch at Nov 19, 2009 at 8:43 am

    Daniel Dalton schrieb:
    Hi,

    Here is my situation:
    I'm using the command line, as in, I'm not starting gnome or kde (I'm on
    linux.)
    I have a string of text attached to a variable,. So I need to use one of
    the browsers on linux, that run under the command line, eg. lynx,
    elinks, links, links2 and do the following.
    1. Open a certain web page and find the first text box on the page, and
    put this text into the form.
    2. Press the submit button, and wait for the result page to load.
    3. Click on the 15th link down the page.

    So, how do I do this, can anyone point me to some docs or modules that
    may help out here?
    Use the module "mechanize". It will mimic a browser, and allows you to
    navigate to pages, fill in forms & submit them.

    And with lxml or BeatifulSoup, you can extract the link you need to
    press, and then make mechanize visit it.

    Diez
  • Daniel Dalton at Nov 21, 2009 at 1:47 am

    On Thu, Nov 19, 2009 at 09:43:50AM +0100, Diez B. Roggisch wrote:
    elinks, links, links2 and do the following.
    1. Open a certain web page and find the first text box on the page, and
    put this text into the form.
    2. Press the submit button, and wait for the result page to load.
    3. Click on the 15th link down the page.

    So, how do I do this, can anyone point me to some docs or modules that
    may help out here?
    Use the module "mechanize". It will mimic a browser, and allows you
    to navigate to pages, fill in forms & submit them.
    Well, what seemed a very daunting task at first, proved to be quite
    easy, after a little bit of research. Thanks very much, I've written my
    script, and it works correctly, using the mechanize module.

    Thanks for everyone's help

    --
    Cheers,
    Dan

    http://members.iinet.net.au/~ddalton/

    -------------- next part --------------
    A non-text attachment was scrubbed...
    Name: not available
    Type: application/pgp-signature
    Size: 198 bytes
    Desc: Digital signature
    URL: <http://mail.python.org/pipermail/python-list/attachments/20091121/e0ceaf5f/attachment-0001.pgp>
  • Gerhard Häring at Nov 19, 2009 at 4:19 pm

    Daniel Dalton wrote:
    Hi,

    Here is my situation:
    I'm using the command line, as in, I'm not starting gnome or kde (I'm on
    linux.)
    I have a string of text attached to a variable,. So I need to use one of
    the browsers on linux, that run under the command line, eg. lynx,
    elinks, links, links2 and do the following.
    No, you don't need any text mode browser. It's much easier to do with
    modules from the Python standard library.
    1. Open a certain web page and find the first text box on the page, and
    put this text into the form.
    I assume you know HTML and HTTP.

    You can use urllib or urllib2 to submit the form.
    2. Press the submit button, and wait for the result page to load.
    3. Click on the 15th link down the page.
    You will then get the returned HTML from a function and you will have to
    parse it for the link you're interested in. There are many approaches to
    get there. Manual parsing, regular expressions or one of the XML parsers
    in the standard library (etree is the easiest).
    So, how do I do this, can anyone point me to some docs or modules that
    may help out here?
    While all of this may seem overwhelming at first, I guess the solution
    can be written in 20 - 30 lines of code.

    -- Gerhard

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppython-list @
categoriespython
postedNov 19, '09 at 7:10a
activeNov 21, '09 at 1:47a
posts4
users3
websitepython.org

People

Translate

site design / logo © 2022 Grokbase