FAQ
Greetings,


What's the easy way to calculate a webpage's downloading time?
(not only the page, but all the elements in this page, like images,
JS, css etc).
For example, I want to get a function:

my $dl_time = fetch_a_page("www.yahoo.com");


Thanks in advance.

Regards.

Search Discussions

  • Jayesh Thakrar at Apr 15, 2009 at 2:00 pm
    Just a thought -

    A web page (say www.yahoo.com) may contain several links, some of which could be non-traditional html (e.g. media files, Flash files, etc). Do you want "fetch_a_page" to just read the URI of the hyperlink or even "consume" (parse, read, etc.) the URI?

    Also, since reading the links serially is not "fast enought", browsers often read/parse/download several links in parallel.

    -- Jayesh



    ----- Original Message ----
    From: "practicalperl@gmail.com" <practicalperl@gmail.com>
    To: Perl Beginners <beginners@perl.org>
    Sent: Wednesday, April 15, 2009 8:38:33 AM
    Subject: calc page's downloading time

    Greetings,


    What's the easy way to calculate a webpage's downloading time?
    (not only the page, but all the elements in this page, like images,
    JS, css etc).
    For example, I want to get a function:

    my $dl_time = fetch_a_page("www.yahoo.com");


    Thanks in advance.

    Regards.

    --
    To unsubscribe, e-mail: beginners-unsubscribe@perl.org
    For additional commands, e-mail: beginners-help@perl.org
    http://learn.perl.org/
  • Chas. Owens at Apr 15, 2009 at 2:02 pm

    On Wed, Apr 15, 2009 at 09:38, wrote:
    Greetings,


    What's the easy way to calculate a webpage's downloading time?
    (not only the page, but all the elements in this page, like images,
    JS, css etc).
    For example, I want to get a function:

    my $dl_time = fetch_a_page("www.yahoo.com");
    snip

    A page's download time will be different from different machines
    on the net. You are better off figuring out how much data needs
    to be downloaded (the sum of all CSS, HTML, images, Javascript,
    etc. files). If some of this data is can be cached (such as an
    external CSS file that all of the HTML uses) you will want to
    create two sizes: first fetch, and subsequent fetches. Once you
    have the total size you can calculate the download times for a
    variety of download speeds.

    If this is someone else's page you can probably fetch each the
    page with LWP::Simple[1], then parse it with HTML::Parser[2],
    and then download each of the items that would normally be
    downloaded (like external CSS and JavaScript files and images).

    Get the file sizes and add them all up.

    1. http://search.cpan.org/dist/libwww-perl/lib/LWP/Simple.pm
    2. http://search.cpan.org/dist/HTML-Parser/Parser.pm

    --
    Chas. Owens
    wonkden.net
    The most important skill a programmer can have is the ability to read.
  • Jay Savage at Apr 15, 2009 at 2:33 pm

    On Wed, Apr 15, 2009 at 10:02 AM, Chas. Owens wrote:
    On Wed, Apr 15, 2009 at 09:38,  wrote:
    Greetings,


    What's the easy way to calculate a webpage's downloading time?
    (not only the page, but all the elements in this page, like images,
    JS, css etc).
    For example, I want to get a function:

    my $dl_time = fetch_a_page("www.yahoo.com");
    snip

    A page's download time will be different from different machines
    on the net.  You are better off figuring out how much data needs
    to be downloaded (the sum of all CSS, HTML, images, Javascript,
    etc. files).  If some of this data is can be cached (such as an
    external CSS file that all of the HTML uses) you will want to
    create two sizes: first fetch, and subsequent fetches. Once you
    have the total size you can calculate the download times for a
    variety of download speeds.

    If this is someone else's page you can probably fetch each the
    page with LWP::Simple[1], then parse it with HTML::Parser[2],
    and then download each of the items that would normally be
    downloaded (like external CSS and JavaScript files and images).

    Get the file sizes and add them all up.

    1. http://search.cpan.org/dist/libwww-perl/lib/LWP/Simple.pm
    2. http://search.cpan.org/dist/HTML-Parser/Parser.pm
    MWWW::Mechanzie
    (http://search.cpan.org/dist/WWW-Mechanize/lib/WWW/Mechanize.pm) was
    designed to do just this. There's no need to reinvent the wheel by
    trying to cobble something out of LWP::Simple and HTML::Parser
    yourself.

    The real difficulty, as I see it, is defining what you men by "download time."

    Raw transfer time is simple to calculate, as Chas. has laid out: (KB *
    8)/Kbps = s, where 'KB' is download size in kilobytes, 'Kbps' is
    download speed in kilobits/second, and 's' is seconds. If all you want
    it raw transfer times, the 'ab' (Apache Benchmark) tool that ships
    with Apache is useful, too. It's called Apache Benchmark, but it will
    happily flood just about any web server. It's also built on LWP, so
    taking a look at the code can be useful, even if you decide to go your
    own way.

    Most of the time, though, when people talk about the download speed of
    a web page, they're really talking about page load time. That is, how
    long it takes a page to show up in an end user's browser. Page load
    times are only loosely related to actual download speeds. Once you
    start talking about javascript, flash, etc.--even images in some
    cases--the time it takes to execute and display the scripts is
    normally several orders of magnitude longer than the time it takes to
    actually transfer them.

    HTH,

    -- jay
    --------------------------------------------------
    This email and attachment(s): [ ] blogable; [ x ] ask first; [ ]
    private and confidential

    daggerquill [at] gmail [dot] com
    http://www.tuaw.com http://www.downloadsquad.com http://www.engatiki.org

    values of β will give rise to dom!
  • Practicalperl at Apr 16, 2009 at 12:23 am
    Thanks Jay, Chas and all others.Will try WWW:Mechanzie.

    2009/4/15 Jay Savage <daggerquill@gmail.com>:
    MWWW::Mechanzie
    (http://search.cpan.org/dist/WWW-Mechanize/lib/WWW/Mechanize.pm) was
    designed to do just this. There's no need to reinvent the wheel by
    trying to cobble something out of LWP::Simple and HTML::Parser
    yourself.

    The real difficulty, as I see it, is defining what you men by "download time."

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupbeginners @
categoriesperl
postedApr 15, '09 at 1:38p
activeApr 16, '09 at 12:23a
posts5
users4
websiteperl.org

People

Translate

site design / logo © 2022 Grokbase