I am trying to crawl BBC Hindi site "http://www.bbc.co.uk/hindi/ "
but after depth 1 it shows, stopping at depth-1, no more urls to fetch.
Looking at the dump for depth-1, I realised there is no content fetched from
the page, could any one help me to figure out the root cause of the problem,
why it's not fetching any content from the page?
Had any one tried to crawl the site http://www.bbc.co.uk/hindi/ ??
thanks in advance
--
Ankur Garg
अँकुर गर्ग
