FAQ
Hey all,

Having kind of a weird problem here. Probably due to me not fully
understanding how Go hands memory and pointers around but here goes.
  I have a recursive function (below) that is supposed to add things to an
array inside a struct if they are not already there. This works fine if I
hardcode values in, i.e. every recursion add a new Page struct with fields
incremented from the previous one. However what happens when I actually run
the code below is that it works for a little while, and then elements
randomly start disappearing from the site.Pages array. There also seems to
be random failures to actually append to the array. For the record this is
only recursion; no goroutines nor is the struct/array in question being
written to anywhere else. Any chance someone could point out where I've
goof'd?

func (site Site) Spider(link Link) {

if strings.HasPrefix(link.Url, site.Domain) && !site.vistedLink(link) {
fmt.Println("...")
fmt.Printf("%d\n", len(site.Pages))
fmt.Println("link: " + link.Url)
                 //print list of pages
site.PrintV()
                 //goquery->get page->get array of links
doc := GetPage(link.Url)
links := site.GetLinks(doc)

site.Pages = append(site.Pages, Page{link.Url, links})

for _, l := range links {
site.Spider(l)
}
}

}


Actual output (I've sensored the URL but otherwise this is accurate)
...
11
link: http://domain.com/100_0970.jpg
http://domain.com
http://domain.com/domain972013_002.htm
http://domain.com/domain972013_005.htm
http://domain.com/domain972013_004.htm
http://domain.com/index.html
http://domain.com/domain972013_006.htm
http://domain.com/domain972013_007.htm
http://domain.com/domain972013_008.htm
http://domain.com/dsc_0001.jpg
http://domain.com/
http://domain.com/domain972013_003.htm
...
11
link: http://domain.com/100_1076.jpg
http://domain.com
http://domain.com/domain972013_002.htm
http://domain.com/domain972013_005.htm
http://domain.com/domain972013_004.htm
http://domain.com/index.html
http://domain.com/domain972013_006.htm
http://domain.com/domain972013_007.htm
http://domain.com/domain972013_008.htm
http://domain.com/dsc_0001.jpg
http://domain.com/
http://domain.com/domain972013_003.htm
...
10
link: http://domain.com/domain972013_009.htm
http://domain.com
http://domain.com/domain972013_002.htm
http://domain.com/domain972013_005.htm
http://domain.com/domain972013_004.htm
http://domain.com/index.html
http://domain.com/domain972013_006.htm
http://domain.com/domain972013_007.htm
http://domain.com/domain972013_008.htm
http://domain.com/dsc_0001.jpg
http://domain.com/
...
10
link: http://domain.com/mailto:domain@suddenlink.net
http://domain.com
http://domain.com/domain972013_002.htm
http://domain.com/domain972013_005.htm
http://domain.com/domain972013_004.htm
http://domain.com/index.html
http://domain.com/domain972013_006.htm
http://domain.com/domain972013_007.htm
http://domain.com/domain972013_008.htm
http://domain.com/dsc_0001.jpg
http://domain.com/
...
8
link: http://domain.com/dsc_0052.jpg
http://domain.com
http://domain.com/domain972013_002.htm
http://domain.com/domain972013_005.htm
http://domain.com/domain972013_004.htm
http://domain.com/index.html
http://domain.com/domain972013_006.htm
http://domain.com/domain972013_007.htm
http://domain.com/domain972013_008.htm


--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedDec 11, '14 at 1:06a
activeDec 11, '14 at 3:54p
posts3
users2
websitegolang.org

2 users in discussion

Turtlebait13: 2 posts Jesse McNelis: 1 post

People

Translate

site design / logo © 2022 Grokbase