On Fri, May 1, 2015 at 4:44 PM, wrote:
pprof --alloc_space reports around 370MB for one function:
This isn't all directly related to your memory question, but a few
style notes on using the html Parser:

First, the middle line in:

var err error
n := new(html.Node)
n, err = html.Parse(r)

is unnecessary, and an unnecessary memory allocation: you're assigning
a newly allocated Node to n, but then immediately overwriting it on
the next line. These three lines can be just:

n, err := htm.Parse(r)

with the colon-equals instead of the bare equals.


s := strings.ToLower(n.Data)
if s == "script" {
   script = 1
} etc

could probably be

if n.DataAtom == atom.Script {
   script = 1
} etc

after you import "golang.org/x/net/html/atom". The atom comparison
should be faster (comparing ints instead of strings is O(1) instead of
worst-case O(length of string)), but more importantly, it allocates
fewer temporary strings. I don't have your full source code, but I
suspect you are calling strings.ToLower on your text nodes as well as
your element nodes, which is needless allocation of garbage that needs
collecting, and I think it will be a false positive when matching the
text node inside "<b>script</b>".

Third, the "<!--" check in

if n.Type == html.TextNode {
   tmp := strings.TrimSpace(n.Data)
   if len(tmp) > 3 && tmp[0:4] == "<!--" {

should be unnecessary. The HTML parser already recognizes HTML
comments, and returns those as Comment nodes and not Text nodes.

I thought garbage collection should take care of it. But seems not.
Please correct me if I'm wrong here.
It's been a while since I've used pprof, but I believe that
--alloc_space is the total allocations, and garbage-collecting
previous allocations doesn't lower this number: if you allocate 100MB
and then GC 100MB of no-longer-used heap, alloc_space will still
report 100MB, and the alloc_space number only grows upwards over time.
Looking at --inuse_space may be a more appropriate metric.

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

Discussion Posts


Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 3 of 5 | next ›
Discussion Overview
groupgolang-nuts @
postedMay 1, '15 at 12:37p
activeMay 4, '15 at 7:31a



site design / logo © 2021 Grokbase