FAQ
Hello all, I have a very simple topology that I'm running that takes in a
tweet from the 1% sample and stores it in a mysql database. My problem
comes when I allow the application to run overnight it runs out of heap
space. No matter how much heap I allocate using the -Xmx argument it fills
up after running for about 24 hours or longer (I've tried up to 16GB
allocated to it). I use the code for the twitter sample spout that's
included in the storm sample code and for the time being I'm running the
topology from Eclipse until I get the bugs worked out of it. For the MySQL
bolt I have the following code:

public void execute(Tuple input, BasicOutputCollector collector) {

try {

con = connector.getConnection(sqlDBUrl, sqlUser, sqlPassword);

} catch (ClassNotFoundException e) {

e.printStackTrace();

} catch (SQLException e) {

e.printStackTrace();

} finally {

//System.out.print("Connection created.");

}

PreparedStatement pst = null;

//parse out the Status object from the first tuple

Status s = (Status) input.getValue(0);

try {

pst = con.prepareStatement("INSERT INTO " + db + " (tweet)" +

"VALUES (?);");


pst.setString(1, s.toString());


//execute the SQL

pst.executeUpdate();

} catch (SQLException e) {

// TODO Auto-generated catch block

e.printStackTrace();

} finally {

     if(pst != null) {try {pst.close();} catch (SQLException e) {e.
printStackTrace();}}

     if(con != null) {try {con.close();} catch (SQLException e) {e.
printStackTrace();}}

}
}


As you can see there doesn't seem to be anywhere for a memory leak. Can
anyone point me in the right direction for fixing this problem? I'm using
Storm 0.7.1 for the time being.

Search Discussions

  • Michael Rose at Jan 10, 2013 at 6:58 pm
    Have you added -XX:+HeapDumpOnOutOfMemoryError to your JVM options?

    Once it OOMs with that you'll have a heap dump to work with and run jhat over to explore your object graph.

    Alternately, you can attach a profiler like YourKit to your running topology and watch your object allocations.

    --
    Michael Rose (@Xorlev (https://twitter.com/xorlev))
    Senior Platform Engineer, FullContact (http://fullcontact.com/)
    michael@fullcontact.com (mailto:michael@fullcontact.com)

    On Thursday, January 10, 2013 at 11:55 AM, Chris Maness wrote:

    Hello all, I have a very simple topology that I'm running that takes in a tweet from the 1% sample and stores it in a mysql database. My problem comes when I allow the application to run overnight it runs out of heap space. No matter how much heap I allocate using the -Xmx argument it fills up after running for about 24 hours or longer (I've tried up to 16GB allocated to it). I use the code for the twitter sample spout that's included in the storm sample code and for the time being I'm running the topology from Eclipse until I get the bugs worked out of it. For the MySQL bolt I have the following code:
    public void execute(Tuple input, BasicOutputCollector collector) {
    try {
    con = connector.getConnection(sqlDBUrl, sqlUser, sqlPassword);
    } catch (ClassNotFoundException e) {
    e.printStackTrace();
    } catch (SQLException e) {
    e.printStackTrace();
    } finally {
    //System.out.print("Connection created.");
    }
    PreparedStatement pst = null;
    //parse out the Status object from the first tuple
    Status s = (Status) input.getValue(0);
    try {
    pst = con.prepareStatement("INSERT INTO " + db + " (tweet)" +
    "VALUES (?);");

    pst.setString(1, s.toString());

    //execute the SQL
    pst.executeUpdate();
    } catch (SQLException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
    } finally {
    if(pst != null) {try {pst.close();} catch (SQLException e) {e.printStackTrace();}}
    if(con != null) {try {con.close();} catch (SQLException e) {e.printStackTrace();}}
    }
    }
    As you can see there doesn't seem to be anywhere for a memory leak. Can anyone point me in the right direction for fixing this problem? I'm using Storm 0.7.1 for the time being.





  • Chris Maness at Jan 10, 2013 at 7:48 pm
    I didn't even realize that was an option for the JVM. Thanks. I'll do that
    and be back if I have any problems or need any help.
    On Thursday, January 10, 2013 1:58:04 PM UTC-5, Michael Rose wrote:

    Have you added -XX:+HeapDumpOnOutOfMemoryError to your JVM options?

    Once it OOMs with that you'll have a heap dump to work with and run jhat
    over to explore your object graph.

    Alternately, you can attach a profiler like YourKit to your running
    topology and watch your object allocations.

    --
    Michael Rose (@Xorlev <https://twitter.com/xorlev>)
    Senior Platform Engineer, FullContact <http://fullcontact.com/>
    mic...@fullcontact.com <javascript:>

    On Thursday, January 10, 2013 at 11:55 AM, Chris Maness wrote:

    Hello all, I have a very simple topology that I'm running that takes in a
    tweet from the 1% sample and stores it in a mysql database. My problem
    comes when I allow the application to run overnight it runs out of heap
    space. No matter how much heap I allocate using the -Xmx argument it fills
    up after running for about 24 hours or longer (I've tried up to 16GB
    allocated to it). I use the code for the twitter sample spout that's
    included in the storm sample code and for the time being I'm running the
    topology from Eclipse until I get the bugs worked out of it. For the MySQL
    bolt I have the following code:

    public void execute(Tuple input, BasicOutputCollector collector) {

    try {

    con = connector.getConnection(sqlDBUrl, sqlUser, sqlPassword);

    } catch (ClassNotFoundException e) {

    e.printStackTrace();

    } catch (SQLException e) {

    e.printStackTrace();

    } finally {

    //System.out.print("Connection created.");

    }

    PreparedStatement pst = null;

    //parse out the Status object from the first tuple

    Status s = (Status) input.getValue(0);

    try {

    pst = con.prepareStatement("INSERT INTO " + db + " (tweet)" +

    "VALUES (?);");


    pst.setString(1, s.toString());


    //execute the SQL

    pst.executeUpdate();

    } catch (SQLException e) {

    // TODO Auto-generated catch block

    e.printStackTrace();

    } finally {

    if(pst != null) {try {pst.close();} catch (SQLException e) {e.
    printStackTrace();}}

    if(con != null) {try {con.close();} catch (SQLException e) {e.
    printStackTrace();}}

    }
    }


    As you can see there doesn't seem to be anywhere for a memory leak. Can
    anyone point me in the right direction for fixing this problem? I'm using
    Storm 0.7.1 for the time being.

  • Vinh at Jan 10, 2013 at 7:57 pm
    Add to storm.yaml something like:

    worker.childopts: "-Xmx768m -XX:+HeapDumpOnOutOfMemoryError"

    For more info on what you can customize in storm.yaml:
    https://github.com/nathanmarz/storm/blob/master/conf/defaults.yaml

    On Jan 10, 2013, at 11:48 AM, Chris Maness wrote:

    I didn't even realize that was an option for the JVM. Thanks. I'll do that and be back if I have any problems or need any help.

    On Thursday, January 10, 2013 1:58:04 PM UTC-5, Michael Rose wrote:
    Have you added -XX:+HeapDumpOnOutOfMemoryError to your JVM options?

    Once it OOMs with that you'll have a heap dump to work with and run jhat over to explore your object graph.

    Alternately, you can attach a profiler like YourKit to your running topology and watch your object allocations.

    --
    Michael Rose (@Xorlev)
    Senior Platform Engineer, FullContact
    mic...@fullcontact.com
    On Thursday, January 10, 2013 at 11:55 AM, Chris Maness wrote:

    Hello all, I have a very simple topology that I'm running that takes in a tweet from the 1% sample and stores it in a mysql database. My problem comes when I allow the application to run overnight it runs out of heap space. No matter how much heap I allocate using the -Xmx argument it fills up after running for about 24 hours or longer (I've tried up to 16GB allocated to it). I use the code for the twitter sample spout that's included in the storm sample code and for the time being I'm running the topology from Eclipse until I get the bugs worked out of it. For the MySQL bolt I have the following code:

    public void execute(Tuple input, BasicOutputCollector collector) {try {con = connector.getConnection(sqlDBUrl, sqlUser, sqlPassword);} catch (ClassNotFoundException e) {e.printStackTrace();} catch (SQLException e) {e.printStackTrace();} finally {
    //System.out.print("Connection created.");

    }PreparedStatement pst = null;
    //parse out the Status object from the first tuple

    Status s = (Status) input.getValue(0);try {pst = con.prepareStatement("INSERT INTO " + db + " (tweet)" +"VALUES (?);");


    pst.setString(1, s.toString());


    //execute the SQL

    pst.executeUpdate();} catch (SQLException e) {// TODO Auto-generated catch blocke.printStackTrace();} finally { if(pst != null) {try {pst.close();} catch (SQLException e) {e.printStackTrace();}} if(con != null) {try {con.close();} catch (SQLException e) {e.printStackTrace();}}}}

    As you can see there doesn't seem to be anywhere for a memory leak. Can anyone point me in the right direction for fixing this problem? I'm using Storm 0.7.1 for the time being.



  • Chris Maness at Jan 14, 2013 at 3:41 pm
    So after poking around with the Eclipse Memory Analyzer I see that *
    java.util.concurrent.LinkedBlockingQueue* is the root of my problem. Why is
    this taking up so much room in memory? I'm guessing this is the queue that
    serves the tuples up to the next bolt.
  • William Sell at May 8, 2013 at 8:02 pm
    Same problem here-- how did you resolve this?
    On Monday, January 14, 2013 7:41:39 AM UTC-8, Chris Maness wrote:

    So after poking around with the Eclipse Memory Analyzer I see that *
    java.util.concurrent.LinkedBlockingQueue* is the root of my problem. Why
    is this taking up so much room in memory? I'm guessing this is the queue
    that serves the tuples up to the next bolt.
    --
    You received this message because you are subscribed to the Google Groups "storm-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.
  • Chris Maness at May 8, 2013 at 8:05 pm
    I stopped running it through Eclipse. There's something about running a
    topology through eclipse that messes up pieces of the configuration it
    seems. Make sure you run it through Storm or on a test cluster.

    On Wed, May 8, 2013 at 5:03 AM, William Sell wrote:

    Same problem here-- how did you resolve this?

    On Monday, January 14, 2013 7:41:39 AM UTC-8, Chris Maness wrote:

    So after poking around with the Eclipse Memory Analyzer I see that *
    java.util.concurrent.LinkedBlockingQueue* is the root of my problem. Why
    is this taking up so much room in memory? I'm guessing this is the queue
    that serves the tuples up to the next bolt.
    --
    You received this message because you are subscribed to a topic in the
    Google Groups "storm-user" group.
    To unsubscribe from this topic, visit
    https://groups.google.com/d/topic/storm-user/4CPHtPfSdVY/unsubscribe?hl=en
    .
    To unsubscribe from this group and all its topics, send an email to
    storm-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.



    --
    -Chris Maness
    Software Developer and Graphic Designer
    M#: 4235025115

    --
    You received this message because you are subscribed to the Google Groups "storm-user" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/groups/opt_out.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupstorm-user @
postedJan 10, '13 at 6:55p
activeMay 8, '13 at 8:05p
posts7
users4
websitestorm-project.net
irc#storm-user

People

Translate

site design / logo © 2022 Grokbase