Hello all, I have a very simple topology that I'm running that takes in a
tweet from the 1% sample and stores it in a mysql database. My problem
comes when I allow the application to run overnight it runs out of heap
space. No matter how much heap I allocate using the -Xmx argument it fills
up after running for about 24 hours or longer (I've tried up to 16GB
allocated to it). I use the code for the twitter sample spout that's
included in the storm sample code and for the time being I'm running the
topology from Eclipse until I get the bugs worked out of it. For the MySQL
bolt I have the following code:
public void execute(Tuple input, BasicOutputCollector collector) {
try {
con = connector.getConnection(sqlDBUrl, sqlUser, sqlPassword);
} catch (ClassNotFoundException e) {
e.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
} finally {
//System.out.print("Connection created.");
}
PreparedStatement pst = null;
//parse out the Status object from the first tuple
Status s = (Status) input.getValue(0);
try {
pst = con.prepareStatement("INSERT INTO " + db + " (tweet)" +
"VALUES (?);");
pst.setString(1, s.toString());
//execute the SQL
pst.executeUpdate();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
if(pst != null) {try {pst.close();} catch (SQLException e) {e.
printStackTrace();}}
if(con != null) {try {con.close();} catch (SQLException e) {e.
printStackTrace();}}
}
}
As you can see there doesn't seem to be anywhere for a memory leak. Can
anyone point me in the right direction for fixing this problem? I'm using
Storm 0.7.1 for the time being.
