FAQ
[ https://issues.apache.org/jira/browse/HBASE-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15337786#comment-15337786 ]

Hadoop QA commented on HBASE-16032:
-----------------------------------
(x) *{color:red}-1 overall{color}* |
\\
\\
Vote || Subsystem || Runtime || Comment ||
{color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
{color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} |
{color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} |
{color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 58s {color} | {color:green} master passed {color} |
{color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s {color} | {color:green} master passed with JDK v1.8.0 {color} |
{color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s {color} | {color:green} master passed with JDK v1.7.0_80 {color} |
{color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 51s {color} | {color:green} master passed {color} |
{color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 16s {color} | {color:green} master passed {color} |
{color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 55s {color} | {color:red} hbase-server in master has 1 extant Findbugs warnings. {color} |
{color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s {color} | {color:green} master passed with JDK v1.8.0 {color} |
{color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s {color} | {color:green} master passed with JDK v1.7.0_80 {color} |
{color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 44s {color} | {color:green} the patch passed {color} |
{color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s {color} | {color:green} the patch passed with JDK v1.8.0 {color} |
{color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s {color} | {color:green} the patch passed {color} |
{color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s {color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
{color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s {color} | {color:green} the patch passed {color} |
{color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 52s {color} | {color:green} the patch passed {color} |
{color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 15s {color} | {color:green} the patch passed {color} |
{color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} |
{color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 26m 6s {color} | {color:green} Patch does not cause any errors with Hadoop 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
{color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 11s {color} | {color:green} the patch passed {color} |
{color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s {color} | {color:green} the patch passed with JDK v1.8.0 {color} |
{color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s {color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
{color:green}+1{color} | {color:green} unit {color} | {color:green} 73m 26s {color} | {color:green} hbase-server in the patch passed. {color} |
{color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 18s {color} | {color:green} Patch does not generate ASF License warnings. {color} |
{color:black}{color} | {color:black} {color} | {color:black} 114m 38s {color} | {color:black} {color} |
\\
\\
Subsystem || Report/Notes ||
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12811577/HBASE-16032_v4.patch |
JIRA Issue | HBASE-16032 |
Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile |
uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
git revision | master / 6717f0e |
Default Java | 1.7.0_80 |
Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0 /home/jenkins/jenkins-slave/tools/hudson.model.JDK/JDK_1.7_latest_:1.7.0_80 |
findbugs | v3.0.0 |
findbugs | https://builds.apache.org/job/PreCommit-HBASE-Build/2291/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html |
Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/2291/testReport/ |
modules | C: hbase-server U: hbase-server |
Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/2291/console |
Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |

This message was automatically generated.


Possible memory leak in StoreScanner
------------------------------------

Key: HBASE-16032
URL: https://issues.apache.org/jira/browse/HBASE-16032
Project: HBase
Issue Type: Bug
Affects Versions: 1.2.1, 1.1.5, 0.98.20
Reporter: Yu Li
Assignee: Yu Li
Fix For: 2.0.0, 1.1.6, 1.3.1, 0.98.21, 1.2.3

Attachments: HBASE-16032.patch, HBASE-16032_v2.patch, HBASE-16032_v3.patch, HBASE-16032_v4.patch


We observed frequent fullGC of RS in our production environment, and after analyzing the heapdump, we found large memory occupancy by HStore#changedReaderObservers, the map is surprisingly containing 7500w objects...
After some debugging, I located some possible memory leak in StoreScanner constructor:
{code}
public StoreScanner(Store store, ScanInfo scanInfo, Scan scan, final NavigableSet<byte[]> columns,
long readPt)
throws IOException {
this(store, scan, scanInfo, columns, readPt, scan.getCacheBlocks());
if (columns != null && scan.isRaw()) {
throw new DoNotRetryIOException("Cannot specify any column for a raw scan");
}
matcher = new ScanQueryMatcher(scan, scanInfo, columns,
ScanType.USER_SCAN, Long.MAX_VALUE, HConstants.LATEST_TIMESTAMP,
oldestUnexpiredTS, now, store.getCoprocessorHost());
this.store.addChangedReaderObserver(this);
// Pass columns to try to filter out unnecessary StoreFiles.
List<KeyValueScanner> scanners = getScannersNoCompaction();
...
seekScanners(scanners, matcher.getStartKey(), explicitColumnQuery
&& lazySeekEnabledGlobally, parallelSeekEnabled);
...
resetKVHeap(scanners, store.getComparator());
}
{code}
If there's any Exception thrown after {{this.store.addChangedReaderObserver(this)}}, the returned scanner might be null and there's no chance to remove the scanner from changedReaderObservers, like in {{HRegion#get}}
{code}
RegionScanner scanner = null;
try {
scanner = getScanner(scan);
scanner.next(results);
} finally {
if (scanner != null)
scanner.close();
}
{code}
What's more, all exception thrown in the {{HRegion#getScanner}} path will cause scanner==null then memory leak, so we also need to handle this part.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Search Discussions

Discussion Posts

Previous

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 36 of 36 | next ›
Discussion Overview
groupissues @
categorieshbase, hadoop
postedJun 15, '16 at 2:51p
activeJun 18, '16 at 11:54a
posts36
users1
websitehbase.apache.org

1 user in discussion

Hadoop QA (JIRA): 36 posts

People

Translate

site design / logo © 2019 Grokbase