FAQ
Hi All,


Time: Sunday 29 November 2009 - 23:39:49
Status: Temporary error, restart node
Message: System error, node killed during node restart by other node
(Internal error, programming error or missing error
message, please report a bug)
Error: 2303
Error data: Node 3 killed this node because GCP stop was detected
Error object: NDBCNTR (Line: 263) 0x0000000a
Program: ndbd
Pid: 3600
Trace: /usr/local/mysql/data/ndb_3_trace.log.3
Version: mysql-5.1.35 ndb-7.0.7
***EOM***

Can anyone provide any input on this. After this error When I restart data
node which crashed. This error message change to

Time: Monday 30 November 2009 - 08:48:34
Status: Temporary error, restart node
Message: System error, node killed during node restart by other node
(Internal error, programming error or missing error message, please report
a bug)
Error: 2303
Error data: Killed by node 3 as copyfrag failed, error: 1501
Error object: NDBCNTR (Line: 263) 0x0000000a
Program: ndbd
Pid: 14566
Trace: /usr/local/mysql/data/ndb_3_trace.log.4
Version: mysql-5.1.35 ndb-7.0.7
***EOM***

Thanks in advance
--
Regards,
Manasi Save

Search Discussions

  • Jeff Sturm at Nov 30, 2009 at 2:47 pm
    Manasi,

    Your "GCP stop" node failure is a documented issue. I have also
    encountered it when loading large data sets into disk data tables.

    See the following documentation for a description and recommendations to
    avoid the problem:

    http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-ndbd-definition.htm
    l#mysql-cluster-ndbd-definition-gcp-stop-errors

    More info in this bug report:

    http://bugs.mysql.com/bug.php?id=37227

    In short, be sure you are using fast disks for storage and reserve
    sufficient buffer memory.

    I'm not certain why you were not able to restart the failed node. You
    may be able to restart with --initial.

    -----Original Message-----
    From: Manasi Save
    Sent: Monday, November 30, 2009 9:01 AM
    To: cluster@lists.mysql.com
    Subject: Need Urgent Help data node crashed

    Hi All,


    Time: Sunday 29 November 2009 - 23:39:49
    Status: Temporary error, restart node
    Message: System error, node killed during node restart by other node
    (Internal error, programming error or missing error
    message, please report a bug)
    Error: 2303
    Error data: Node 3 killed this node because GCP stop was detected
    Error object: NDBCNTR (Line: 263) 0x0000000a
    Program: ndbd
    Pid: 3600
    Trace: /usr/local/mysql/data/ndb_3_trace.log.3
    Version: mysql-5.1.35 ndb-7.0.7
    ***EOM***

    Can anyone provide any input on this. After this error When I restart
    data
    node which crashed. This error message change to

    Time: Monday 30 November 2009 - 08:48:34
    Status: Temporary error, restart node
    Message: System error, node killed during node restart by other node
    (Internal error, programming error or missing error message, please
    report
    a bug)
    Error: 2303
    Error data: Killed by node 3 as copyfrag failed, error: 1501
    Error object: NDBCNTR (Line: 263) 0x0000000a
    Program: ndbd
    Pid: 14566
    Trace: /usr/local/mysql/data/ndb_3_trace.log.4
    Version: mysql-5.1.35 ndb-7.0.7
    ***EOM***

    Thanks in advance
    --
    Regards,
    Manasi Save




    --
    MySQL Cluster Mailing List
    For list archives: http://lists.mysql.com/cluster
    To unsubscribe:
    http://lists.mysql.com/cluster?unsub=jeff.sturm@eprize.com
  • Richard McCluskey at Nov 30, 2009 at 2:56 pm
    Manasi,

    I have had exactly that error. I found that it was due to using 7,200rpm
    SATA disks in the data node, rather than fast SCSI (15,000rpm) disks. As
    soon as I upgraded my data node to SCSI the problem went away. As for
    the node not coming back on line, that is also something I encountered,
    and it was solved by upgrading to 7.0.9 from 7.0.7.

    as for restarting the data node, --initial will not necessarily help you
    unless you are giving up on the data that is already on that node, and
    even then (in my experience) the node will not come back online. Also,
    if you do choose to use --initial, then you will have to first clean up
    all the data on the hard drive (in my case " rm
    -rf /var/lib/mysql/ndb_2_fs/* "). Then when you start with --initial it
    will rebuild everything. I seriously recommend you upgrading to 7.0.9.

    Once you upgrade, just start the node with : ndbd -v.
    If this fails then you can resort to disk cleaning and the --initial
    option.

    good luck,

    Richard
    On Mon, 2009-11-30 at 09:45 -0500, Jeff Sturm wrote:
    Manasi,

    Your "GCP stop" node failure is a documented issue. I have also
    encountered it when loading large data sets into disk data tables.

    See the following documentation for a description and recommendations to
    avoid the problem:

    http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-ndbd-definition.htm
    l#mysql-cluster-ndbd-definition-gcp-stop-errors

    More info in this bug report:

    http://bugs.mysql.com/bug.php?id=37227

    In short, be sure you are using fast disks for storage and reserve
    sufficient buffer memory.

    I'm not certain why you were not able to restart the failed node. You
    may be able to restart with --initial.

    -----Original Message-----
    From: Manasi Save
    Sent: Monday, November 30, 2009 9:01 AM
    To: cluster@lists.mysql.com
    Subject: Need Urgent Help data node crashed

    Hi All,


    Time: Sunday 29 November 2009 - 23:39:49
    Status: Temporary error, restart node
    Message: System error, node killed during node restart by other node
    (Internal error, programming error or missing error
    message, please report a bug)
    Error: 2303
    Error data: Node 3 killed this node because GCP stop was detected
    Error object: NDBCNTR (Line: 263) 0x0000000a
    Program: ndbd
    Pid: 3600
    Trace: /usr/local/mysql/data/ndb_3_trace.log.3
    Version: mysql-5.1.35 ndb-7.0.7
    ***EOM***

    Can anyone provide any input on this. After this error When I restart
    data
    node which crashed. This error message change to

    Time: Monday 30 November 2009 - 08:48:34
    Status: Temporary error, restart node
    Message: System error, node killed during node restart by other node
    (Internal error, programming error or missing error message, please
    report
    a bug)
    Error: 2303
    Error data: Killed by node 3 as copyfrag failed, error: 1501
    Error object: NDBCNTR (Line: 263) 0x0000000a
    Program: ndbd
    Pid: 14566
    Trace: /usr/local/mysql/data/ndb_3_trace.log.4
    Version: mysql-5.1.35 ndb-7.0.7
    ***EOM***

    Thanks in advance
    --
    Regards,
    Manasi Save




    --
    MySQL Cluster Mailing List
    For list archives: http://lists.mysql.com/cluster
    To unsubscribe:
    http://lists.mysql.com/cluster?unsub=jeff.sturm@eprize.com



    --
    Richard McCluskey
    Senior Engineer
    go2Media, Inc.
    rmccluskey@go2.com
    (617) 671-0057

    http://go2.com - The best entertainment guide on mobile.
  • Geert Vanderkelen at Nov 30, 2009 at 3:08 pm
    Constant monitoring of the dangerous --initial option:

    On Nov 30, 2009, at 15:56 , Richard McCluskey wrote:
    ..
    if you do choose to use --initial, then you will have to first clean up
    all the data on the hard drive (in my case " rm
    -rf /var/lib/mysql/ndb_2_fs/* "). Then when you start with --initial it
    will rebuild everything. I seriously recommend you upgrading to 7.0.9.
    --initial will remove the files, it will not do anything else!
    Since there is no filesystem, the data node will re-sync with it's partner. It would do the same when you would put in new disks and start without the --initial option.

    You _do not_ need the --initial if you remove files manually. In fact, forget about --initial al together (if removing data files manually).
    You do not want `ndbd --initial` in your bash history :)

    -Geert
    On Mon, 2009-11-30 at 09:45 -0500, Jeff Sturm wrote:
    Manasi,

    Your "GCP stop" node failure is a documented issue. I have also
    encountered it when loading large data sets into disk data tables.

    See the following documentation for a description and recommendations to
    avoid the problem:

    http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-ndbd-definition.htm
    l#mysql-cluster-ndbd-definition-gcp-stop-errors

    More info in this bug report:

    http://bugs.mysql.com/bug.php?id=37227

    In short, be sure you are using fast disks for storage and reserve
    sufficient buffer memory.

    I'm not certain why you were not able to restart the failed node. You
    may be able to restart with --initial.

    -----Original Message-----
    From: Manasi Save
    Sent: Monday, November 30, 2009 9:01 AM
    To: cluster@lists.mysql.com
    Subject: Need Urgent Help data node crashed

    Hi All,


    Time: Sunday 29 November 2009 - 23:39:49
    Status: Temporary error, restart node
    Message: System error, node killed during node restart by other node
    (Internal error, programming error or missing error
    message, please report a bug)
    Error: 2303
    Error data: Node 3 killed this node because GCP stop was detected
    Error object: NDBCNTR (Line: 263) 0x0000000a
    Program: ndbd
    Pid: 3600
    Trace: /usr/local/mysql/data/ndb_3_trace.log.3
    Version: mysql-5.1.35 ndb-7.0.7
    ***EOM***

    Can anyone provide any input on this. After this error When I restart
    data
    node which crashed. This error message change to

    Time: Monday 30 November 2009 - 08:48:34
    Status: Temporary error, restart node
    Message: System error, node killed during node restart by other node
    (Internal error, programming error or missing error message, please
    report
    a bug)
    Error: 2303
    Error data: Killed by node 3 as copyfrag failed, error: 1501
    Error object: NDBCNTR (Line: 263) 0x0000000a
    Program: ndbd
    Pid: 14566
    Trace: /usr/local/mysql/data/ndb_3_trace.log.4
    Version: mysql-5.1.35 ndb-7.0.7
    ***EOM***

    Thanks in advance
    --
    Regards,
    Manasi Save




    --
    MySQL Cluster Mailing List
    For list archives: http://lists.mysql.com/cluster
    To unsubscribe:
    http://lists.mysql.com/cluster?unsub=jeff.sturm@eprize.com



    --
    Richard McCluskey
    Senior Engineer
    go2Media, Inc.
    rmccluskey@go2.com
    (617) 671-0057

    http://go2.com - The best entertainment guide on mobile.

    --
    MySQL Cluster Mailing List
    For list archives: http://lists.mysql.com/cluster
    To unsubscribe: http://lists.mysql.com/cluster?unsub=geert@mysql.com
    --
    Geert Vanderkelen, MySQL Senior Support Engineer
    Sun Microsystems GmbH Sonnenallee 1, D-85551 Kirchheim-Heimstetten

    Geschaeftsfuehrer: Thomas Schröder, Wolfgang Engels, Wolf Frenkel
    Vorsitz d. Aufs.rat.: Martin Haering HRB MUC 161028

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcluster @
categoriesmysql
postedNov 30, '09 at 2:02p
activeNov 30, '09 at 3:08p
posts4
users4
websitemysql.com
irc#mysql

People

Translate

site design / logo © 2018 Grokbase