FAQ

[mongodb-user] mongodb lost data! seems it happened when inserting while it's performing auto sharding

于峰
Dec 15, 2011 at 10:40 am
I started three shards on a physical server like below

mongos> db.shards.find()
{ "_id" : "shard0000", "host" : "10.232.128.104:10001" }
{ "_id" : "shard0001", "host" : "10.232.128.104:10002" }
{ "_id" : "shard0002", "host" : "10.232.128.104:10003" }

Then i insert 4,000,000 records , after a while mongodb began to auto
sharding
While it's sharding i insert another 5,000,000 records

but at the end i got only 8,999,997 records when the whole sharding is
completed!!!

Using great efforts, i finally find that cid=6270001 lost!! see below:

mongos> db.customers.find({"cid":6270001})
mongos> db.customers.find({"cid":6270002})
{ "_id" : ObjectId("4ee9c466498e6a4d818fe861"), "cid" :
NumberLong(6270002), "cname" : "test", "balance" : 1234, "settlestat" : 0,
"settlereason" : "1", "memberid" : NumberLong("101102000436"), "mactid" :
100295272 }
mongos> db.customers.find({"cid":6270000})
{ "_id" : ObjectId("4ee9c4bc498e6a4d819c099b"), "cid" :
NumberLong(6270000), "cname" : "test", "balance" : 1234, "settlestat" : 0,
"settlereason" : "1", "memberid" : NumberLong("101102000436"), "mactid" :
100295272 }

I reviewed the output of mongostat of one shard,i find the info below

<https://lh5.googleusercontent.com/-MxxeYORj_pI/TunOklgFPyI/AAAAAAAAADc/yo7y1ATwFus/s1600/2.jpg>
who can explain the fact?

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To view this discussion on the web visit https://groups.google.com/d/msg/mongodb-user/-/0HXrKSnxDTMJ.
To post to this group, send email to mongodb-user@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
reply

Search Discussions

6 responses

  • Sergei Tulentsev at Dec 15, 2011 at 10:51 am
    faults/s in mongostat log means that mongodb had to go to disk in order to
    perform the operation. It's not an error.

    As for you missing document: maybe there was an error during insertion and
    it didn't get written.
    By default, mongodb client libraries (at least those that I worked with)
    treat writes as 'fire and forget'. So, if there's an error, you won't know.
    To capture write errors, you have to run them in "safe mode". Please,
    consult your driver's documentation.

    On Thu, Dec 15, 2011 at 2:40 PM, 于峰 wrote:

    I started three shards on a physical server like below

    mongos> db.shards.find()
    { "_id" : "shard0000", "host" : "10.232.128.104:10001" }
    { "_id" : "shard0001", "host" : "10.232.128.104:10002" }
    { "_id" : "shard0002", "host" : "10.232.128.104:10003" }

    Then i insert 4,000,000 records , after a while mongodb began to auto
    sharding
    While it's sharding i insert another 5,000,000 records

    but at the end i got only 8,999,997 records when the whole sharding is
    completed!!!

    Using great efforts, i finally find that cid=6270001 lost!! see below:

    mongos> db.customers.find({"cid":6270001})
    mongos> db.customers.find({"cid":6270002})
    { "_id" : ObjectId("4ee9c466498e6a4d818fe861"), "cid" :
    NumberLong(6270002), "cname" : "test", "balance" : 1234, "settlestat" : 0,
    "settlereason" : "1", "memberid" : NumberLong("101102000436"), "mactid" :
    100295272 }
    mongos> db.customers.find({"cid":6270000})
    { "_id" : ObjectId("4ee9c4bc498e6a4d819c099b"), "cid" :
    NumberLong(6270000), "cname" : "test", "balance" : 1234, "settlestat" : 0,
    "settlereason" : "1", "memberid" : NumberLong("101102000436"), "mactid" :
    100295272 }

    I reviewed the output of mongostat of one shard,i find the info below


    <https://lh5.googleusercontent.com/-MxxeYORj_pI/TunOklgFPyI/AAAAAAAAADc/yo7y1ATwFus/s1600/2.jpg>
    who can explain the fact?

    --
    You received this message because you are subscribed to the Google Groups
    "mongodb-user" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/mongodb-user/-/0HXrKSnxDTMJ.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to
    mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/mongodb-user?hl=en.


    --
    Best regards,
    Sergei Tulentsev

    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • 于峰 at Dec 15, 2011 at 2:53 pm
    Thanks a lot~
    Now i see what the reason is ! I indeed should use "safe mode"
    However, the "safe mode" will reduce the performance of mongodb. Now i have
    to think about how to use mongodb properly in my apps.

    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/mongodb-user/-/6AxA5bf2oDkJ.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • Sergei Tulentsev at Dec 15, 2011 at 3:00 pm
    Incorrect. Safe mode reduces performance of your client, as it has to wait
    for ack from the db. The DB server does the same amount of work.
    Though, it depends on how you define "safe mode". If you require fsync,
    then yes, mongodb performance will somewhat decrease. Otherwise, it
    shouldn't.
    On Thu, Dec 15, 2011 at 6:52 PM, 于峰 wrote:

    Thanks a lot~
    Now i see what the reason is ! I indeed should use "safe mode"
    However, the "safe mode" will reduce the performance of mongodb. Now i
    have to think about how to use mongodb properly in my apps.

    --
    You received this message because you are subscribed to the Google Groups
    "mongodb-user" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/mongodb-user/-/6AxA5bf2oDkJ.

    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to
    mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/mongodb-user?hl=en.


    --
    Best regards,
    Sergei Tulentsev

    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • Eliot Horowitz at Dec 15, 2011 at 1:30 pm
    Also - what version od the server, and what code are you using?
    On Thu, Dec 15, 2011 at 5:40 AM, 于峰 wrote:

    I started three shards on a physical server like below

    mongos> db.shards.find()
    { "_id" : "shard0000", "host" : "10.232.128.104:10001" }
    { "_id" : "shard0001", "host" : "10.232.128.104:10002" }
    { "_id" : "shard0002", "host" : "10.232.128.104:10003" }

    Then i insert 4,000,000 records , after a while mongodb began to auto
    sharding
    While it's sharding i insert another 5,000,000 records

    but at the end i got only 8,999,997 records when the whole sharding is
    completed!!!

    Using great efforts, i finally find that cid=6270001 lost!! see below:

    mongos> db.customers.find({"cid":6270001})
    mongos> db.customers.find({"cid":6270002})
    { "_id" : ObjectId("4ee9c466498e6a4d818fe861"), "cid" :
    NumberLong(6270002), "cname" : "test", "balance" : 1234, "settlestat" : 0,
    "settlereason" : "1", "memberid" : NumberLong("101102000436"), "mactid" :
    100295272 }
    mongos> db.customers.find({"cid":6270000})
    { "_id" : ObjectId("4ee9c4bc498e6a4d819c099b"), "cid" :
    NumberLong(6270000), "cname" : "test", "balance" : 1234, "settlestat" : 0,
    "settlereason" : "1", "memberid" : NumberLong("101102000436"), "mactid" :
    100295272 }

    I reviewed the output of mongostat of one shard,i find the info below


    <https://lh5.googleusercontent.com/-MxxeYORj_pI/TunOklgFPyI/AAAAAAAAADc/yo7y1ATwFus/s1600/2.jpg>
    who can explain the fact?

    --
    You received this message because you are subscribed to the Google Groups
    "mongodb-user" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/mongodb-user/-/0HXrKSnxDTMJ.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to
    mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • 于峰 at Dec 15, 2011 at 2:56 pm
    I think Sergei Tulentsev gives me the probably answer
    The mongodb version i'm using for test is v2.0.1 and the codes are on gists
    https://gist.github.com/1479752

    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To view this discussion on the web visit https://groups.google.com/d/msg/mongodb-user/-/r-vZiiR6APYJ.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • Eliot Horowitz at Dec 15, 2011 at 2:59 pm
    I don't see any inserts or upserts in that gist.
    On Thu, Dec 15, 2011 at 9:56 AM, 于峰 wrote:
    I think Sergei Tulentsev gives me the probably answer
    The mongodb version i'm using for test is v2.0.1 and the codes are on gists
    https://gist.github.com/1479752

    --
    You received this message because you are subscribed to the Google Groups
    "mongodb-user" group.
    To view this discussion on the web visit
    https://groups.google.com/d/msg/mongodb-user/-/r-vZiiR6APYJ.

    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to
    mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at
    http://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

Related Discussions

Discussion Navigation
viewthread | post