FAQ

[mongodb-user] When I set chunksize to 500MB, it doesn't work

Gen liu
Mar 5, 2012 at 9:40 am
Hi, in my cluster,
1. I add first shard "set1 " to cluster.
2. I insert some large files, each file's size is 430MB.
3. I add the second shard "set2" to cluster.
Then in the mongos's long ,I found:

Mon Mar 5 17:24:33 [Balancer] moveChunk result: { chunkTooBig: true,
estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
0.0 }
Mon Mar 5 17:24:33 [Balancer] balancer move failed: { chunkTooBig:
true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
Mon Mar 5 17:24:33 [Balancer] forcing a split because migrate failed
for size reasons
Mon Mar 5 17:24:33 [Balancer] want to split chunk, but can't find
split point chunk ns:mongo.gridfs.file.chunks at:
set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
{ files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
Mon Mar 5 17:24:33 [Balancer] forced split results: {}
Mon Mar 5 17:24:33 [Balancer] distributed lock 'balancer/
localhost.localdomain:30000:1330936112:1804289383' unlocked.


As you saw, split chunk failed for chunk too big, Then I do:
use config
db.settings.save({_id:"chunksize", value:500})
10 minutes later, i find mongos's log is :
Mon Mar 5 17:34:01 [Balancer] moving chunk ns:
mongo.gridfs.file.chunks moving ( ns:mongo.gridfs.file.chunks at:
set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
{ files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
ObjectId('4f5476c5dffe1fd3f6f1a122') })
set1:set1/192.168.1.97:37017,192.168.1.99:37017 ->
set2:set2/192.168.1.99:37018,192.168.1.121:37018
Mon Mar 5 17:34:02 [Balancer] moveChunk result: { chunkTooBig: true,
estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
0.0 }
Mon Mar 5 17:34:02 [Balancer] balancer move failed: { chunkTooBig:
true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
Mon Mar 5 17:34:02 [Balancer] forcing a split because migrate failed
for size reasons
Mon Mar 5 17:34:02 [Balancer] want to split chunk, but can't find
split point chunk ns:mongo.gridfs.file.chunks at:
set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
{ files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
Mon Mar 5 17:34:02 [Balancer] forced split results: {}
Mon Mar 5 17:34:02 [Balancer] distributed lock 'balancer/
localhost.localdomain:30000:1330936112:1804289383' unlocked.

Yes, i failed too.
So how to use the chunksize correctly?

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongodb-user@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
reply

Search Discussions

10 responses

  • Scott Hernandez at Mar 5, 2012 at 7:11 pm
    You have to restart mongos for that setting to be applied. In future
    versions "flushRouterConfig" fixes this so you can manually reload
    while the server is live.
    On Mon, Mar 5, 2012 at 4:39 AM, gen liu wrote:
    Hi, in my cluster,
    1. I add first shard "set1 " to cluster.
    2. I insert some large files, each file's size is 430MB.
    3. I add the second shard "set2" to cluster.
    Then in the mongos's long ,I found:

    Mon Mar  5 17:24:33 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar  5 17:24:33 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar  5 17:24:33 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar  5 17:24:33 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar  5 17:24:33 [Balancer] forced split results: {}
    Mon Mar  5 17:24:33 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.


    As you saw, split chunk failed for chunk too big, Then I do:
    use config
    db.settings.save({_id:"chunksize", value:500})
    10 minutes later, i find mongos's log is :
    Mon Mar  5 17:34:01 [Balancer] moving chunk ns:
    mongo.gridfs.file.chunks moving ( ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') })
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 ->
    set2:set2/192.168.1.99:37018,192.168.1.121:37018
    Mon Mar  5 17:34:02 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar  5 17:34:02 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar  5 17:34:02 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar  5 17:34:02 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar  5 17:34:02 [Balancer] forced split results: {}
    Mon Mar  5 17:34:02 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.

    Yes, i failed too.
    So how to use the chunksize correctly?

    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • Scott Hernandez at Mar 5, 2012 at 7:22 pm
    Sorry, I misread this. The chunkSize setting is read at the beginning
    of each balancing round.

    On Mon, Mar 5, 2012 at 2:11 PM, Scott Hernandez
    wrote:
    You have to restart mongos for that setting to be applied. In future
    versions "flushRouterConfig" fixes this so you can manually reload
    while the server is live.
    On Mon, Mar 5, 2012 at 4:39 AM, gen liu wrote:
    Hi, in my cluster,
    1. I add first shard "set1 " to cluster.
    2. I insert some large files, each file's size is 430MB.
    3. I add the second shard "set2" to cluster.
    Then in the mongos's long ,I found:

    Mon Mar  5 17:24:33 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar  5 17:24:33 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar  5 17:24:33 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar  5 17:24:33 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar  5 17:24:33 [Balancer] forced split results: {}
    Mon Mar  5 17:24:33 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.


    As you saw, split chunk failed for chunk too big, Then I do:
    use config
    db.settings.save({_id:"chunksize", value:500})
    10 minutes later, i find mongos's log is :
    Mon Mar  5 17:34:01 [Balancer] moving chunk ns:
    mongo.gridfs.file.chunks moving ( ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') })
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 ->
    set2:set2/192.168.1.99:37018,192.168.1.121:37018
    Mon Mar  5 17:34:02 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar  5 17:34:02 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar  5 17:34:02 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar  5 17:34:02 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar  5 17:34:02 [Balancer] forced split results: {}
    Mon Mar  5 17:34:02 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.

    Yes, i failed too.
    So how to use the chunksize correctly?

    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • Gen liu at Mar 6, 2012 at 1:01 am
    But in my cluster, the chunksize seems doesn't work.
    On 3月6日, 上午3时21分, Scott Hernandez wrote:
    Sorry, I misread this. The chunkSize setting is read at the beginning
    of each balancing round.

    On Mon, Mar 5, 2012 at 2:11 PM, Scott Hernandez







    wrote:
    You have to restart mongos for that setting to be applied. In future
    versions "flushRouterConfig" fixes this so you can manually reload
    while the server is live.
    On Mon, Mar 5, 2012 at 4:39 AM, gen liu wrote:
    Hi, in my cluster,
    1. I add first shard "set1 " to cluster.
    2. I insert some large files, each file's size is 430MB.
    3. I add the second shard "set2" to cluster.
    Then in the mongos's long ,I found:
    Mon Mar  5 17:24:33 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar  5 17:24:33 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar  5 17:24:33 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar  5 17:24:33 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar  5 17:24:33 [Balancer] forced split results: {}
    Mon Mar  5 17:24:33 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    As you saw, split chunk failed for chunk too big, Then I do:
    use config
    db.settings.save({_id:"chunksize", value:500})
    10 minutes later, i find mongos's log is :
    Mon Mar  5 17:34:01 [Balancer] moving chunk ns:
    mongo.gridfs.file.chunks moving ( ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') })
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 ->
    set2:set2/192.168.1.99:37018,192.168.1.121:37018
    Mon Mar  5 17:34:02 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar  5 17:34:02 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar  5 17:34:02 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar  5 17:34:02 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar  5 17:34:02 [Balancer] forced split results: {}
    Mon Mar  5 17:34:02 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    Yes, i failed too.
    So how to use the chunksize correctly?
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • Scott Hernandez at Mar 6, 2012 at 1:05 am
    What version is this? Can you turn verbose logging on?
    On Mon, Mar 5, 2012 at 8:01 PM, gen liu wrote:
    But in my cluster, the chunksize seems doesn't work.
    On 3月6日, 上午3时21分, Scott Hernandez wrote:
    Sorry, I misread this. The chunkSize setting is read at the beginning
    of each balancing round.

    On Mon, Mar 5, 2012 at 2:11 PM, Scott Hernandez







    wrote:
    You have to restart mongos for that setting to be applied. In future
    versions "flushRouterConfig" fixes this so you can manually reload
    while the server is live.
    On Mon, Mar 5, 2012 at 4:39 AM, gen liu wrote:
    Hi, in my cluster,
    1. I add first shard "set1 " to cluster.
    2. I insert some large files, each file's size is 430MB.
    3. I add the second shard "set2" to cluster.
    Then in the mongos's long ,I found:
    Mon Mar  5 17:24:33 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar  5 17:24:33 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar  5 17:24:33 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar  5 17:24:33 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar  5 17:24:33 [Balancer] forced split results: {}
    Mon Mar  5 17:24:33 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    As you saw, split chunk failed for chunk too big, Then I do:
    use config
    db.settings.save({_id:"chunksize", value:500})
    10 minutes later, i find mongos's log is :
    Mon Mar  5 17:34:01 [Balancer] moving chunk ns:
    mongo.gridfs.file.chunks moving ( ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') })
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 ->
    set2:set2/192.168.1.99:37018,192.168.1.121:37018
    Mon Mar  5 17:34:02 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar  5 17:34:02 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar  5 17:34:02 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar  5 17:34:02 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar  5 17:34:02 [Balancer] forced split results: {}
    Mon Mar  5 17:34:02 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    Yes, i failed too.
    So how to use the chunksize correctly?
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • Gen liu at Mar 6, 2012 at 1:11 am
    Mongodb 2.0.2.
    turn verbose on mongod or mongos ?
    On 3月6日, 上午9时04分, Scott Hernandez wrote:
    What version is this? Can you turn verbose logging on?






    On Mon, Mar 5, 2012 at 8:01 PM, gen liu wrote:
    But in my cluster, the chunksize seems doesn't work.
    On 3月6日, 上午3时21分, Scott Hernandez wrote:
    Sorry, I misread this. The chunkSize setting is read at the beginning
    of each balancing round.
    On Mon, Mar 5, 2012 at 2:11 PM, Scott Hernandez
    wrote:
    You have to restart mongos for that setting to be applied. In future
    versions "flushRouterConfig" fixes this so you can manually reload
    while the server is live.
    On Mon, Mar 5, 2012 at 4:39 AM, gen liu wrote:
    Hi, in my cluster,
    1. I add first shard "set1 " to cluster.
    2. I insert some large files, each file's size is 430MB.
    3. I add the second shard "set2" to cluster.
    Then in the mongos's long ,I found:
    Mon Mar 5 17:24:33 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar 5 17:24:33 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar 5 17:24:33 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar 5 17:24:33 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar 5 17:24:33 [Balancer] forced split results: {}
    Mon Mar 5 17:24:33 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    As you saw, split chunk failed for chunk too big, Then I do:
    use config
    db.settings.save({_id:"chunksize", value:500})
    10 minutes later, i find mongos's log is :
    Mon Mar 5 17:34:01 [Balancer] moving chunk ns:
    mongo.gridfs.file.chunks moving ( ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') })
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 ->
    set2:set2/192.168.1.99:37018,192.168.1.121:37018
    Mon Mar 5 17:34:02 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar 5 17:34:02 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar 5 17:34:02 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar 5 17:34:02 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar 5 17:34:02 [Balancer] forced split results: {}
    Mon Mar 5 17:34:02 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    Yes, i failed too.
    So how to use the chunksize correctly?
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • Scott Hernandez at Mar 6, 2012 at 1:23 am
    mongos, with setParameter:
    http://www.mongodb.org/display/DOCS/setParameter+Command

    2012/3/5 gen liu <tomliugen@gmail.com>:
    Mongodb 2.0.2.
    turn verbose on mongod or mongos ?
    On 3月6日, 上午9时04分, Scott Hernandez wrote:
    What version is this? Can you turn verbose logging on?






    On Mon, Mar 5, 2012 at 8:01 PM, gen liu wrote:
    But in my cluster, the chunksize seems doesn't work.
    On 3月6日, 上午3时21分, Scott Hernandez wrote:
    Sorry, I misread this. The chunkSize setting is read at the beginning
    of each balancing round.
    On Mon, Mar 5, 2012 at 2:11 PM, Scott Hernandez
    wrote:
    You have to restart mongos for that setting to be applied. In future
    versions "flushRouterConfig" fixes this so you can manually reload
    while the server is live.
    On Mon, Mar 5, 2012 at 4:39 AM, gen liu wrote:
    Hi, in my cluster,
    1. I add first shard "set1 " to cluster.
    2. I insert some large files, each file's size is 430MB.
    3. I add the second shard "set2" to cluster.
    Then in the mongos's long ,I found:
    Mon Mar 5 17:24:33 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar 5 17:24:33 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar 5 17:24:33 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar 5 17:24:33 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar 5 17:24:33 [Balancer] forced split results: {}
    Mon Mar 5 17:24:33 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    As you saw, split chunk failed for chunk too big, Then I do:
    use config
    db.settings.save({_id:"chunksize", value:500})
    10 minutes later, i find mongos's log is :
    Mon Mar 5 17:34:01 [Balancer] moving chunk ns:
    mongo.gridfs.file.chunks moving ( ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') })
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 ->
    set2:set2/192.168.1.99:37018,192.168.1.121:37018
    Mon Mar 5 17:34:02 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar 5 17:34:02 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar 5 17:34:02 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar 5 17:34:02 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar 5 17:34:02 [Balancer] forced split results: {}
    Mon Mar 5 17:34:02 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    Yes, i failed too.
    So how to use the chunksize correctly?
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • Gen liu at Mar 6, 2012 at 2:35 am
    After I restart one mongos, it seems work well.
    Maybe I should restart the mongos when I reset the chunksize.
    I will have more test for this and thanks a lot.
    On 3月6日, 上午9时22分, Scott Hernandez wrote:
    mongos, with setParameter:http://www.mongodb.org/display/DOCS/setParameter+Command

    2012/3/5 gen liu <tomliu...@gmail.com>:






    Mongodb 2.0.2.
    turn verbose on mongod or mongos ?
    On 3月6日, 上午9时04分, Scott Hernandez wrote:
    What version is this? Can you turn verbose logging on?
    On Mon, Mar 5, 2012 at 8:01 PM, gen liu wrote:
    But in my cluster, the chunksize seems doesn't work.
    On 3月6日, 上午3时21分, Scott Hernandez wrote:
    Sorry, I misread this. The chunkSize setting is read at the beginning
    of each balancing round.
    On Mon, Mar 5, 2012 at 2:11 PM, Scott Hernandez
    wrote:
    You have to restart mongos for that setting to be applied. In future
    versions "flushRouterConfig" fixes this so you can manually reload
    while the server is live.
    On Mon, Mar 5, 2012 at 4:39 AM, gen liu wrote:
    Hi, in my cluster,
    1. I add first shard "set1 " to cluster.
    2. I insert some large files, each file's size is 430MB.
    3. I add the second shard "set2" to cluster.
    Then in the mongos's long ,I found:
    Mon Mar 5 17:24:33 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar 5 17:24:33 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar 5 17:24:33 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar 5 17:24:33 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar 5 17:24:33 [Balancer] forced split results: {}
    Mon Mar 5 17:24:33 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    As you saw, split chunk failed for chunk too big, Then I do:
    use config
    db.settings.save({_id:"chunksize", value:500})
    10 minutes later, i find mongos's log is :
    Mon Mar 5 17:34:01 [Balancer] moving chunk ns:
    mongo.gridfs.file.chunks moving ( ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') })
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 ->
    set2:set2/192.168.1.99:37018,192.168.1.121:37018
    Mon Mar 5 17:34:02 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar 5 17:34:02 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar 5 17:34:02 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar 5 17:34:02 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar 5 17:34:02 [Balancer] forced split results: {}
    Mon Mar 5 17:34:02 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    Yes, i failed too.
    So how to use the chunksize correctly?
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • Gen liu at Mar 6, 2012 at 8:51 am
    Hi, Scott, as you said, "The chunksize setting is read at the
    beginning of each balancing round".
    How long of a balancing round? In my test, I wait for 10 minutes, and
    the new chunksize doesn't work.
    So I restart the mongos, moving chunk works.
    Do I need restart mongos when I update the chunksize?

    On 3月6日, 上午3时21分, Scott Hernandez wrote:
    Sorry, I misread this. The chunkSize setting is read at the beginning
    of each balancing round.

    On Mon, Mar 5, 2012 at 2:11 PM, Scott Hernandez







    wrote:
    You have to restart mongos for that setting to be applied. In future
    versions "flushRouterConfig" fixes this so you can manually reload
    while the server is live.
    On Mon, Mar 5, 2012 at 4:39 AM, gen liu wrote:
    Hi, in my cluster,
    1. I add first shard "set1 " to cluster.
    2. I insert some large files, each file's size is 430MB.
    3. I add the second shard "set2" to cluster.
    Then in the mongos's long ,I found:
    Mon Mar  5 17:24:33 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar  5 17:24:33 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar  5 17:24:33 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar  5 17:24:33 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar  5 17:24:33 [Balancer] forced split results: {}
    Mon Mar  5 17:24:33 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    As you saw, split chunk failed for chunk too big, Then I do:
    use config
    db.settings.save({_id:"chunksize", value:500})
    10 minutes later, i find mongos's log is :
    Mon Mar  5 17:34:01 [Balancer] moving chunk ns:
    mongo.gridfs.file.chunks moving ( ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') })
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 ->
    set2:set2/192.168.1.99:37018,192.168.1.121:37018
    Mon Mar  5 17:34:02 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar  5 17:34:02 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar  5 17:34:02 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar  5 17:34:02 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar  5 17:34:02 [Balancer] forced split results: {}
    Mon Mar  5 17:34:02 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    Yes, i failed too.
    So how to use the chunksize correctly?
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • Scott Hernandez at Mar 6, 2012 at 1:38 pm
    It depends how much balancing needs to be done in the current round.
    You can look at the locks for the balancer until it changes, and the
    new round begins.

    You do not need to restart mongos (the one doing the balancing) but
    that will end the round so the next one can restart, once the lock
    times out and gets re-acquired.
    On Tue, Mar 6, 2012 at 3:51 AM, gen liu wrote:
    Hi, Scott, as you said, "The chunksize setting is read at the
    beginning of each balancing round".
    How long of a balancing round? In my test, I wait for 10 minutes, and
    the new chunksize doesn't work.
    So I restart the mongos,  moving chunk works.
    Do I need restart mongos when I update the chunksize?

    On 3月6日, 上午3时21分, Scott Hernandez wrote:
    Sorry, I misread this. The chunkSize setting is read at the beginning
    of each balancing round.

    On Mon, Mar 5, 2012 at 2:11 PM, Scott Hernandez







    wrote:
    You have to restart mongos for that setting to be applied. In future
    versions "flushRouterConfig" fixes this so you can manually reload
    while the server is live.
    On Mon, Mar 5, 2012 at 4:39 AM, gen liu wrote:
    Hi, in my cluster,
    1. I add first shard "set1 " to cluster.
    2. I insert some large files, each file's size is 430MB.
    3. I add the second shard "set2" to cluster.
    Then in the mongos's long ,I found:
    Mon Mar  5 17:24:33 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar  5 17:24:33 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar  5 17:24:33 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar  5 17:24:33 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar  5 17:24:33 [Balancer] forced split results: {}
    Mon Mar  5 17:24:33 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    As you saw, split chunk failed for chunk too big, Then I do:
    use config
    db.settings.save({_id:"chunksize", value:500})
    10 minutes later, i find mongos's log is :
    Mon Mar  5 17:34:01 [Balancer] moving chunk ns:
    mongo.gridfs.file.chunks moving ( ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') })
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 ->
    set2:set2/192.168.1.99:37018,192.168.1.121:37018
    Mon Mar  5 17:34:02 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar  5 17:34:02 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar  5 17:34:02 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar  5 17:34:02 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar  5 17:34:02 [Balancer] forced split results: {}
    Mon Mar  5 17:34:02 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    Yes, i failed too.
    So how to use the chunksize correctly?
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • Gen liu at Mar 7, 2012 at 2:06 am
    Thanks a lot.
    On 3月6日, 下午9时37分, Scott Hernandez wrote:
    It depends how much balancing needs to be done in the current round.
    You can look at the locks for the balancer until it changes, and the
    new round begins.

    You do not need to restart mongos (the one doing the balancing) but
    that will end the round so the next one can restart, once the lock
    times out and gets re-acquired.






    On Tue, Mar 6, 2012 at 3:51 AM, gen liu wrote:
    Hi, Scott, as you said, "The chunksize setting is read at the
    beginning of each balancing round".
    How long of a balancing round? In my test, I wait for 10 minutes, and
    the new chunksize doesn't work.
    So I restart the mongos,  moving chunk works.
    Do I need restart mongos when I update the chunksize?
    On 3月6日, 上午3时21分, Scott Hernandez wrote:
    Sorry, I misread this. The chunkSize setting is read at the beginning
    of each balancing round.
    On Mon, Mar 5, 2012 at 2:11 PM, Scott Hernandez
    wrote:
    You have to restart mongos for that setting to be applied. In future
    versions "flushRouterConfig" fixes this so you can manually reload
    while the server is live.
    On Mon, Mar 5, 2012 at 4:39 AM, gen liu wrote:
    Hi, in my cluster,
    1. I add first shard "set1 " to cluster.
    2. I insert some large files, each file's size is 430MB.
    3. I add the second shard "set2" to cluster.
    Then in the mongos's long ,I found:
    Mon Mar  5 17:24:33 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar  5 17:24:33 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar  5 17:24:33 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar  5 17:24:33 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar  5 17:24:33 [Balancer] forced split results: {}
    Mon Mar  5 17:24:33 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    As you saw, split chunk failed for chunk too big, Then I do:
    use config
    db.settings.save({_id:"chunksize", value:500})
    10 minutes later, i find mongos's log is :
    Mon Mar  5 17:34:01 [Balancer] moving chunk ns:
    mongo.gridfs.file.chunks moving ( ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') })
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 ->
    set2:set2/192.168.1.99:37018,192.168.1.121:37018
    Mon Mar  5 17:34:02 [Balancer] moveChunk result: { chunkTooBig: true,
    estimatedChunkSize: 432819556, errmsg: "chunk too big to move", ok:
    0.0 }
    Mon Mar  5 17:34:02 [Balancer] balancer move failed: { chunkTooBig:
    true, estimatedChunkSize: 432819556, errmsg: "chunk too big to move",
    ok: 0.0 } from: set1 to: set2 chunk: { _id: "mongo.gridfs.file.chunks-
    files_id_ObjectId('4f5476c5dffe1fd3f6f1a121')", lastmod: Timestamp
    2000|1, ns: "mongo.gridfs.file.chunks", min: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a121') }, max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') }, shard: "set1" }
    Mon Mar  5 17:34:02 [Balancer] forcing a split because migrate failed
    for size reasons
    Mon Mar  5 17:34:02 [Balancer] want to split chunk, but can't find
    split point chunk ns:mongo.gridfs.file.chunks at:
    set1:set1/192.168.1.97:37017,192.168.1.99:37017 lastmod: 2|1 min:
    { files_id: ObjectId('4f5476c5dffe1fd3f6f1a121') } max: { files_id:
    ObjectId('4f5476c5dffe1fd3f6f1a122') } got: <empty>
    Mon Mar  5 17:34:02 [Balancer] forced split results: {}
    Mon Mar  5 17:34:02 [Balancer] distributed lock 'balancer/
    localhost.localdomain:30000:1330936112:1804289383' unlocked.
    Yes, i failed too.
    So how to use the chunksize correctly?
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

Related Discussions

Discussion Navigation
viewthread | post

2 users in discussion

Gen liu: 6 posts Scott Hernandez: 5 posts