FAQ
In my system, since Lucene.NET is used to index Web data I crawled, it is
easy to exceed 10GB. At least, the performance is not good. I have to do
indexing in memory and load some hot data in cache.
On Thu, Mar 18, 2010 at 5:33 PM, Michael Mitiaguin wrote:

I guess, you sent your question my mistake. You problem is for Java group.
I
am just curious now do people have more than 10 GB indices with Lucene.Net
?

On Thu, Mar 18, 2010 at 6:39 PM, surendra sedhai <mesurendra@gmail.com
wrote:
Hi,
I am indexing my database using Lucene 2.9. Upto 10 GB of index it was
working fine. I have appened no. of time to create an index of around 10
GB
. for indexing i am using :

* IndexWriter writer = new IndexWriter(dir, analyzer, false,
IndexWriter.MaxFieldLength.**UNLIMITED)
*
However, when i further try to add more document in index i got following
error:

Exception in thread "Lucene Merge Thread #0"
org.apache.lucene.index.MergePolicy$MergeException: java.io.IOException:
File too large
at

org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:351)
at

org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:315)
Caused by: java.io.IOException: File too large
at java.io.RandomAccessFile.setLength(Native Method)
at

org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexOutput.setLength(SimpleFSDirectory.java:224)
at

org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java:175)
at

org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:217)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:5114)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4614)
at

org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:235)
at

org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:291)

I am using Ubuntu 8.04. Is this issue due to OS file size limit? Looking
for
your support.

Best Regards,
Surendra

Search Discussions

Discussion Posts

Previous

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 3 of 3 | next ›
Discussion Overview
grouplucene-net-user @
categorieslucene
postedMar 18, '10 at 8:53a
activeMar 18, '10 at 9:42a
posts3
users3
websitelucene.apache.org

People

Translate

site design / logo © 2022 Grokbase