optimizes on their Solr indexes taking a really long time. When
investigating, they found that during the optimize, *reads* were
happening on their SSD disk at over 800MB/s, but *writes* were
proceeding at only 20 MB/s.
Looking into ConcurrentMergeScheduler, I discovered that it does indeed
have a default write throttle of only 20 MB/s. I saw code that would
sometimes set the speed to unlimited, but had a hard time figuring out
what circumstances will result in the different settings, so based on
the user experience, I assume that the 20MB/s throttle must be applied
for Solr optimizes.
From what I can see in the code, there's currently no way insolrconfig.xml to configure scheduler options like the maximum write
speed. Before I an open an issue to add additional configuration
options for the merge scheduler, I thought it might be a good idea to
just double-check with everyone here to see whether there's something I
This is likely even affecting people who are not using SSD storage.
Most modern magnetic disks can easily exceed 20MB/s on both reads and
writes. Some RAID arrays can write REALLY fast.