On Tue, Apr 5, 2011 at 2:21 PM, Mischa Sandberg wrote:
Came across the following in a paper from Oct 2010. Was wondering is this is
old news I missed in this group.http://pdos.csail.mit.edu/papers/linux:osdi10.pdf
about Linux optimization on multi-core CPU’s.
The group at MIT were exploring how some Linux apps were scaling up ---
sometimes badly, mostly due to hidden contention over cache-line consistency
across the cores’ caches.
In a nutshell: if an app, or the system calls it uses, tries to modify
anything in a cache line (32-64 byte slice of memory) that another core is
using, there’s a lot of fumbling in the dark to make sure there is no
conflict. When I saw PostgreSQL named in the abstract, I thought, “Aha!
Contention over shm”. Not so. Skip to page 11 (section 5.5) for most of the
Someone posted this before, but unfortunately making this really work
in PG is more of a research project than something we can just go do.
I made a stab at writing a spinlock-free version of the LWLock code a
few months ago (which is one of the things they did in the paper) and
I wasn't able to show a lick of benefit. Part of that may be because
I didn't have access to anything bigger than an 8-core box, but it's
also because these things are fairly workload-dependent. In the test
cases I tried I kept bottlenecking on WALInsertLock or, on read-only
workloads, the lock manager partition lock for whichever table I was
hitting, and the changes they made don't address those bottlenecks.
As they write - regarding their benchmark - "This workload is
intended to minimize application-level contention within PostgreSQL in
order to maximize the stress PostgreSQL places on the kernel." -- i.e.
PostgreSQL wasn't really the thing they were trying to stress. It's
interesting stuff - I'm just not sure how much near-term practical
benefit we can get out of it.