On Tue, Apr 5, 2011 at 2:21 PM, Mischa Sandberg wrote:Came across the following in a paper from Oct 2010. Was wondering is this is
old news I missed in this group.
http://pdos.csail.mit.edu/papers/linux:osdi10.pdfabout Linux optimization on multi-core CPU’s.
The group at MIT were exploring how some Linux apps were scaling up ---
sometimes badly, mostly due to hidden contention over cache-line consistency
across the cores’ caches.
In a nutshell: if an app, or the system calls it uses, tries to modify
anything in a cache line (32-64 byte slice of memory) that another core is
using, there’s a lot of fumbling in the dark to make sure there is no
conflict. When I saw PostgreSQL named in the abstract, I thought, “Aha!
Contention over shm”. Not so. Skip to page 11 (section 5.5) for most of the
PG specifics.
Someone posted this before, but unfortunately making this really work
in PG is more of a research project than something we can just go do.
I made a stab at writing a spinlock-free version of the LWLock code a
few months ago (which is one of the things they did in the paper) and
I wasn't able to show a lick of benefit. Part of that may be because
I didn't have access to anything bigger than an 8-core box, but it's
also because these things are fairly workload-dependent. In the test
cases I tried I kept bottlenecking on WALInsertLock or, on read-only
workloads, the lock manager partition lock for whichever table I was
hitting, and the changes they made don't address those bottlenecks.
As they write - regarding their benchmark - "This workload is
intended to minimize application-level contention within PostgreSQL in
order to maximize the stress PostgreSQL places on the kernel." -- i.e.
PostgreSQL wasn't really the thing they were trying to stress. It's
interesting stuff - I'm just not sure how much near-term practical
benefit we can get out of it.
