[go-nuts] Question on thread safety of maps

On Tue, Apr 29, 2014 at 11:38 AM, Abhijit Kadam wrote:
It is advised to use read write locks to read and write from maps from
concurrent routines http://blog.golang.org/go-maps-in-action#Concurrency

Now if I create a map
aMap := make(map[int][]string)

Then create some X routines. Many routines write to the maps however each
routine ALWAYS adds new element to the map and would NEVER modify existing.
After that they will send keys over a channel.

There is one single routine that reads keys on this channel and accesses map
for the values corresponding to passed keys and once done delete the key
value entry from the map.

If none of the routines modify a value that other routines do, I still need
a lock? Also the delete is when we are sure no one would read the key's
value or modify it.


map[int][]string<-------------------reads &
remove<---------------------------
^
X
routines---------------------------------------------------------------[keys]---------------------->
ch-----------------> reader routine

--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

On Wednesday, April 30, 2014 12:16:10 AM UTC+5:30, Caleb Spare wrote:
If you have a read/write concurrent with a write, you need some kind
of lock. No exceptions.

On Tue, Apr 29, 2014 at 12:14 PM, Abhijit Kadam wrote:
What if I use slice instead?

aSlice := make([][][]string, 10, 10)

Many writer routines would always append data to a unique index in slice
(indexes assigned to them before they start) and then pass the index to
another routine. Other routine which is single reads the data. It will not
delete it. It will just make the aSlice[passedindex] empty.

You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

On Tue, Apr 29, 2014 at 7:22 PM, Abhijit Kadam wrote:

That is first thing I will do, pass everything by value. I am thinking of
passing index to save memory, you see the actual data is [][]string string.

Another reason is to maintain sequence of data to be written by consumer
to file. This is also I am thinking of how to do.

Also consumer is writing data to CSV file. I will apply ParallelFor with
NUMCPU producer routines on original data. Then producers would pass
processed data to a consumer that would write it to file. Do I need to use
buffered channels?

The processing is on 40 MB array. At some cases although it is on large
array, a single processing is not that much and there is no map reduce kind
of process where rows are reduced depending on conditions. At such cases I
will have single routine do all the work as anyways the consumer writer
routine would be slower as it is doing I/O. Is this correct?

At other places where there is data transformation I will apply
concurrency.

On Tue, Apr 29, 2014 at 8:18 PM, Abhijit Kadam wrote:

So passing pointers over channel is safe? And it will ensure the data is
always there for the consumer and not garbage collected ?

On Tuesday, April 29, 2014 10:09:43 PM UTC-7, Abhijit Kadam wrote:


It even works if you pass a stack variable and the function from which it
is passed has returned. So the are blocking the stack or may be not
allocating the variable on a stack.

http://play.golang.org/p/L-EN0bDulb