FAQ
I'm new to Go, and loving what I'm learning so far. But, I have questions
of course.

I'm processing a text file and part of that process is to see if a phone
number is in as many as 8 different tables (Exist)
and I will update a corresponding field in my text file.
Is it worth setting up concurrency for this? Basically a query to a MySQL
database for each of the 8 tables.

If I should use a concurrent set of processes, what should I do for
signaling? Decrement a counter each time
a Go Routine returns? Is this too trivial a process for concurrency?

I think the limiting factor is the SQL server, but what do you think?

And the rest of the fields in the file are just transforms - from one file
into my current file into a set location.
I process that record by record.

Thank you for your input!
Bryan

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Mihai B at Oct 9, 2014 at 11:46 pm
    You should run benchmarks to find if it is worth it. You may use the
    sync/WaitGroup <http://golang.org/pkg/sync/#WaitGroup> to manage the Go
    routines. Most likely the bottleneck is the SQL server, though you may
    set-up a cluster.


    On Thursday, 9 October 2014 22:29:05 UTC+1, Bryan Jarvis wrote:

    I'm new to Go, and loving what I'm learning so far. But, I have questions
    of course.

    I'm processing a text file and part of that process is to see if a phone
    number is in as many as 8 different tables (Exist)
    and I will update a corresponding field in my text file.
    Is it worth setting up concurrency for this? Basically a query to a MySQL
    database for each of the 8 tables.

    If I should use a concurrent set of processes, what should I do for
    signaling? Decrement a counter each time
    a Go Routine returns? Is this too trivial a process for concurrency?

    I think the limiting factor is the SQL server, but what do you think?

    And the rest of the fields in the file are just transforms - from one file
    into my current file into a set location.
    I process that record by record.

    Thank you for your input!
    Bryan
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Tamás Gulácsi at Oct 10, 2014 at 4:37 am
    Is performance the main goal?
    If yes, slurp all tables into memory. Beforehand. If won't fit, slurp into a Bloom filter.
    Optimize your MySql query, and have only one roundtrip: create a query which checks all tables at once.

    As you have to wait the result to update the row, concurrency won't help much.
    But you could use concurrency for processing multiple rows at once.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Bryan Jarvis at Oct 10, 2014 at 9:49 pm
    Thank you all. I will play around with it for the sake of timing. If
    nothing else, it will be a good learning experience.
    On Thursday, October 9, 2014 4:29:05 PM UTC-5, Bryan Jarvis wrote:

    I'm new to Go, and loving what I'm learning so far. But, I have questions
    of course.

    I'm processing a text file and part of that process is to see if a phone
    number is in as many as 8 different tables (Exist)
    and I will update a corresponding field in my text file.
    Is it worth setting up concurrency for this? Basically a query to a MySQL
    database for each of the 8 tables.

    If I should use a concurrent set of processes, what should I do for
    signaling? Decrement a counter each time
    a Go Routine returns? Is this too trivial a process for concurrency?

    I think the limiting factor is the SQL server, but what do you think?

    And the rest of the fields in the file are just transforms - from one file
    into my current file into a set location.
    I process that record by record.

    Thank you for your input!
    Bryan
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Alex Skinner at Oct 11, 2014 at 3:50 am
    It's important to understand how/why you would introduce concurrency. If
    this were just picking through a text file, of course the answer would be
    no. However, you mention checking tables of what I assume is a database.
      Especially if remote, this is where concurrency would come into play. If
    you imagine a 250ms round trip time for such a query, doing 8 of them would
    take two seconds. Assuming further network latency were the sole cause of
    the delay, it would make sense to send out 8 at the same time to reduce the
    time back down to 250ms.
    Concurrency you'll find works best when you're blocked by things you
    shouldn't be - usually network overhead or processing where multicore
    processor exists.
    On Thursday, October 9, 2014 5:29:05 PM UTC-4, Bryan Jarvis wrote:

    I'm new to Go, and loving what I'm learning so far. But, I have questions
    of course.

    I'm processing a text file and part of that process is to see if a phone
    number is in as many as 8 different tables (Exist)
    and I will update a corresponding field in my text file.
    Is it worth setting up concurrency for this? Basically a query to a MySQL
    database for each of the 8 tables.

    If I should use a concurrent set of processes, what should I do for
    signaling? Decrement a counter each time
    a Go Routine returns? Is this too trivial a process for concurrency?

    I think the limiting factor is the SQL server, but what do you think?

    And the rest of the fields in the file are just transforms - from one file
    into my current file into a set location.
    I process that record by record.

    Thank you for your input!
    Bryan
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Bryan Jarvis at Oct 13, 2014 at 5:33 pm
    Yes, Alex - They are MySQL database tables, and your logic is why I was
    going to attempt (experiment with) concurrency. The tables may not all be
    on the
    same server - they are for development but not necessarily in the future.
    Thank you for the sounding board.
    On Friday, October 10, 2014 10:50:26 PM UTC-5, Alex Skinner wrote:

    It's important to understand how/why you would introduce concurrency. If
    this were just picking through a text file, of course the answer would be
    no. However, you mention checking tables of what I assume is a database.
    Especially if remote, this is where concurrency would come into play. If
    you imagine a 250ms round trip time for such a query, doing 8 of them would
    take two seconds. Assuming further network latency were the sole cause of
    the delay, it would make sense to send out 8 at the same time to reduce the
    time back down to 250ms.
    Concurrency you'll find works best when you're blocked by things you
    shouldn't be - usually network overhead or processing where multicore
    processor exists.
    On Thursday, October 9, 2014 5:29:05 PM UTC-4, Bryan Jarvis wrote:

    I'm new to Go, and loving what I'm learning so far. But, I have
    questions of course.

    I'm processing a text file and part of that process is to see if a phone
    number is in as many as 8 different tables (Exist)
    and I will update a corresponding field in my text file.
    Is it worth setting up concurrency for this? Basically a query to a
    MySQL database for each of the 8 tables.

    If I should use a concurrent set of processes, what should I do for
    signaling? Decrement a counter each time
    a Go Routine returns? Is this too trivial a process for concurrency?

    I think the limiting factor is the SQL server, but what do you think?

    And the rest of the fields in the file are just transforms - from one
    file into my current file into a set location.
    I process that record by record.

    Thank you for your input!
    Bryan
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedOct 9, '14 at 9:29p
activeOct 13, '14 at 5:33p
posts6
users4
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase