FAQ
I have a TCP server that without saving to Couchbase is able to receive
around 130,000 messages / second, but when I try to save those messages to
Couchbase it goes down to 6000 messages / second. How can I improve the
performance of it?

I am using the official Go client from Couchbase.

The server and client are local and the Couchbase server is in my intranet.

I tried using one connection to Couchbase and one connection for each
connection to my server, but I get the same result.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Dave Cheney at May 12, 2016 at 7:25 am
    The first step to solving this problem is to profile your program. I recommend using my github.com/pkg/profile helper which will make it very easy to produce a profile from your same program. Once you have a profile it should be easy to spot where the bottleneck is (hint, it's couchvase)

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Rayland at May 12, 2016 at 10:20 am
    Tried using a goroutine for each insert and I'm up to 63,000 messages /
    second. Don't know if this is the intended usage.
    On Thursday, May 12, 2016 at 10:25:30 AM UTC+3, Dave Cheney wrote:

    The first step to solving this problem is to profile your program. I
    recommend using my github.com/pkg/profile helper which will make it very
    easy to produce a profile from your same program. Once you have a profile
    it should be easy to spot where the bottleneck is (hint, it's couchvase)
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Rayland at May 12, 2016 at 10:56 am
    Tried using multiple connections, and got to 135,000 messages / second
    using 60 connections. How can I find the sweetspot of the connections
    number?
    On Thursday, May 12, 2016 at 10:25:30 AM UTC+3, Dave Cheney wrote:

    The first step to solving this problem is to profile your program. I
    recommend using my github.com/pkg/profile helper which will make it very
    easy to produce a profile from your same program. Once you have a profile
    it should be easy to spot where the bottleneck is (hint, it's couchvase)
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at May 12, 2016 at 11:11 am

    Tried using multiple connections, and got to 135,000 messages / second
    using 60 connections. How can I find the sweetspot of the connections
    number?
    Sorry to appear facetious, but you find the answer by profiling your
    application. By profiling your application, you'll know where the
    bottleneck is, you won't have to guess.

    http://dave.cheney.net/2014/10/22/simple-profiling-package-moved-updated

    On Thursday, May 12, 2016 at 10:25:30 AM UTC+3, Dave Cheney wrote:

    The first step to solving this problem is to profile your program. I
    recommend using my github.com/pkg/profile helper which will make it very
    easy to produce a profile from your same program. Once you have a profile
    it should be easy to spot where the bottleneck is (hint, it's couchvase)
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Manlio Perillo at May 12, 2016 at 3:14 pm
    Il giorno giovedì 12 maggio 2016 12:56:23 UTC+2, Rayland ha scritto:
    Tried using multiple connections, and got to 135,000 messages / second
    using 60 connections. How can I find the sweetspot of the connections
    number?
    Can you post the code? 135,000 messages/second seems suspicious.


    Manlio

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Manlio Perillo at May 12, 2016 at 3:24 pm
    Il giorno giovedì 12 maggio 2016 17:14:01 UTC+2, Manlio Perillo ha scritto:


    Il giorno giovedì 12 maggio 2016 12:56:23 UTC+2, Rayland ha scritto:
    Tried using multiple connections, and got to 135,000 messages / second
    using 60 connections. How can I find the sweetspot of the connections
    number?
    Can you post the code? 135,000 messages/second seems suspicious.
    I also suggest you to run the server under the race detector:
         $ go run -race server_example.go
    and run the client with a large number of concurrent connections.

    I suspect that OpenBucket is not concurrency-safe.


    Manlio

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Rayland at May 17, 2016 at 7:06 am
    I did. I get the same results. No errors.
    On Thursday, May 12, 2016 at 6:24:38 PM UTC+3, Manlio Perillo wrote:



    Il giorno giovedì 12 maggio 2016 17:14:01 UTC+2, Manlio Perillo ha scritto:


    Il giorno giovedì 12 maggio 2016 12:56:23 UTC+2, Rayland ha scritto:
    Tried using multiple connections, and got to 135,000 messages / second
    using 60 connections. How can I find the sweetspot of the connections
    number?
    Can you post the code? 135,000 messages/second seems suspicious.
    I also suggest you to run the server under the race detector:
    $ go run -race server_example.go
    and run the client with a large number of concurrent connections.

    I suspect that OpenBucket is not concurrency-safe.


    Manlio
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Dave Cheney at May 17, 2016 at 7:48 am
    Your program spends a long time in fmt.Printf, you're probably benchmarking
    the speed of your terminal emulator.
    On Tuesday, 17 May 2016 17:06:51 UTC+10, Rayland wrote:

    I did. I get the same results. No errors.
    On Thursday, May 12, 2016 at 6:24:38 PM UTC+3, Manlio Perillo wrote:



    Il giorno giovedì 12 maggio 2016 17:14:01 UTC+2, Manlio Perillo ha
    scritto:


    Il giorno giovedì 12 maggio 2016 12:56:23 UTC+2, Rayland ha scritto:
    Tried using multiple connections, and got to 135,000 messages / second
    using 60 connections. How can I find the sweetspot of the connections
    number?
    Can you post the code? 135,000 messages/second seems suspicious.
    I also suggest you to run the server under the race detector:
    $ go run -race server_example.go
    and run the client with a large number of concurrent connections.

    I suspect that OpenBucket is not concurrency-safe.


    Manlio
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedMay 12, '16 at 7:13a
activeMay 17, '16 at 7:48a
posts9
users3
websitegolang.org

People

Translate

site design / logo © 2022 Grokbase