FAQ
DB.SetMaxOpenConns could reduce the connection pool size, but the
unnecssary connections won't close until DB's free connection size >= DB's
idle connections size.
That is to say, when the pool is busy (lots of requests), the unnecessary
connections won't be closed.

I found the key point code in src/database/sql/sql.go

func (db *DB) putConnDBLocked(dc *driverConn, err error) bool {
if c := len(db.connRequests); c > 0 {
req := db.connRequests[0]
// This copy is O(n) but in practice faster than a linked list.
// TODO: consider compacting it down less often and
// moving the base instead?
copy(db.connRequests, db.connRequests[1:])
db.connRequests = db.connRequests[:c-1]
if err == nil {
dc.inUse = true
}
req <- connRequest{
conn: dc,
err: err,
}
return true
} else if err == nil && !db.closed && db.maxIdleConnsLocked() >
len(db.freeConn) {
db.freeConn = append(db.freeConn, dc)
return true
}
return false
}


And what I want:
func (db *DB) putConnDBLocked(dc *driverConn, err error) bool {
if db.numOpen > db.maxOpen {
return false
}
if c := len(db.connRequests); c > 0 {
req := db.connRequests[0]
........

I want to know that whether I should modify that code to achive my goal. Is
that the right way?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Tad Glines at Dec 25, 2014 at 7:00 pm
    Your suggested change would cause requests (in db.connRequests) to be
    satisfied late (or not at all). So, not a good change.

    DB.SetMaxOpenConns will also reduce maxIdleConns if it is greater than the
    new maxOpenConns. This will cause all idle conns over the limit to be
    closed immediately.

    Can you provide more detail on what problem you are trying to solve?

    -Tad
    On Wed, Dec 24, 2014 at 9:32 PM, Lynx Lo wrote:

    DB.SetMaxOpenConns could reduce the connection pool size, but the
    unnecssary connections won't close until DB's free connection size >= DB's
    idle connections size.
    That is to say, when the pool is busy (lots of requests), the unnecessary
    connections won't be closed.

    I found the key point code in src/database/sql/sql.go

    func (db *DB) putConnDBLocked(dc *driverConn, err error) bool {
    if c := len(db.connRequests); c > 0 {
    req := db.connRequests[0]
    // This copy is O(n) but in practice faster than a linked list.
    // TODO: consider compacting it down less often and
    // moving the base instead?
    copy(db.connRequests, db.connRequests[1:])
    db.connRequests = db.connRequests[:c-1]
    if err == nil {
    dc.inUse = true
    }
    req <- connRequest{
    conn: dc,
    err: err,
    }
    return true
    } else if err == nil && !db.closed && db.maxIdleConnsLocked() >
    len(db.freeConn) {
    db.freeConn = append(db.freeConn, dc)
    return true
    }
    return false
    }


    And what I want:
    func (db *DB) putConnDBLocked(dc *driverConn, err error) bool {
    if db.numOpen > db.maxOpen {
    return false
    }
    if c := len(db.connRequests); c > 0 {
    req := db.connRequests[0]
    ........

    I want to know that whether I should modify that code to achive my goal.
    Is that the right way?

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Lynx Lo at Dec 26, 2014 at 2:35 am
    I wrote a sample:

    package main

    import (
    "database/sql"
    "fmt"
    _ "github.com/go-sql-driver/mysql"
    "log"
    )

    func wait(tip string) {
    var s string
    fmt.Printf("%s >", tip)
    fmt.Scanln(&s)
    }

    func main() {
    db, err := sql.Open("mysql", "user:passwd@/mydb")
    if err != nil {
    log.Fatal(err)
    }
    defer db.Close()

    db.SetMaxIdleConns(4)
    db.SetMaxOpenConns(4)
    tx1, _ := db.Begin()
    tx2, _ := db.Begin()
    tx3, _ := db.Begin()
    tx4, _ := db.Begin()
    wait("tx 1,2,3,4 ok") // numOpen: 4

    db.SetMaxOpenConns(2)
    wait("maxOpenConns set to 2") // numOpen: 4

    // closed 2 of conns, they should be closed
    tx1.Rollback()
    tx2.Rollback()
    wait("tx closed 1,2") // numOpen: 4, expect 2

    // simulate high load, there are many pending requests
    tx5, _ := db.Begin()
    tx6, _ := db.Begin()
    wait("tx opened 5,6") // numOpen: 4, expect 2

    tx3.Rollback()
    tx4.Rollback()
    wait("tx closed 3,4") // numOpen: 4, expect 2

    tx5.Rollback()
    tx6.Rollback()
    }

    The unnecssary connections won't be closed until they are idle.
    If the pool is being busy for a long time, they will keep opening.

    在 2014年12月26日星期五UTC+8上午3时00分40秒,Tad Glines写道:
    Your suggested change would cause requests (in db.connRequests) to be
    satisfied late (or not at all). So, not a good change.

    DB.SetMaxOpenConns will also reduce maxIdleConns if it is greater than the
    new maxOpenConns. This will cause all idle conns over the limit to be
    closed immediately.

    Can you provide more detail on what problem you are trying to solve?

    -Tad

    On Wed, Dec 24, 2014 at 9:32 PM, Lynx Lo <lond...@gmail.com <javascript:>>
    wrote:
    DB.SetMaxOpenConns could reduce the connection pool size, but the
    unnecssary connections won't close until DB's free connection size >= DB's
    idle connections size.
    That is to say, when the pool is busy (lots of requests), the unnecessary
    connections won't be closed.

    I found the key point code in src/database/sql/sql.go

    func (db *DB) putConnDBLocked(dc *driverConn, err error) bool {
    if c := len(db.connRequests); c > 0 {
    req := db.connRequests[0]
    // This copy is O(n) but in practice faster than a linked list.
    // TODO: consider compacting it down less often and
    // moving the base instead?
    copy(db.connRequests, db.connRequests[1:])
    db.connRequests = db.connRequests[:c-1]
    if err == nil {
    dc.inUse = true
    }
    req <- connRequest{
    conn: dc,
    err: err,
    }
    return true
    } else if err == nil && !db.closed && db.maxIdleConnsLocked() >
    len(db.freeConn) {
    db.freeConn = append(db.freeConn, dc)
    return true
    }
    return false
    }


    And what I want:
    func (db *DB) putConnDBLocked(dc *driverConn, err error) bool {
    if db.numOpen > db.maxOpen {
    return false
    }
    if c := len(db.connRequests); c > 0 {
    req := db.connRequests[0]
    ........

    I want to know that whether I should modify that code to achive my goal.
    Is that the right way?

    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Marko at Dec 26, 2014 at 3:03 pm

    On Friday, December 26, 2014 4:35:01 AM UTC+2, Lynx Lo wrote:
    I wrote a sample:
    OK, this demonstrates that it's not working as you expect.

    But what's the use case? Why do you need to control the number of
    connections on the fly like this? Why are MaxIdleConns and MaxOpenConns
    not enough?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Lynx Lo at Dec 27, 2014 at 2:36 am
    The number of database connections is strictly controlled (upstream system
    limited). But somtimes I want to split some of them to other machines. In
    this case, the number of connections should be reduced in time.

    在 2014年12月26日星期五UTC+8下午11时03分46秒,ma...@joh.to写道:
    On Friday, December 26, 2014 4:35:01 AM UTC+2, Lynx Lo wrote:

    I wrote a sample:
    OK, this demonstrates that it's not working as you expect.

    But what's the use case? Why do you need to control the number of
    connections on the fly like this? Why are MaxIdleConns and MaxOpenConns
    not enough?
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Tad Glines at Dec 26, 2014 at 7:24 pm
    Ok, I see the problem now. I also had a second look at the connection
    management code. I think your suggested change is safe and will do what you
    want.

    Typically, maxOpenConns is set once and then left alone. But, if you think
    tight dynamic control of the number of maxOpenCons is needed, then I
    suggest that you create an issue and include your suggested change.

    -Tad
    On Thu, Dec 25, 2014 at 6:35 PM, Lynx Lo wrote:

    I wrote a sample:

    package main

    import (
    "database/sql"
    "fmt"
    _ "github.com/go-sql-driver/mysql"
    "log"
    )

    func wait(tip string) {
    var s string
    fmt.Printf("%s >", tip)
    fmt.Scanln(&s)
    }

    func main() {
    db, err := sql.Open("mysql", "user:passwd@/mydb")
    if err != nil {
    log.Fatal(err)
    }
    defer db.Close()

    db.SetMaxIdleConns(4)
    db.SetMaxOpenConns(4)
    tx1, _ := db.Begin()
    tx2, _ := db.Begin()
    tx3, _ := db.Begin()
    tx4, _ := db.Begin()
    wait("tx 1,2,3,4 ok") // numOpen: 4

    db.SetMaxOpenConns(2)
    wait("maxOpenConns set to 2") // numOpen: 4

    // closed 2 of conns, they should be closed
    tx1.Rollback()
    tx2.Rollback()
    wait("tx closed 1,2") // numOpen: 4, expect 2

    // simulate high load, there are many pending requests
    tx5, _ := db.Begin()
    tx6, _ := db.Begin()
    wait("tx opened 5,6") // numOpen: 4, expect 2

    tx3.Rollback()
    tx4.Rollback()
    wait("tx closed 3,4") // numOpen: 4, expect 2

    tx5.Rollback()
    tx6.Rollback()
    }

    The unnecssary connections won't be closed until they are idle.
    If the pool is being busy for a long time, they will keep opening.

    在 2014年12月26日星期五UTC+8上午3时00分40秒,Tad Glines写道:
    Your suggested change would cause requests (in db.connRequests) to be
    satisfied late (or not at all). So, not a good change.

    DB.SetMaxOpenConns will also reduce maxIdleConns if it is greater than
    the new maxOpenConns. This will cause all idle conns over the limit to be
    closed immediately.

    Can you provide more detail on what problem you are trying to solve?

    -Tad
    On Wed, Dec 24, 2014 at 9:32 PM, Lynx Lo wrote:

    DB.SetMaxOpenConns could reduce the connection pool size, but the
    unnecssary connections won't close until DB's free connection size >= DB's
    idle connections size.
    That is to say, when the pool is busy (lots of requests), the
    unnecessary connections won't be closed.

    I found the key point code in src/database/sql/sql.go

    func (db *DB) putConnDBLocked(dc *driverConn, err error) bool {
    if c := len(db.connRequests); c > 0 {
    req := db.connRequests[0]
    // This copy is O(n) but in practice faster than a linked list.
    // TODO: consider compacting it down less often and
    // moving the base instead?
    copy(db.connRequests, db.connRequests[1:])
    db.connRequests = db.connRequests[:c-1]
    if err == nil {
    dc.inUse = true
    }
    req <- connRequest{
    conn: dc,
    err: err,
    }
    return true
    } else if err == nil && !db.closed && db.maxIdleConnsLocked() >
    len(db.freeConn) {
    db.freeConn = append(db.freeConn, dc)
    return true
    }
    return false
    }


    And what I want:
    func (db *DB) putConnDBLocked(dc *driverConn, err error) bool {
    if db.numOpen > db.maxOpen {
    return false
    }
    if c := len(db.connRequests); c > 0 {
    req := db.connRequests[0]
    ........

    I want to know that whether I should modify that code to achive my goal.
    Is that the right way?

    --
    You received this message because you are subscribed to the Google
    Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-nuts...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Lynx Lo at Dec 27, 2014 at 2:37 am
    Ok, I will create a issue.

    在 2014年12月27日星期六UTC+8上午3时24分22秒,Tad Glines写道:
    Ok, I see the problem now. I also had a second look at the connection
    management code. I think your suggested change is safe and will do what you
    want.

    Typically, maxOpenConns is set once and then left alone. But, if you think
    tight dynamic control of the number of maxOpenCons is needed, then I
    suggest that you create an issue and include your suggested change.

    -Tad

    On Thu, Dec 25, 2014 at 6:35 PM, Lynx Lo <lond...@gmail.com <javascript:>>
    wrote:
    I wrote a sample:

    package main

    import (
    "database/sql"
    "fmt"
    _ "github.com/go-sql-driver/mysql"
    "log"
    )

    func wait(tip string) {
    var s string
    fmt.Printf("%s >", tip)
    fmt.Scanln(&s)
    }

    func main() {
    db, err := sql.Open("mysql", "user:passwd@/mydb")
    if err != nil {
    log.Fatal(err)
    }
    defer db.Close()

    db.SetMaxIdleConns(4)
    db.SetMaxOpenConns(4)
    tx1, _ := db.Begin()
    tx2, _ := db.Begin()
    tx3, _ := db.Begin()
    tx4, _ := db.Begin()
    wait("tx 1,2,3,4 ok") // numOpen: 4

    db.SetMaxOpenConns(2)
    wait("maxOpenConns set to 2") // numOpen: 4

    // closed 2 of conns, they should be closed
    tx1.Rollback()
    tx2.Rollback()
    wait("tx closed 1,2") // numOpen: 4, expect 2

    // simulate high load, there are many pending requests
    tx5, _ := db.Begin()
    tx6, _ := db.Begin()
    wait("tx opened 5,6") // numOpen: 4, expect 2

    tx3.Rollback()
    tx4.Rollback()
    wait("tx closed 3,4") // numOpen: 4, expect 2

    tx5.Rollback()
    tx6.Rollback()
    }

    The unnecssary connections won't be closed until they are idle.
    If the pool is being busy for a long time, they will keep opening.

    在 2014年12月26日星期五UTC+8上午3时00分40秒,Tad Glines写道:
    Your suggested change would cause requests (in db.connRequests) to be
    satisfied late (or not at all). So, not a good change.

    DB.SetMaxOpenConns will also reduce maxIdleConns if it is greater than
    the new maxOpenConns. This will cause all idle conns over the limit to be
    closed immediately.

    Can you provide more detail on what problem you are trying to solve?

    -Tad
    On Wed, Dec 24, 2014 at 9:32 PM, Lynx Lo wrote:

    DB.SetMaxOpenConns could reduce the connection pool size, but the
    unnecssary connections won't close until DB's free connection size >= DB's
    idle connections size.
    That is to say, when the pool is busy (lots of requests), the
    unnecessary connections won't be closed.

    I found the key point code in src/database/sql/sql.go

    func (db *DB) putConnDBLocked(dc *driverConn, err error) bool {
    if c := len(db.connRequests); c > 0 {
    req := db.connRequests[0]
    // This copy is O(n) but in practice faster than a linked list.
    // TODO: consider compacting it down less often and
    // moving the base instead?
    copy(db.connRequests, db.connRequests[1:])
    db.connRequests = db.connRequests[:c-1]
    if err == nil {
    dc.inUse = true
    }
    req <- connRequest{
    conn: dc,
    err: err,
    }
    return true
    } else if err == nil && !db.closed && db.maxIdleConnsLocked() >
    len(db.freeConn) {
    db.freeConn = append(db.freeConn, dc)
    return true
    }
    return false
    }


    And what I want:
    func (db *DB) putConnDBLocked(dc *driverConn, err error) bool {
    if db.numOpen > db.maxOpen {
    return false
    }
    if c := len(db.connRequests); c > 0 {
    req := db.connRequests[0]
    ........

    I want to know that whether I should modify that code to achive my
    goal. Is that the right way?

    --
    You received this message because you are subscribed to the Google
    Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send
    an email to golang-nuts...@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups
    "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an
    email to golang-nuts...@googlegroups.com <javascript:>.
    For more options, visit https://groups.google.com/d/optout.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedDec 25, '14 at 5:32a
activeDec 27, '14 at 2:37a
posts7
users3
websitegolang.org

3 users in discussion

Lynx Lo: 4 posts Tad Glines: 2 posts Marko: 1 post

People

Translate

site design / logo © 2022 Grokbase