[go-nuts] Any technical reasons why conversions between []byte and string needs a memmove?

On Thu, Apr 25, 2013 at 12:09 PM, Jingcheng Zhang wrote:
Hello,

Just search the list but didn't find any results.
So I'm curious, nobody cares about this problem?
Why not just change the pointer instead of a new allocation & memmove?
It wastes a lot of CPU time when performing the conversion.

Can someone explain this behavior?

Thanks very much.

--
Best regards,
Jingcheng Zhang
Beijing, P.R.China

--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.

On Wed, Apr 24, 2013 at 7:09 PM, Jingcheng Zhang wrote:

Hello,

Just search the list but didn't find any results.
So I'm curious, nobody cares about this problem?
Why not just change the pointer instead of a new allocation & memmove?
It wastes a lot of CPU time when performing the conversion.

Can someone explain this behavior?

Thanks very much.

--
Best regards,
Jingcheng Zhang
Beijing, P.R.China

--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.

On Thu, Apr 25, 2013 at 10:19 AM, Brad Fitzpatrick wrote:

string is a (*byte, length) that is immutable (you can't change it, and
nobody else can change it)
[]byte is a (*byte, length) that is mutable (you can change it, and others
might change it)

A conversion either way risks violating the immutable rule, so a copy
needs to be made.

What we need is a way to have a type that is (*byte, length) but you can't
change it, but others might. Imagine that is "byteview". Then you could
assign a []byte or a string to a byteview without a copy. But this is a
major language change, and should be done in a generic way for all slice
types, not just []byte.

On Wed, Apr 24, 2013 at 7:52 PM, Jingcheng Zhang wrote:

Hi Dave and Brad,

When we say that string is immutable, doesn't it mean that we can't change
it directly through s[0] = 'a' because it is forbidden by the compiler?
It is the grammar's restriction, not the implementation restriction, is
that right?
The language doesn't forbid changing the content of a []byte, so the
following codes are legal (and maybe common):

s := "hello"
b := []byte(s)
b[0] = 'a'
s = string(b)
println(s) // aello

If the contract of "string is immutable" is followed by compiler, why
can't we optimize the conversion implementation in runtime by changing the
pointer?

There are so many conversions between []byte and string in practice,
causing a lot of waste of CPU time.



--
Best regards,
Jingcheng Zhang
Beijing, P.R.China

On Thu, Apr 25, 2013 at 10:55 AM, Brad Fitzpatrick wrote:

It's forbidden by the compiler because it's forbidden by the language.

You can't do either conversion safely.

Once you do:

s := string(b)

... that 's' can never change. If you shared its pointer with b's
pointer, then:

b[0] = 'x'

... could mutate the memory used by 's'.

Likewise the other way.



--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.

On 25/04/2013, at 1:03 PM, Jingcheng Zhang wrote:

Got it, thanks very much.

However it is always a pain that there are so many conversions between []byte and string, each of which wastes memory and CPU time to run malloc() and memmove().
If the concept of "byteview" is accepted, hope it will solve this problem...



--
Best regards,
Jingcheng Zhang
Beijing, P.R.China
--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.

On Wednesday, April 24, 2013 8:19:29 PM UTC-6, bradfitz wrote:

What we need is a way to have a type that is (*byte, length) but you can't
change it, but others might. Imagine that is "byteview". Then you could
assign a []byte or a string to a byteview without a copy. But this is a
major language change, and should be done in a generic way for all slice
types, not just []byte.

On Apr 24, 2013 8:50 PM, "Kevin Gillette" wrote:
Since strings would still be immutable, the hypothetical byteview would
need to be backed by a []byte, not a string, which still means there'd be
plenty of cases where byteviews don't help efficiency.

There are cases where escape analysis can determine that zero-copy
conversions of []byte <-> string are safe. A subset of these cases may be
trivial to determine -- it seems like implementing that optimization, even
for the easy subset, would have much more practical value than a byteview
(and even if we did have byteviews, the aforementioned optimization would
still be useful).

--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.