[go-nuts] Char and char substitute

Not sure about your trouble.

String in go is a slice of bytes(uint8), and many build-in functions
consider those bytes as utf8 encoding text. A string can be converted to a
slice of rune, i.e. []rune.

And randChar can be implemented in a simpler way:

func randChar() byte {
return
"0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"[rand.Intn(26+26+10)]
}


or http://play.golang.org/p/4Mul1X3NpR

Hope any of the above helps.

David

Obviously, randChar()is not working for utf8:
http://play.golang.org/p/YuXUgPLJgd

--

To be exact, strings in Go are slices of bytes. But the catch is, there
isn't a 1on1 byte per character match.

A char, or rune, in Go is a int32 literal, while byte is uint8. That means
a utf8 string will usually have more bytes than runes:
http://play.golang.org/p/adAqyin7P9

--

I've made a more elaborate demo to illustrate the relations between bytes
and runes and strings:

http://play.golang.org/p/qUT3DH6U1Y

--

So if I understand it right it should look like this:

string -> is an array of runes == "int32"
runes -> is an array of bytes == ASCII

so if u have the value of '^' and 'a' you can make this 'â'
and rune of 'â' is []byte{ '^' , 'a' }.

then this -> "är du" -> rune {[ ¨, a], r , d, u} right??

string is represented as a slice of byte (uint8) [without the cap], but
string definitely is not array.

normally, Go strings are encoded as utf-8, but that's not required.

however, if it's, Go provides some functionality to range over runes
in it and convert them from/into a slice of rune, etc.

runes -> is an array of bytes == ASCII
>
rune is an alias int32, and represent a unicode code point.

in loose terms, a unicode code point is what you'd call a character, and
by default, Go stores these "character values" as Go string encoded in a
variable length encoding named utf-8. (you might want to read:
http://en.wikipedia.org/wiki/Unicode and
http://en.wikipedia.org/wiki/UTF-8
)
seems you are talking about combining diacritical marks.
they are quite different beasts, and strings in Go doesn't combine them
as you've expected.

--

The ASCII-UTF-8 is a bit of a long discussion. In short:

A byte is a 8bit number.

A UTF-8 char means a variable number of bytes, 1 byte up to four bytes.
This means that a UTF-8 string like this "șț" is, in fact, an interpreted
stream of bytes.

Why variable number of bytes per char? The transformation is smart enough
not to always use 4 bytes per char. It assigns as many bytes there are
needed, a variable amount, so as to not waste 3 bytes for a 1 byte char.

This stream of bytes, this encoding of variable byte size chars, has a the
perfect Go equivalent: a slice. A slice is a variable size type. On the
other hand, an array in Go is a fixed sized type: [4]byte, [2]byte.

So a UTF-8 string in Go is perfectly reflecting the UTF-8 mechanism, an
interpreted slice of variable size bytes: []byte.



TL;DR
You need more than 1 byte to construct a UTF-8 character (up to 4 bytes).
Which, incidentally, is the rune type in Go: int32.

But UTF-8 chars have variable byte size. Slices are variable in size. The
bytes in them respect the UTF-8 encoding for byte transformation in chars
(runes).

--

Probably helps to get your head around UTF8 first [1], then Chapter 3 of
[2] has a good exploration of Go's implementation.

[1] http://www.joelonsoftware.com/articles/Unicode.html
[2] http://www.qtrac.eu/gobook.html

Tack you all it help allot and i think that is have a grasp on the subject
to a level of good understanding.

But I'm still wandering if there is a way of combing bytes in to runes??

//w/ thx BJ

can you give an example what do you want to do?

--

Yes... but I'm afraid it's not the outcome you've expecting:
http://play.golang.org/p/EOSh_Aef_K

byte (uint8) and rune(int32) are numbers.
An *interpreted* rune (int32) value can be a char:
http://play.golang.org/p/wBhcIZ3LQK
An *interpreted* stream of numbers can be a string:
http://play.golang.org/p/HFEI75jdpq

--

no wandering it does work!!

that's because your add the bytes together utf 8 is a table no
a mathematics allgoertem what ur doing is accssing the position of (eg.)
'a'+'b'
eg. if we say that a -> 1 and b -> 2 by doing the a+b your getting c that
is in position 3.

what is was thinkin is more around the line of

var r1 rune = string ([]byte {1 , 2 })

rune and string are very different.

To combine two bytes to get a rune:
http://play.golang.org/p/Ord10p_9hQ

To combine two bytes to get a string:
http://play.golang.org/p/7B9Q2JSaFu

--

http://play.golang.org/p/lRoaRoTuMT

--

And this is how you go from a string to a rune in it, if you know the byte
order and encoding: http://play.golang.org/p/XG2J83ldmr

Otherwise, you use the Rune related Go packages: utf8, bytes, strings.

--

ahw! So there is package

tanks Mate very use full!

:)
Yes, there are, right here: http://golang.org/pkg/

--

Is it safe to say that the Jedi Council approves of go then?


--

dude becuerful Jedi has been my family name for 3 or 4 genration

One quick thing that hasn't been explicitly said (other than via links) is
that UTF8 actually includes all of ASCII. Anything that is valid ASCII is
valid UTF8, UTF8 just also includes a whole lot more. So if you know your
code is only ever going to deal with ascii, you can use the individual
bytes of a string as if they're C style characters.... but that's not
always safe to do.