FAQ
using c I would create a union of a 32bit integer and a four byte array,
initialize the integer with an eight character hex constant and then check
the first byte of the array.

of course I can't cast an integer pointer to a byte pointer in go

has anybody got a neat trick to determine endian-ness in go without
resorting to incorporating c code?

--

Search Discussions

  • David Symonds at Sep 18, 2012 at 9:36 am

    On Tue, Sep 18, 2012 at 7:11 PM, ris wrote:

    of course I can't cast an integer pointer to a byte pointer in go
    Not true. You can if you use the unsafe package and cast via unsafe.Pointer.

    --
  • Rémy Oudompheng at Sep 18, 2012 at 9:37 am

    On 2012/9/18 ris wrote:
    using c I would create a union of a 32bit integer and a four byte array,
    initialize the integer with an eight character hex constant and then check
    the first byte of the array.
    I'm not sure what you are trying to do. Is your result supposed to be
    endianness-dependent?
    You are either trying to convert the integer to an uint8 (which is
    straightforward), or trying to introspect the memory representation of
    the integer (which you can do using unsafe).

    Rémy.

    --
  • Rémy Oudompheng at Sep 18, 2012 at 9:38 am

    On 2012/9/18 Rémy Oudompheng wrote:
    On 2012/9/18 ris wrote:
    using c I would create a union of a 32bit integer and a four byte array,
    initialize the integer with an eight character hex constant and then check
    the first byte of the array.
    I'm not sure what you are trying to do. Is your result supposed to be
    endianness-dependent?
    Sorry I think I simply misread the message.

    Rémy.

    --
  • Jan Mercl at Sep 18, 2012 at 9:47 am

    On Tue, Sep 18, 2012 at 11:11 AM, ris wrote:
    has anybody got a neat trick to determine endian-ness in go without
    resorting to incorporating c code?
    Recommended reading
    http://commandcenter.blogspot.de/2012/04/byte-order-fallacy.html

    -j

    --
  • Ris at Sep 18, 2012 at 9:59 am
    Unhelpfully, this kinda assumes I am an idiot. If I wish to display the
    contents of binary files that have been produced on any one of a number of
    different machines I might need to analyze 32 bits of data from the file as
    four EBCDIC characters, an IEEE floating point number or z80 assembler
    code. If I want a utility to tell me the endian-ness of the current machine
    then I can write it very neatly in C. I'd be interested if anybody knows
    how I can determine the endian-ness of a machine using go.
    On Tuesday, September 18, 2012 10:47:10 AM UTC+1, Jan Mercl wrote:

    On Tue, Sep 18, 2012 at 11:11 AM, ris <ris.w...@gmail.com <javascript:>>
    wrote:
    has anybody got a neat trick to determine endian-ness in go without
    resorting to incorporating c code?
    Recommended reading
    http://commandcenter.blogspot.de/2012/04/byte-order-fallacy.html

    -j
    --
  • Rémy Oudompheng at Sep 18, 2012 at 10:03 am

    On 2012/9/18 ris wrote:
    Unhelpfully, this kinda assumes I am an idiot. If I wish to display the
    contents of binary files that have been produced on any one of a number of
    different machines I might need to analyze 32 bits of data from the file as
    four EBCDIC characters, an IEEE floating point number or z80 assembler code.
    If I want a utility to tell me the endian-ness of the current machine then I
    can write it very neatly in C. I'd be interested if anybody knows how I can
    determine the endian-ness of a machine using go.
    Can you give an example of what you would like to write in C? It
    should make answers more helpful.

    Rémy.

    --
  • Ris at Sep 18, 2012 at 10:15 am
    C code to determine endian-ness

    typedef union {long n; char b[sizeof(long)];} number;

    int main(int argc, char ** argv) {
         int i;
         number x;
         char * type = "unknown";

         x.n = 0x0123456789abcdef;
         if (0x01 == x.b[0]) {
             type = "big";
         } else if ((0xef & 0xff) == (x.b[0] & 0xff)) {
             type = "little";
         } else {
             printf("long number = %016lx hex\n", x.n);
             for (i=0; i<sizeof(long); i++) {
                 printf("byte[%d] = %02x hex\n", i, x.b[i] & 0xff);
             }
         }
         printf("%s endian (long is %d bits)\n", type, sizeof(long) * 8);

         return 0;
    }

    and that code nearly translated to go

    func main() {
         var x int = 0x012345678
         var p unsafe.Pointer = unsafe.Pointer(&x)
         var bp *byte = unsafe.Pointer(p)

         var endian string = "unknown"

         if 0x01 == **(*byte)(p)* {
             endian = "big"
         } else if (0x78 & 0xff) == ((**(*byte)(p)*) & 0xff) {
             endian = "little"
         } else {
             var i uintptr = 0
             for i = 0; i < unsafe.Sizeof(x); i++ {
                 fmt.Printf("byte[%d] = %d\n", i, **(bp+1)*)
             }
         }
         fmt.Printf("%s endian (int is %d bits)\n", endian, unsafe.Sizeof(x)*8)
    }

    Casting to byte pointer and dereferencing the first byte works fine but how
    can I display each byte of the integer without integer arithmetic.
    On Tuesday, September 18, 2012 11:02:58 AM UTC+1, Rémy Oudompheng wrote:
    On 2012/9/18 ris <ris.w...@gmail.com <javascript:>> wrote:
    Unhelpfully, this kinda assumes I am an idiot. If I wish to display the
    contents of binary files that have been produced on any one of a number of
    different machines I might need to analyze 32 bits of data from the file as
    four EBCDIC characters, an IEEE floating point number or z80 assembler code.
    If I want a utility to tell me the endian-ness of the current machine then I
    can write it very neatly in C. I'd be interested if anybody knows how I can
    determine the endian-ness of a machine using go.
    Can you give an example of what you would like to write in C? It
    should make answers more helpful.

    Rémy.
    --
  • Rémy Oudompheng at Sep 18, 2012 at 10:49 am

    On 2012/9/18 ris wrote:
    C code to determine endian-ness
    I thought your goal was to "analyze 32 bits of data from binary files
    produced by different machines", and that determining endianess was
    only an intermediate step (for what?).
    and that code nearly translated to go

    func main() {
    var x int = 0x012345678
    var p unsafe.Pointer = unsafe.Pointer(&x)
    var bp *byte = unsafe.Pointer(p)

    var endian string = "unknown"

    if 0x01 == *(*byte)(p) {
    endian = "big"
    } else if (0x78 & 0xff) == ((*(*byte)(p)) & 0xff) {
    endian = "little"
    } else {
    var i uintptr = 0
    for i = 0; i < unsafe.Sizeof(x); i++ {
    fmt.Printf("byte[%d] = %d\n", i, *(bp+1))
    }
    }
    fmt.Printf("%s endian (int is %d bits)\n", endian, unsafe.Sizeof(x)*8)
    }

    Casting to byte pointer and dereferencing the first byte works fine but how
    can I display each byte of the integer without integer arithmetic.
    Ints are not 32-bit large. Use uint32.
    Don't use pointer arithmetic which is even uglier. Cast the
    unsafe.Pointer to a *[4]byte instead.

    Rémy.

    --
  • Ris at Sep 18, 2012 at 11:06 am
    Like this

                 fmt.Printf("byte[%d] = %d\n", i, (*[4]byte(p))[i])

    but that doesn't compile
    On Tuesday, September 18, 2012 11:49:10 AM UTC+1, Rémy Oudompheng wrote:


    Cast the unsafe.Pointer to a *[4]byte instead.

    Rémy.
    --
  • Jan Mercl at Sep 18, 2012 at 10:20 am

    On Tue, Sep 18, 2012 at 11:59 AM, ris wrote:
    Unhelpfully, this kinda assumes I am an idiot. No.
    If I wish to display the
    contents of binary files that have been produced on any one of a number of
    different machines I might need to analyze 32 bits of data from the file as
    four EBCDIC characters, an IEEE floating point number or z80 assembler code.
    Which has nothing to do with endianess of your machine(*) - as the
    article explains.
    If I want a utility to tell me the endian-ness of the current machine then I
    can write it very neatly in C. I'd be interested if anybody knows how I can
    determine the endian-ness of a machine using go.
    Writing Go in C usually gets as hard as is the opposite way.

    -j

    (*) Assuming you're not doing e.g. fbdev DMA or simillar things ;-)

    --
  • Ris at Sep 18, 2012 at 10:49 am
    It's really just a lack of imagination that makes you say that. If I want
    to run a data analysis utility on some data from a machine with a known
    endian-ness I might want to check the endian-ness of the current machine
    first. I'd need a utility to tell me that. Simples.
    On Tuesday, September 18, 2012 11:21:01 AM UTC+1, Jan Mercl wrote:

    On Tue, Sep 18, 2012 at 11:59 AM, ris <ris.w...@gmail.com <javascript:>>
    wrote:
    Unhelpfully, this kinda assumes I am an idiot. No.
    If I wish to display the
    contents of binary files that have been produced on any one of a number of
    different machines I might need to analyze 32 bits of data from the file as
    four EBCDIC characters, an IEEE floating point number or z80 assembler
    code.

    Which has nothing to do with endianess of your machine(*) - as the
    article explains.
    If I want a utility to tell me the endian-ness of the current machine then I
    can write it very neatly in C. I'd be interested if anybody knows how I can
    determine the endian-ness of a machine using go.
    Writing Go in C usually gets as hard as is the opposite way.

    -j

    (*) Assuming you're not doing e.g. fbdev DMA or simillar things ;-)
    --
  • Rémy Oudompheng at Sep 18, 2012 at 10:56 am

    On 2012/9/18 ris wrote:
    It's really just a lack of imagination that makes you say that. If I want to
    run a data analysis utility on some data from a machine with a known
    endian-ness I might want to check the endian-ness of the current machine
    first. I'd need a utility to tell me that. Simples.
    Reading binary data encoded with a known endianness is usually done
    using the methods from the standard library:
    binary.LittleEndian.Uint32 or binary.BigEndian.Uint32 for example.

    Rémy.

    --
  • Rémy Oudompheng at Sep 18, 2012 at 10:56 am

    On 2012/9/18 Rémy Oudompheng wrote:
    On 2012/9/18 ris wrote:
    It's really just a lack of imagination that makes you say that. If I want to
    run a data analysis utility on some data from a machine with a known
    endian-ness I might want to check the endian-ness of the current machine
    first. I'd need a utility to tell me that. Simples.
    Reading binary data encoded with a known endianness is usually done
    using the methods from the standard library:
    binary.LittleEndian.Uint32 or binary.BigEndian.Uint32 for example.
    In the encoding/binary package: http://golang.org/pkg/encoding/binary/

    --
  • Minux at Sep 18, 2012 at 10:58 am

    On Tue, Sep 18, 2012 at 6:49 PM, ris wrote:

    It's really just a lack of imagination that makes you say that. If I want
    to run a data analysis utility on some data from a machine with a known
    endian-ness I might want to check the endian-ness of the current machine
    first. I'd need a utility to tell me that. Simples.
    If you already know the endianness of the data file, why don't use what
    Rob's article suggests?
    If you are using Go, encoding/binary could help you.

    --
  • Jan Mercl at Sep 18, 2012 at 11:04 am

    On Tue, Sep 18, 2012 at 12:49 PM, ris wrote:
    It's really just a lack of imagination that makes you say that. If I want to
    run a data analysis utility on some data from a machine with a known
    endian-ness I might want to check the endian-ness of the current machine
    first. I'd need a utility to tell me that. Simples.
    If you know the endianness of data in a file then you (have to)
    process that data endianessaware. Processing external data sources
    (like files, network packets, ...) in a
    endianess-of-that-external-data-aware-way doesn't need to know the
    endianess of the machine executing the processing program.

    You might want to check the endianness of the current machine first.
    But you don't have to and doing that is (most often) a mistake.
    Simples.

    -j

    --
  • Ris at Sep 18, 2012 at 11:14 am
    Fair enough. Incidentally, does anyone know how to tell the endian-ness of
    a machine using go?
    On Tuesday, September 18, 2012 12:04:42 PM UTC+1, Jan Mercl wrote:

    On Tue, Sep 18, 2012 at 12:49 PM, ris <ris.w...@gmail.com <javascript:>>
    wrote:
    It's really just a lack of imagination that makes you say that. If I want to
    run a data analysis utility on some data from a machine with a known
    endian-ness I might want to check the endian-ness of the current machine
    first. I'd need a utility to tell me that. Simples.
    If you know the endianness of data in a file then you (have to)
    process that data endianessaware. Processing external data sources
    (like files, network packets, ...) in a
    endianess-of-that-external-data-aware-way doesn't need to know the
    endianess of the machine executing the processing program.

    You might want to check the endianness of the current machine first.
    But you don't have to and doing that is (most often) a mistake.
    Simples.

    -j
    --
  • David Symonds at Sep 18, 2012 at 11:26 am

    On Tue, Sep 18, 2012 at 9:14 PM, ris wrote:

    Fair enough. Incidentally, does anyone know how to tell the endian-ness of a
    machine using go?
    We've told you multiple times: cast an integer to a byte array via
    unsafe.Pointer.

    --
  • Ris at Sep 18, 2012 at 11:56 am
    See earlier posting of my code. Doing so allows me to get first byte but
    not subsequent ones.
    On Tuesday, September 18, 2012 12:26:30 PM UTC+1, David Symonds wrote:

    On Tue, Sep 18, 2012 at 9:14 PM, ris <ris.w...@gmail.com <javascript:>>
    wrote:
    Fair enough. Incidentally, does anyone know how to tell the endian-ness of a
    machine using go?
    We've told you multiple times: cast an integer to a byte array via
    unsafe.Pointer.
    --
  • Dan Kortschak at Sep 18, 2012 at 11:59 am
    *(*[4]byte)(unsafe.Pointer(i))

    On 18/09/2012, at 9:26 PM, "ris" wrote:

    See earlier posting of my code. Doing so allows me to get first byte but not subsequent ones.

    On Tuesday, September 18, 2012 12:26:30 PM UTC+1, David Symonds wrote:
    On Tue, Sep 18, 2012 at 9:14 PM, ris wrote:

    Fair enough. Incidentally, does anyone know how to tell the endian-ness of a
    machine using go?
    We've told you multiple times: cast an integer to a byte array via
    unsafe.Pointer.

    --


    --
  • Francisco Souza at Sep 18, 2012 at 12:03 pm

    On Tue, Sep 18, 2012 at 8:59 AM, Dan Kortschak wrote:
    *(*[4]byte)(unsafe.Pointer(i))
    Just (*[4]byte)(unsafe.Pointer(i)) is enough.

    --
    ~f

    --
  • Ris at Sep 18, 2012 at 12:19 pm
    Are you suggesting I can do something like:

                 fmt.Printf("byte[%d] = %d\n", i, (*[4]byte(p)) +
    (*[4]byte(unsafe.Pointer(i))))

    because that gives compile time erros

    ./ndn.go:21: cannot convert p (type unsafe.Pointer) to type [4]byte
    ./ndn.go:21: invalid indirect of ([4]byte)(p) (type [4]byte)
    ./ndn.go:21: cannot convert unsafe.Pointer(i) (type unsafe.Pointer) to type
    [4]byte
    ./ndn.go:21: invalid indirect of ([4]byte)(unsafe.Pointer(i)) (type [4]byte)
    On Tuesday, September 18, 2012 1:03:34 PM UTC+1, Francisco Souza wrote:

    On Tue, Sep 18, 2012 at 8:59 AM, Dan Kortschak
    <dan.ko...@adelaide.edu.au <javascript:>> wrote:
    *(*[4]byte)(unsafe.Pointer(i))
    Just (*[4]byte)(unsafe.Pointer(i)) is enough.

    --
    ~f
    --
  • Ris at Sep 18, 2012 at 12:08 pm
    I don't understand what to make of this as I have an unsafe.Pointer "p"
    pointing to an int and I have a variable "i" which is a byte offset from
    the first byte of the int.

    The problem is go doesn't allow me to add the offset to the pointer and I
    just don't know how to otherwise get at the internal bytes of the int.
    On Tuesday, September 18, 2012 12:59:41 PM UTC+1, kortschak wrote:

    *(*[4]byte)(unsafe.Pointer(i))

    On 18/09/2012, at 9:26 PM, "ris" <ris.w...@gmail.com <javascript:>> wrote:

    See earlier posting of my code. Doing so allows me to get first byte
    but not subsequent ones.
    On Tuesday, September 18, 2012 12:26:30 PM UTC+1, David Symonds wrote:
    On Tue, Sep 18, 2012 at 9:14 PM, ris wrote:

    Fair enough. Incidentally, does anyone know how to tell the endian-ness of a
    machine using go?
    We've told you multiple times: cast an integer to a byte array via
    unsafe.Pointer.
    --


    --
  • Dan Kortschak at Sep 18, 2012 at 12:13 pm
    why do you need to add an offset when you can index into the array?
    On 18/09/2012, at 9:38 PM, "ris" wrote:

    I don't understand what to make of this as I have an unsafe.Pointer "p" pointing to an int and I have a variable "i" which is a byte offset from the first byte of the int.

    The problem is go doesn't allow me to add the offset to the pointer and I just don't know how to otherwise get at the internal bytes of the int.
    --
  • Ris at Sep 18, 2012 at 12:22 pm
    By index index into the array, do you mean like this:

                             fmt.Printf("byte[%d] = %d\n", i, (*[4]byte(p))[i])

    because that gives compile time errors

    ./ndn.go:21: cannot convert p (type unsafe.Pointer) to type [4]byte
    ./ndn.go:21: invalid indirect of ([4]byte)(p) (type [4]byte)
    On Tuesday, September 18, 2012 1:13:12 PM UTC+1, kortschak wrote:

    why do you need to add an offset when you can index into the array?

    On 18/09/2012, at 9:38 PM, "ris" <ris.w...@gmail.com <javascript:>>
    wrote:
    I don't understand what to make of this as I have an unsafe.Pointer "p"
    pointing to an int and I have a variable "i" which is a byte offset from
    the first byte of the int.
    The problem is go doesn't allow me to add the offset to the pointer and
    I just don't know how to otherwise get at the internal bytes of the int.
    --
  • Rémy Oudompheng at Sep 18, 2012 at 12:28 pm
    The star operator has low priority, just like in C. You must enclose
    the pointer type in parentheses.

    Rémy.


    2012/9/18, ris <ris.wright@gmail.com>:
    By index index into the array, do you mean like this:

    fmt.Printf("byte[%d] = %d\n", i, (*[4]byte(p))[i])

    because that gives compile time errors

    ./ndn.go:21: cannot convert p (type unsafe.Pointer) to type [4]byte
    ./ndn.go:21: invalid indirect of ([4]byte)(p) (type [4]byte)
    On Tuesday, September 18, 2012 1:13:12 PM UTC+1, kortschak wrote:

    why do you need to add an offset when you can index into the array?

    On 18/09/2012, at 9:38 PM, "ris" <ris.w...@gmail.com <javascript:>>
    wrote:
    I don't understand what to make of this as I have an unsafe.Pointer "p"
    pointing to an int and I have a variable "i" which is a byte offset from
    the first byte of the int.
    The problem is go doesn't allow me to add the offset to the pointer and
    I just don't know how to otherwise get at the internal bytes of the int.
    --

    --
  • Larry Clapp at Sep 18, 2012 at 12:27 pm
    Is this what you're looking for?

    func main() {
       var i uint32 = 0x12345678;
       x := (*[4]byte)(unsafe.Pointer(&i))
       for _, xn := range *x {
         fmt.Printf("%x\n", xn)
       }
    }

    =>

    78
    56
    34
    12

    -- L

    On Tuesday, September 18, 2012 8:08:27 AM UTC-4, ris wrote:


    I don't understand what to make of this as I have an unsafe.Pointer "p"
    pointing to an int and I have a variable "i" which is a byte offset from
    the first byte of the int.

    The problem is go doesn't allow me to add the offset to the pointer and I
    just don't know how to otherwise get at the internal bytes of the int.
    On Tuesday, September 18, 2012 12:59:41 PM UTC+1, kortschak wrote:

    *(*[4]byte)(unsafe.Pointer(i))

    On 18/09/2012, at 9:26 PM, "ris" wrote:

    See earlier posting of my code. Doing so allows me to get first byte
    but not subsequent ones.
    On Tuesday, September 18, 2012 12:26:30 PM UTC+1, David Symonds wrote:
    On Tue, Sep 18, 2012 at 9:14 PM, ris wrote:

    Fair enough. Incidentally, does anyone know how to tell the
    endian-ness of a
    machine using go?
    We've told you multiple times: cast an integer to a byte array via
    unsafe.Pointer.
    --


    --
  • Ris at Sep 18, 2012 at 12:24 pm
    You are receiving long and heartfelt applause *thumbsup*
    On Tuesday, September 18, 2012 1:20:48 PM UTC+1, Larry Clapp wrote:

    Is this what you're looking for?

    func main() {
    var i uint32 = 0x12345678;
    x := (*[4]byte)(unsafe.Pointer(&i))
    for _, xn := range *x {
    fmt.Printf("%x\n", xn)
    }
    }

    =>

    78
    56
    34
    12

    -- L

    On Tuesday, September 18, 2012 8:08:27 AM UTC-4, ris wrote:


    I don't understand what to make of this as I have an unsafe.Pointer "p"
    pointing to an int and I have a variable "i" which is a byte offset from
    the first byte of the int.

    The problem is go doesn't allow me to add the offset to the pointer and I
    just don't know how to otherwise get at the internal bytes of the int.
    On Tuesday, September 18, 2012 12:59:41 PM UTC+1, kortschak wrote:

    *(*[4]byte)(unsafe.Pointer(i))

    On 18/09/2012, at 9:26 PM, "ris" wrote:

    See earlier posting of my code. Doing so allows me to get first byte
    but not subsequent ones.
    On Tuesday, September 18, 2012 12:26:30 PM UTC+1, David Symonds wrote:
    On Tue, Sep 18, 2012 at 9:14 PM, ris wrote:

    Fair enough. Incidentally, does anyone know how to tell the
    endian-ness of a
    machine using go?
    We've told you multiple times: cast an integer to a byte array via
    unsafe.Pointer.
    --


    --
  • Larry Clapp at Sep 18, 2012 at 12:44 pm
    Cool, glad to help out.

    Having established some "cred", let me just say that I guess I lack
    imagination, 'cause I find Rob's article pretty convincing. If you know
    the encoding of the source data, why would you care what the current
    machine is?

    I *guess *you're trying to do something like "If endian-ness of incoming
    data is same as endian-ness of current machine, read the data byte for
    byte; else byte-swap incoming data". But why not take the portable
    approach and just do as Rob suggested?

    Let's say your data stream has a little-endian-encoded 32-bit integer.
    Here's how to extract it (assuming unsigned bytes):
    i = (data[0]<<0) | (data[1]<<8) | (data[2]<<16) | (data[3]<<24);
    If it's big-endian, here's how to extract it:
    i = (data[3]<<0) | (data[2]<<8) | (data[1]<<16) | (data[0]<<24);

    Again, I'm not saying you're wrong, just that I don't understand.

    -- L

    On Tuesday, September 18, 2012 8:24:23 AM UTC-4, ris wrote:

    You are receiving long and heartfelt applause *thumbsup*
    On Tuesday, September 18, 2012 1:20:48 PM UTC+1, Larry Clapp wrote:

    Is this what you're looking for?

    func main() {
    var i uint32 = 0x12345678;
    x := (*[4]byte)(unsafe.Pointer(&i))
    for _, xn := range *x {
    fmt.Printf("%x\n", xn)
    }
    }

    =>

    78
    56
    34
    12

    -- L

    On Tuesday, September 18, 2012 8:08:27 AM UTC-4, ris wrote:


    I don't understand what to make of this as I have an unsafe.Pointer "p"
    pointing to an int and I have a variable "i" which is a byte offset from
    the first byte of the int.

    The problem is go doesn't allow me to add the offset to the pointer and
    I just don't know how to otherwise get at the internal bytes of the int.
    On Tuesday, September 18, 2012 12:59:41 PM UTC+1, kortschak wrote:

    *(*[4]byte)(unsafe.Pointer(i))

    On 18/09/2012, at 9:26 PM, "ris" wrote:

    See earlier posting of my code. Doing so allows me to get first byte
    but not subsequent ones.
    On Tuesday, September 18, 2012 12:26:30 PM UTC+1, David Symonds wrote:
    On Tue, Sep 18, 2012 at 9:14 PM, ris wrote:

    Fair enough. Incidentally, does anyone know how to tell the
    endian-ness of a
    machine using go?
    We've told you multiple times: cast an integer to a byte array via
    unsafe.Pointer.
    --


    --
  • Ris at Sep 18, 2012 at 12:48 pm
    So, I have some binary data.

    I know which machine it came from but I don't know the endian-ness of
    that machine.

    I run a little utility on that machine to tell me which endian-ness it
    is.

    Now I can run a data analysis program on another machine (the endian-
    ness of which I don't need to know) with a command line argument or
    environment variable telling the code whether to take bytes front to
    back or back to front as in Rob's article.

    I am hoping not to be inundated with questions about why I cannot run
    the data analysis program on the same machine as the data file came
    from because I'm not saying :-P

    Also, I'm not answering any questions about why I keep putting a dash
    in endian-ness :-)
    On Sep 18, 1:36 pm, Larry Clapp wrote:
    Cool, glad to help out.

    Having established some "cred", let me just say that I guess I lack
    imagination, 'cause I find Rob's article pretty convincing.  If you know
    the encoding of the source data, why would you care what the current
    machine is?

    I *guess *you're trying to do something like "If endian-ness of incoming
    data is same as endian-ness of current machine, read the data byte for
    byte; else byte-swap incoming data".  But why not take the portable
    approach and just do as Rob suggested?

    Let's say your data stream has a little-endian-encoded 32-bit integer.
    Here's how to extract it (assuming unsigned bytes):
    i = (data[0]<<0) | (data[1]<<8) | (data[2]<<16) | (data[3]<<24);
    If it's big-endian, here's how to extract it:
    i = (data[3]<<0) | (data[2]<<8) | (data[1]<<16) | (data[0]<<24);
    Again, I'm not saying you're wrong, just that I don't understand.

    -- L






    On Tuesday, September 18, 2012 8:24:23 AM UTC-4, ris wrote:

    You are receiving long and heartfelt applause *thumbsup*
    On Tuesday, September 18, 2012 1:20:48 PM UTC+1, Larry Clapp wrote:

    Is this what you're looking for?
    func main() {
    var i uint32 = 0x12345678;
    x := (*[4]byte)(unsafe.Pointer(&i))
    for _, xn := range *x {
    fmt.Printf("%x\n", xn)
    }
    }
    =>
    78
    56
    34
    12
    -- L
    On Tuesday, September 18, 2012 8:08:27 AM UTC-4, ris wrote:

    I don't understand what to make of this as I have an unsafe.Pointer "p"
    pointing to an int and I have a variable "i" which is a byte offset from
    the first byte of the int.
    The problem is go doesn't allow me to add the offset to the pointer and
    I just don't know how to otherwise get at the internal bytes of the int.
    On Tuesday, September 18, 2012 12:59:41 PM UTC+1, kortschak wrote:

    *(*[4]byte)(unsafe.Pointer(i))
    On 18/09/2012, at 9:26 PM, "ris" wrote:
    See earlier posting of my code. Doing so allows me to get first byte
    but not subsequent ones.
    On Tuesday, September 18, 2012 12:26:30 PM UTC+1, David Symonds wrote:
    On Tue, Sep 18, 2012 at 9:14 PM, ris wrote:

    Fair enough. Incidentally, does anyone know how to tell the
    endian-ness of a
    machine using go?
    We've told you multiple times: cast an integer to a byte array via
    unsafe.Pointer.
    --
    --
  • Roger peppe at Sep 18, 2012 at 1:03 pm

    On 18 September 2012 13:48, ris wrote:
    I run a little utility on that machine to tell me which endian-ness it
    is.
    Why not just write it in C? It's only two lines of code. :-)

    --
  • Ris at Sep 18, 2012 at 2:40 pm
    Grrrrrrrrrrr
    On Sep 18, 1:57 pm, roger peppe wrote:
    On 18 September 2012 13:48, ris wrote:

    I run a little utility on that machine to tell me which endian-ness it
    is.
    Why not just write it in C? It's only two lines of code. :-)
    --
  • Rémy Oudompheng at Sep 18, 2012 at 1:04 pm

    On 2012/9/18 ris wrote:
    So, I have some binary data.

    I know which machine it came from but I don't know the endian-ness of
    that machine.

    I run a little utility on that machine to tell me which endian-ness it
    is.
    Are you using gccgo? I think the architectures supported by "the main
    Go distribution" are all little-endian.

    Rémy.

    --
  • Ris at Sep 18, 2012 at 1:25 pm
    Please, let us never talk of endian-ness again!

    I have a nice little snippet of go code with examples of casting and array
    ranging which will do nicely as I explore the ramifications of this
    language.

    func main() {
         var x int = 0x012345678
         var bp *[4]byte = (*[4]byte)(unsafe.Pointer(&x))
         var endian string = "unknown"

         if 0x01 == bp[0] {
             endian = "big"
         } else if (0x78 & 0xff) == (bp[0] & 0xff) {
             endian = "little"
         } else {
             for _, b := range bp {
                 fmt.Printf("%x\n", b)
             }
         }
         fmt.Printf("%s endian (int is %d bits)\n", endian, unsafe.Sizeof(x)*8)
    }

    On Tuesday, September 18, 2012 2:04:04 PM UTC+1, Rémy Oudompheng wrote:
    On 2012/9/18 ris <ris.w...@gmail.com <javascript:>> wrote:
    So, I have some binary data.

    I know which machine it came from but I don't know the endian-ness of
    that machine.

    I run a little utility on that machine to tell me which endian-ness it
    is.
    Are you using gccgo? I think the architectures supported by "the main
    Go distribution" are all little-endian.

    Rémy.
    --
  • Roger peppe at Sep 18, 2012 at 1:40 pm

    On 18 September 2012 14:25, ris wrote:
    Please, let us never talk of endian-ness again!

    I have a nice little snippet of go code with examples of casting and array
    ranging which will do nicely as I explore the ramifications of this
    language.

    func main() {
    var x int = 0x012345678
    var bp *[4]byte = (*[4]byte)(unsafe.Pointer(&x))
    var endian string = "unknown"
    FWIW you can use := above, which is a little
    more idiomatic. Also, it's worth not assuming
    that int is 4 bytes.

         x := 0x012345678
         bp := (*[unsafe.Sizeof(x)]byte)(unsafe.Pointer(&x))
         endian := "unknown"

    --
  • Ris at Sep 18, 2012 at 1:41 pm
    Yep. Even better.
    On Tuesday, September 18, 2012 2:35:16 PM UTC+1, rog wrote:
    On 18 September 2012 14:25, ris <ris.w...@gmail.com <javascript:>> wrote:
    Please, let us never talk of endian-ness again!

    I have a nice little snippet of go code with examples of casting and array
    ranging which will do nicely as I explore the ramifications of this
    language.

    func main() {
    var x int = 0x012345678
    var bp *[4]byte = (*[4]byte)(unsafe.Pointer(&x))
    var endian string = "unknown"
    FWIW you can use := above, which is a little
    more idiomatic. Also, it's worth not assuming
    that int is 4 bytes.

    x := 0x012345678
    bp := (*[unsafe.Sizeof(x)]byte)(unsafe.Pointer(&x))
    endian := "unknown"
    --
  • Francisco Souza at Sep 18, 2012 at 12:32 pm
    func main() {
      var x int = 0x012345678
      var p unsafe.Pointer = unsafe.Pointer(&x)
      var bp *[4]byte = (*[4]byte)(p)

      var endian string = "unknown"

      if 0x01 == bp[0] {
       endian = "big"
      } else if (0x78 & 0xff) == (bp[0] & 0xff) {
       endian = "little"
      } else {
       var i uintptr = 0
       for i = 0; i < unsafe.Sizeof(x); i++ {
        fmt.Printf("byte[%d] = %d\n", i, bp[i])
       }
      }
      fmt.Printf("%s endian (int is %d bits)\n", endian, unsafe.Sizeof(x)*8)
    }
    On Tue, Sep 18, 2012 at 9:08 AM, ris wrote:

    I don't understand what to make of this as I have an unsafe.Pointer "p"
    pointing to an int and I have a variable "i" which is a byte offset from the
    first byte of the int.

    The problem is go doesn't allow me to add the offset to the pointer and I
    just don't know how to otherwise get at the internal bytes of the int.
    On Tuesday, September 18, 2012 12:59:41 PM UTC+1, kortschak wrote:

    *(*[4]byte)(unsafe.Pointer(i))

    On 18/09/2012, at 9:26 PM, "ris" wrote:

    See earlier posting of my code. Doing so allows me to get first byte but
    not subsequent ones.
    On Tuesday, September 18, 2012 12:26:30 PM UTC+1, David Symonds wrote:
    On Tue, Sep 18, 2012 at 9:14 PM, ris wrote:

    Fair enough. Incidentally, does anyone know how to tell the endian-ness
    of a
    machine using go?
    We've told you multiple times: cast an integer to a byte array via
    unsafe.Pointer.
    --
    --


    --
    ~f

    --
  • Francisco Souza at Sep 18, 2012 at 12:34 pm
    sorry, looks like my email was out of sync :(
    On Tue, Sep 18, 2012 at 9:31 AM, Francisco Souza wrote:
    func main() {
    var x int = 0x012345678
    var p unsafe.Pointer = unsafe.Pointer(&x)
    var bp *[4]byte = (*[4]byte)(p)

    var endian string = "unknown"

    if 0x01 == bp[0] {
    endian = "big"
    } else if (0x78 & 0xff) == (bp[0] & 0xff) {
    endian = "little"
    } else {
    var i uintptr = 0
    for i = 0; i < unsafe.Sizeof(x); i++ {
    fmt.Printf("byte[%d] = %d\n", i, bp[i])
    }
    }
    fmt.Printf("%s endian (int is %d bits)\n", endian, unsafe.Sizeof(x)*8)
    }
    On Tue, Sep 18, 2012 at 9:08 AM, ris wrote:

    I don't understand what to make of this as I have an unsafe.Pointer "p"
    pointing to an int and I have a variable "i" which is a byte offset from the
    first byte of the int.

    The problem is go doesn't allow me to add the offset to the pointer and I
    just don't know how to otherwise get at the internal bytes of the int.
    On Tuesday, September 18, 2012 12:59:41 PM UTC+1, kortschak wrote:

    *(*[4]byte)(unsafe.Pointer(i))

    On 18/09/2012, at 9:26 PM, "ris" wrote:

    See earlier posting of my code. Doing so allows me to get first byte but
    not subsequent ones.
    On Tuesday, September 18, 2012 12:26:30 PM UTC+1, David Symonds wrote:
    On Tue, Sep 18, 2012 at 9:14 PM, ris wrote:

    Fair enough. Incidentally, does anyone know how to tell the endian-ness
    of a
    machine using go?
    We've told you multiple times: cast an integer to a byte array via
    unsafe.Pointer.
    --
    --


    --
    ~f

    --


    --
    ~f

    --
  • Ris at Sep 18, 2012 at 1:19 pm
    Yes, I see. That works nicely too. The different style of declaration
    between c and go is going to mess with my chi for a while.
    On Sep 18, 1:32 pm, Francisco Souza wrote:
    func main() {
    var x int = 0x012345678
    var p unsafe.Pointer = unsafe.Pointer(&x)
    var bp *[4]byte = (*[4]byte)(p)

    var endian string = "unknown"

    if 0x01 == bp[0] {
    endian = "big"
    } else if (0x78 & 0xff) == (bp[0] & 0xff) {
    endian = "little"
    } else {
    var i uintptr = 0
    for i = 0; i < unsafe.Sizeof(x); i++ {
    fmt.Printf("byte[%d] = %d\n", i, bp[i])
    }
    }
    fmt.Printf("%s endian (int is %d bits)\n", endian, unsafe.Sizeof(x)*8)









    }
    On Tue, Sep 18, 2012 at 9:08 AM, ris wrote:

    I don't understand what to make of this as I have an unsafe.Pointer "p"
    pointing to an int and I have a variable "i" which is a byte offset from the
    first byte of the int.
    The problem is go doesn't allow me to add the offset to the pointer and I
    just don't know how to otherwise get at the internal bytes of the int.
    On Tuesday, September 18, 2012 12:59:41 PM UTC+1, kortschak wrote:

    *(*[4]byte)(unsafe.Pointer(i))
    On 18/09/2012, at 9:26 PM, "ris" wrote:
    See earlier posting of my code. Doing so allows me to get first byte but
    not subsequent ones.
    On Tuesday, September 18, 2012 12:26:30 PM UTC+1, David Symonds wrote:
    On Tue, Sep 18, 2012 at 9:14 PM, ris wrote:

    Fair enough. Incidentally, does anyone know how to tell the endian-ness
    of a
    machine using go?
    We've told you multiple times: cast an integer to a byte array via
    unsafe.Pointer.
    --
    --
    --
    ~f
    --
  • John Asmuth at Sep 18, 2012 at 1:42 pm
    Not true. Once you get the pointer, you can do arithmetic on it for
    whatever you like. Also what kortschak suggests.
    On Tuesday, September 18, 2012 7:56:36 AM UTC-4, ris wrote:

    See earlier posting of my code. Doing so allows me to get first byte but
    not subsequent ones.
    On Tuesday, September 18, 2012 12:26:30 PM UTC+1, David Symonds wrote:
    On Tue, Sep 18, 2012 at 9:14 PM, ris wrote:

    Fair enough. Incidentally, does anyone know how to tell the endian-ness of a
    machine using go?
    We've told you multiple times: cast an integer to a byte array via
    unsafe.Pointer.
    --
  • Rob Pike at Sep 18, 2012 at 3:06 pm
    var x uint32 = 0x01020304
      switch *(*byte)(unsafe.Pointer(&x)) {
      case 0x01:
       println("big")
      case 0x04:
       println("little")
      }

    -rob

    --
  • Jan Mercl at Sep 18, 2012 at 3:06 pm
  • Ohir at Feb 24, 2016 at 1:21 pm
    Beware, it is very old thread (2012). I'm posting an answer for others who
    will search for this topic.
    On Tuesday, September 18, 2012 at 1:14:52 PM UTC+2, Ris Wright wrote:

    does anyone know how to tell the endian-ness of a machine using go?
    import "runtime"

         switch runtime.GOARCH {
         case "mips64", "ppc64":
             ourOrderIsBE = true
         }

    // case clause will need maintenace with more BE archs coming
    // list such architectures with:
    // grep -rn 'BigEndian\s\+=\s\+1' $GOROOT/src/runtime/*|cut -sf2 -d_|cut
    -d. -f1

    http://play.golang.org/p/t6QDLNV-jk

    --
    Wojciech S. Czarnecki
            ^oo^ OHIR-RIPE

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Wojciech S. Czarnecki at Feb 24, 2016 at 1:06 pm
    [Beware. This is a revived thread from 2012. I hope an answer will be of use.]
    On Tuesday, September 18, 2012 at 1:14:52 PM UTC+2, Ris Wright wrote:

    Incidentally, does anyone know how to tell the endian-ness of a machine
    using go?
         switch runtime.GOARCH {
         case "mips64", "ppc64":
             ourOrderIsBE = true
         }

    It will need maintenance with any new BE arch coming into the tree.

    You can list all BE archs with:

    grep -rn 'BigEndian\s\+=\s\+1' $GOROOT/src/runtime/internal/sys/ *|\
    cut -sf2 -d_|cut -d. -f1

    [Check your GOROOT with go env command, it is not set by default.]

    http://play.golang.org/p/t6QDLNV-jk


    --
    Wojciech S. Czarnecki
            ^oo^ OHIR-RIPE

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Marvin Renich at Feb 24, 2016 at 2:39 pm

    * Wojciech S. Czarnecki [160224 08:07]:
    On Tuesday, September 18, 2012 at 1:14:52 PM UTC+2, Ris Wright wrote:
    Incidentally, does anyone know how to tell the endian-ness of a machine
    using go?
    switch runtime.GOARCH {
    case "mips64", "ppc64":
    ourOrderIsBE = true
    }

    It will need maintenance with any new BE arch coming into the tree.
    This isn't very maintainable. Would the Go devs be amenable to adding
    two new constants to the runtime package, one for endian-ness and one
    for size of int?

    Actually, I think a better approach, since these are constant at compile
    time, would be to add build constraints littleEndian, bigEndian,
    intIs32, and intIs64. (Does Go support any 16-bit architectures?)

    If someone really needed these at runtime, they could be built with the
    tags, but they would more likely simply build the appropriate variant of
    the code, obviating the need for a runtime test.

    ...Marvin

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ian Lance Taylor at Feb 24, 2016 at 2:51 pm

    On Wed, Feb 24, 2016 at 6:39 AM, Marvin Renich wrote:
    * Wojciech S. Czarnecki [160224 08:07]:
    On Tuesday, September 18, 2012 at 1:14:52 PM UTC+2, Ris Wright wrote:
    Incidentally, does anyone know how to tell the endian-ness of a machine
    using go?
    switch runtime.GOARCH {
    case "mips64", "ppc64":
    ourOrderIsBE = true
    }

    It will need maintenance with any new BE arch coming into the tree.
    This isn't very maintainable. Would the Go devs be amenable to adding
    two new constants to the runtime package, one for endian-ness and one
    for size of int?

    Actually, I think a better approach, since these are constant at compile
    time, would be to add build constraints littleEndian, bigEndian,
    intIs32, and intIs64. (Does Go support any 16-bit architectures?)

    If someone really needed these at runtime, they could be built with the
    tags, but they would more likely simply build the appropriate variant of
    the code, obviating the need for a runtime test.
    You can get the size of int as a constant via unsafe.Sizeof(0).

    I assert that most code that relies on native endianness is
    unportable, unsafe, or buggy. There are various ways you can
    determine the endianness at run time if you need it (for example, see
    golang.org/x/net/ipv4/helper.go). I don't think it's necessary for
    the Go toolchain to provide build constraints, or simpler methods, for
    this.

    Ian

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Peter Waller at Feb 24, 2016 at 2:58 pm

    On 24 February 2016 at 14:51, Ian Lance Taylor wrote:
    I assert that most code that relies on native endianness is
    unportable, unsafe, or buggy. There are various ways you can
    determine the endianness at run time if you need it (for example, see
    golang.org/x/net/ipv4/helper.go). I don't think it's necessary for
    the Go toolchain to provide build constraints, or simpler methods, for
    this.

    You also remind me of Rob's old blog post, which I guess may have been
    discussed when this thread was originally started.

    It also provides a platform independent solution that covers many cases.

    http://commandcenter.blogspot.co.uk/2012/04/byte-order-fallacy.html

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Wojciech S. Czarnecki at Feb 24, 2016 at 3:19 pm
    Dnia 2016-02-24, o godz. 06:51:16
    Ian Lance Taylor <iant@golang.org> napisał(a):
    switch runtime.GOARCH {
    case "mips64", "ppc64":
    ourOrderIsBE = true
    }
    I assert that most code that relies on native endianness is
    unportable, unsafe, or buggy.
    Nonetheless, in corner cases you just need to know. I revived this thread
    to show a solution without relying on unsafe.
    I don't think it's necessary for the Go toolchain to provide build
    constraints, or simpler methods, for this.
    I agree. One who needs such arch dependent details may simply use
    already defined build constraints: simply put whatever arch dependent things
    you want to inspect in an _arch.go file (possibly generated one).
    Ian
    --
    Wojciech S. Czarnecki
            ^oo^ OHIR-RIPE

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Marvin Renich at Feb 24, 2016 at 3:40 pm

    * Ian Lance Taylor [160224 09:51]:
    On Wed, Feb 24, 2016 at 6:39 AM, Marvin Renich wrote:
    Actually, I think a better approach, since these are constant at compile
    time, would be to add build constraints littleEndian, bigEndian,
    intIs32, and intIs64. (Does Go support any 16-bit architectures?)
    You can get the size of int as a constant via unsafe.Sizeof(0).

    I assert that most code that relies on native endianness is
    unportable, unsafe, or buggy.
    I think the fact that golang.org/x/net/ipv4/helper.go exists is a
    counter-example.
    There are various ways you can
    determine the endianness at run time if you need it (for example, see
    golang.org/x/net/ipv4/helper.go). I don't think it's necessary for
    the Go toolchain to provide build constraints, or simpler methods, for
    this.
    I can agree with this. Consider my suggestion retracted.

    Thanks...Marvin

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Ian Lance Taylor at Feb 24, 2016 at 6:27 pm

    On Wed, Feb 24, 2016 at 7:39 AM, Marvin Renich wrote:
    * Ian Lance Taylor [160224 09:51]:
    On Wed, Feb 24, 2016 at 6:39 AM, Marvin Renich wrote:
    Actually, I think a better approach, since these are constant at compile
    time, would be to add build constraints littleEndian, bigEndian,
    intIs32, and intIs64. (Does Go support any 16-bit architectures?)
    You can get the size of int as a constant via unsafe.Sizeof(0).

    I assert that most code that relies on native endianness is
    unportable, unsafe, or buggy.
    I think the fact that golang.org/x/net/ipv4/helper.go exists is a
    counter-example.
    Well, sure. But, 1) when I say "most" instead of "all" I'm accepting
    that counter-examples exist; 2) I personally think that the code there
    is a reflection of bugs in the low-level IPv4 interface provided by
    some *BSD systems.

    Ian

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Marvin Renich at Feb 24, 2016 at 7:58 pm

    * Ian Lance Taylor [160224 13:27]:
    I think the fact that golang.org/x/net/ipv4/helper.go exists is a
    counter-example.
    Well, sure. But, 1) when I say "most" instead of "all" I'm accepting
    that counter-examples exist; 2) I personally think that the code there
    is a reflection of bugs in the low-level IPv4 interface provided by
    some *BSD systems.
    After reading Rob's post referred to in an earlier message, I am willing
    to concede this point entirely; he is extremely convincing.

    ...Marvin

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

People

Translate

site design / logo © 2021 Grokbase