[go-nuts] Find ID during real time data streaming efficiently

I would suggest to profile the app to identify the slow code path first. Go
makes profiling really easy, have a look at
http://blog.golang.org/profiling-go-programs

I had done that, it consumes 26%. That's why I asked if there is a better
way to implement it.


Total: 430ms 800ms (flat, cum) 26.40%
     151 . 10ms func searchUID(r *bufio.Reader, uid [16]byte) (n int, Size int32, Type int32, err error) {
      152 . . var seq [16]byte
      153 . .
      154 . . n = 0
      155 280ms 410ms for seq != uid {
      156 50ms 100ms copy(seq[:15], seq[1:])
      157 30ms 190ms seq[15], err = r.ReadByte()
      158 50ms 50ms if err != nil {
      159 . . fmt.Println("Error during ReadByte (UID Search)", err)
      160 . . return n, 0, 0, err
      161 . . }
      162 . .
      163 . . // fmt.Printf("%x, %x, %v \n", uid, seq, n)
      164 . . if n > 4096 {
      165 . . fmt.Printf("UID not found, Err :Out of search range \n")
      166 . . return n, 0, 0, errors.New("UID not found, Out of search range")
      167 . . }
      168 . . n = n + 1
      169 . . }
      170 . .
      171 . . // fmt.Printf("Search end successfully in %d iterations! (%x vs %x)\n", n, uid, seq)
      172 . . // Read the other parts of the header
      173 . 10ms err = binary.Read(r, binary.LittleEndian, &Size)
      174 20ms 20ms err = binary.Read(r, binary.LittleEndian, &Type)
      175 . .
      176 . . return n, Size, Type, nil
      177 . 10ms }

You might want to try a few simple to implement possible optimizations:
- use circular buffer to avoid copying (i.e. seq[k] = ReadByte(), k = (k +
1) % 16)
- custom function to compare arrays: compare first byte in uid and seq and
if they are the same compare the rest (taking into account circular buffer).

Alternatively you can use more optimal algorithm to search a sub-string in
the stream.
E.g. https://en.wikipedia.org/wiki/Knuth%E2%80%93Morris%E2%80%93Pratt_algorithm

Another idea: use two uint64 instead of [16]byte inside the function. The
comparison should be much faster. Construct these two numbers from the last
16 bytes using bit shift operations.

Also you can try using Peek (http://golang.org/pkg/bufio/#Reader.Peek) and
Buffered (http://golang.org/pkg/bufio/#Reader.Buffered) methods to avoid
reading one byte at a time.