Gloss certainly handles variable-size frames: the wiki even has a
number of examples of the different built-in ways you can encode the
size, and you can write a custom size-detector if none of those suit
you. As for streams vs channels, my understanding is that the java.nio
package Gloss uses was introduced as a higher-performance option
compared to java.io. Like most people, I don't think I've ever really
understood nio, but I suspect that Zack does and chose it for a
reason.
My point is that Gloss is a fantastic library focused at exactly your
problem domain, and you should probably do some more investigating
before dismissing it as unsuitable (I infer you haven't done much
research because you think it isn't focused and that it's not possible/
easy to handle variable-length arrays).
For example, here's how to do your example with Gloss:
(def codec (compile-frame {:type :uint32 :data
(repeated :uint32 :prefix :uint32)}))
(encode codec {:type 3 :data [1 2 3]})
;=> (#<HeapByteBuffer java.nio.HeapByteBuffer[pos=0 lim=16 cap=16]>
#<HeapByteBuffer java.nio.HeapByteBuffer[pos=0 lim=4 cap=4]>)
(decode codec *1)
;=> {:type 3, :data [1 2 3]}
If you really-really want the :size to be present in the Clojure data
structures (rather than just ignoring it as an artifact of the
protocol), you can do that by messing with a header in the codec. You
can also make type an enum instead of just an int32, so that Gloss
converts it into a meaningful keyword for you instead of having to
deal with the number yourself. I ignored both of these issues for
simplicity and due to my ignorance of your actual protocol.
Perhaps more importantly than any of this, I suspect you're relying on
the maps you use to retain their order - that is, you assume because
you specify type, then size, then data, they will be encoded in that
order. But maps are not sorted; it's only a coincidence that for small
maps, on the current version of Clojure, they happen to be in the
order you specified them. If you encoded a structure with ten or more
fields, your library would suddenly break because the fields would be
ordered wrong. For regular maps, Gloss sorts the maps alphabetically
by key - in my example it encoded data before type. This is consistent
across versions of Clojure and all map sizes (great!) but not what you
actually want when you interoperate with C. So instead, Gloss allows
you to give it a vector of pairs (or something like that), or a
special map that promises to retain the order you specify.
This is an example of the sort of tricky edge-case (or scalability
issue) that would surely slip by me when implementing my own binary-
encoding library, but which has already been addressed by Gloss. By
choosing a mature and well-known library instead of making your own,
you get more features and fewer bugs.
On Feb 5, 4:09 pm, Russell Christopher wrote:I did look at Gloss before writing Marshal and decided against using it in
our application suite, I wanted something more focused utilizing
InputStream/OutputStream interface and able to handle variable sized
arrays/strings (size of array or string specified in packet).
On Sun, Feb 5, 2012 at 6:10 PM, Alan Malloy wrote:Zack Tellman's library Gloss is excellent for this, if a bit confusing
to understand at first.
On Feb 5, 2:50 pm, russellc wrote:
I needed a library to handle marshaling application defined (C
struct) binary TCP/IP protocol packets in a Clojure client app
replacing a legacy C++ client and there didn't seem to be an existing
library that performed as I needed. So, I've written a Marshal library
that marshals from the network to/from Clojure.
Github:http://github.com/russellc/Marshal
Clojars:https://clojars.org/marshal
//C header file definition
struct packet {
unsigned long type;
unsigned long size;
long data[1];
(require '[marshal.core :as m])
(def packet (m/struct :type m/uint32 :size m/uint32 :data (m/array m/
sint32 :size)))
(m/write output-stream packet {:type 1 :size 2 :data [1 -1]})
(m/read input-stream packet)
=> {:type 1 :size 2 :data [1 -1]}
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en --
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
