FAQ
Recently, I was interested in using some hardware accelerated crypto
operations through Linux's AF_ALG interface. It makes a big difference on
reduced resource environments, like ARM.

For example, ARM architectures with the Kirkwood hardware crypto support
chip (with appropriate kernel extensions enabled), can hash SHA1 at 70MB/s
by calling into the kernel with AF_ALG sockets, whereas the Go stdlib on
the same ARM chip can do about 5 MB/s.

It's actually pretty simple to set up, but it left me with a couple of
questions about holes in the syscall standard library.

Here's my
implementation: https://github.com/jtolds/go-af-alg/blob/master/sha1/sha1_linux.go

Specifically - I'm wondering what the Go community's opinion would be of me
pushing patches to the syscall library that do any subset of these things:

1) add support for a sockaddr_alg type on Linux.
2) add support for a generic raw sockaddr type that can actually be used by
source outside of the syscall package.
3) provide a version of accept that does not return or ask for a sockaddr,
or alternatively, does, but doesn't fail if it doesn't understand the type.
4) add the send() call.
5) extend the sendto() call to allow a nil sockaddr.

The motivation for the above requests is documented in comments in the
above implementation link. As I said before, that link can get 14x faster
throughput on SHA1 operations on my ARM device (after patching my kernel
with Phil Sutter's mv_crypto DMA extensions). The AF_ALG interface the
kernel supports is of course not at all restricted to SHA1. It can do all
sorts of encryption, hashing, and other crypto operations. SHA1 is
currently my motivating example.

Thoughts?

-JT

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search Discussions

  • Mikio Hara at Mar 28, 2014 at 6:02 am
    The net package is not appropriate place for us, probably, because,
    - AF_ALG and friends (AF_KEY, AF_IEEE802154, AF_NFC, AF_NETLINK on
    Linux, AF_ROUTE, AF_IEEE80211 on BSDs) are really
    feature/platform-specific,
    - what those stuff really want doesn't stay in the net package. I
    guess it stays in the runtime package as runtime-integrated network
    poller.

    So, just a guess:
    1) adding platform-specific stuff into syscall is perhaps okay unless
    maintainers dislike it.
    2) pulling up or injecting feature/platform-specific IO operations
    from/to net package would require hard justification.
    3) a new public interface to runtime-integrated network poller might
    be a solution; we can create each AF_XXX package independently, but to
    be honest, not sure.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • JT Olds at Mar 28, 2014 at 6:10 am
    Yeah, I'm only proposing syscall additions, no pkg/net changes whatsoever.
    On Friday, March 28, 2014 12:02:09 AM UTC-6, Mikio Hara wrote:

    The net package is not appropriate place for us, probably, because,
    - AF_ALG and friends (AF_KEY, AF_IEEE802154, AF_NFC, AF_NETLINK on
    Linux, AF_ROUTE, AF_IEEE80211 on BSDs) are really
    feature/platform-specific,
    - what those stuff really want doesn't stay in the net package. I
    guess it stays in the runtime package as runtime-integrated network
    poller.

    So, just a guess:
    1) adding platform-specific stuff into syscall is perhaps okay unless
    maintainers dislike it.
    2) pulling up or injecting feature/platform-specific IO operations
    from/to net package would require hard justification.
    3) a new public interface to runtime-integrated network poller might
    be a solution; we can create each AF_XXX package independently, but to
    be honest, not sure.
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Mikio Hara at Mar 28, 2014 at 6:19 am

    On Fri, Mar 28, 2014 at 3:10 PM, JT Olds wrote:

    Yeah, I'm only proposing syscall additions, no pkg/net changes whatsoever.
    But you need performance too. ;) I think feature/platform-specific
    types don't matter. We can hold such stuff in own packages. But socket
    IO operations do matter, introducing runtime code fragments to own
    packages would be a dragon.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • JT Olds at Mar 28, 2014 at 6:24 am

    But you need performance too. ;) I think feature/platform-specific
    types don't matter. We can hold such stuff in own packages. But socket
    IO operations do matter, introducing runtime code fragments to own
    packages would be a dragon.
    I guess I'm a little confused. The AF_ALG interface is very heavily
    platform-specific and not portable. The header types only exist on Linux. I
    wouldn't expect things that use it to compile on other platforms.

    The syscall library has 90% of what I need, it's just missing a few
    platform-specific things.

    Are you proposing a more general interface in the net package?

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Mikio Hara at Mar 28, 2014 at 6:52 am

    On Fri, Mar 28, 2014 at 3:24 PM, JT Olds wrote:

    I guess I'm a little confused. The AF_ALG interface is very heavily
    platform-specific and not portable. The header types only exist on Linux. I
    wouldn't expect things that use it to compile on other platforms.

    The syscall library has 90% of what I need, it's just missing a few
    platform-specific things.

    Are you proposing a more general interface in the net package?
    Nope, as you know there's a subtle difference btw
    syscall.Syscall(SYS_RECVFROM, ...) and syscall.Recvfrom. Perhaps your
    proposal would be rejected like:
    https://groups.google.com/d/msg/golang-nuts/B-meiFfkmH0/HiY_td7HMlEJ

    Also I'm just guessing what happens once you get what you want. If you
    need a bit good throughput for your AF_ALG package what would be a
    option? The difference btw net.TypedConn.Read and syscall.Recvfrom is
    pretty huge especially for performance and it comes from
    runtime-integrated network poller, well, that's about it.

    Sorry for the confusion.

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • Nick Craig-Wood at Mar 28, 2014 at 5:29 pm

    On 28/03/14 03:07, JT Olds wrote:
    Recently, I was interested in using some hardware accelerated crypto
    operations through Linux's AF_ALG interface. It makes a big difference
    on reduced resource environments, like ARM.

    For example, ARM architectures with the Kirkwood hardware crypto support
    chip (with appropriate kernel extensions enabled), can hash SHA1 at
    70MB/s by calling into the kernel with AF_ALG sockets, whereas the Go
    stdlib on the same ARM chip can do about 5 MB/s.
    Did you try go tip? I re-wrote the ARM sha1 routines in assembler which
    run very much quicker!

    Due for release with go 1.3

    https://codereview.appspot.com/56990044/

    --
    Nick Craig-Wood <nick@craig-wood.com> -- http://www.craig-wood.com/nick

    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.
  • JT Olds at Mar 28, 2014 at 7:32 pm
    Nick, thanks, that's great, I will try tip, but I know for sure the CPU on
    the device I'm using simply can't outperform the built in
    hardware-accelerated crypto operations. I have specialized hardware that
    does SHA1.
    On Friday, March 28, 2014 11:29:26 AM UTC-6, Nick Craig-Wood wrote:
    On 28/03/14 03:07, JT Olds wrote:
    Recently, I was interested in using some hardware accelerated crypto
    operations through Linux's AF_ALG interface. It makes a big difference
    on reduced resource environments, like ARM.

    For example, ARM architectures with the Kirkwood hardware crypto support
    chip (with appropriate kernel extensions enabled), can hash SHA1 at
    70MB/s by calling into the kernel with AF_ALG sockets, whereas the Go
    stdlib on the same ARM chip can do about 5 MB/s.
    Did you try go tip? I re-wrote the ARM sha1 routines in assembler which
    run very much quicker!

    Due for release with go 1.3

    https://codereview.appspot.com/56990044/

    --
    Nick Craig-Wood <ni...@craig-wood.com <javascript:>> --
    http://www.craig-wood.com/nick
    --
    You received this message because you are subscribed to the Google Groups "golang-nuts" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
    For more options, visit https://groups.google.com/d/optout.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-nuts @
categoriesgo
postedMar 28, '14 at 3:07a
activeMar 28, '14 at 7:32p
posts8
users3
websitegolang.org

People

Translate

site design / logo © 2021 Grokbase