On Sunday, November 24, 2013 1:11:39 PM UTC-7, simon place wrote:
no existing code is effected in that it cant make these as is, you have to
ask for a capacity beyond the current backing array, which is an error
currently, but you will be able to take the new AOA backed slices and
fire them into existing code, (the real point) but surely the slice code is
localised and can hide its implementation here, so its only the externally
exposed implementation calls that are the issue.
Currently it's specified as a runtime error (and it never can be purely a
compile-time error). As such, you can't change out-of-bounds slicing
semantics while being transparently compatible with all existing programs.
You could potentially get away with adding a new builtin to cover this
functionality, or modifying append to use the new mechanism under certain
conditions. Unless people *really* know what they're doing, they don't
slice into capacity (or in your case, slice past capacity) to indicate that
they want to append... they use append. I believe the most
beginner-friendly outlet for the idea is to modify append, even if there
were no definite compatibility issues arising from slicing past capacity.
but aren't these generally going to be fixed length, and so should be
using arrays not slices, although i can see that that wont always be the
case, handing over a particular part of an array, using a slice to define
it, could be unfortunately common.
Not sure what you're getting at here -- the difference between slices and
arrays in Go is not that one has fixed length, but rather than one is a
reference and the other is a direct value. If you say byte, it'll
generally be stack allocated (unless a pointer to it *could* escape to the
heap, in which case it is allocated the same way that a call to make would
be). Conversely, a non-escaping make(byte, 32) will be stack allocated,
since its size is known at compile time.
Slices should really be used when you're using a variable
length paradyne and then, externally, going thought a fixed length buffer,
so generally going to have to be chunking somewhere anyway, chunking a
array of arrays doesn't seem that difficult in code that is already
re-chunking, isn't it basically just a loop?
Because of the above, the choice between slices and arrays here isn't
particularly important. Being a direct value, arrays can be used as map
keys or tested for equality (assuming that the element type is comparable),
while slices cannot on the language level.
i see, so better for novice users, but not experienced ones, yes would
*Marginally* better, in a "saving them from themselves" sense. If they
don't want to move beyond that level of insufficiency, then this is hardly
the only place their code will suffer. On the other hand, for experienced
users intentionally writing suboptimal code in order to maximize
"programmer productivity", that part of the code will surely be
inconsequential, with no noticeable difference between optimal and
suboptimal (regardless of the underlying mechanism).
The other tradeoff with this is that because of the above-noted
incompatibilities, and the necessary special handling of anything that will
be passed to C or the kernel or whatever, any gains for novices may become
confusion points for intermediates, kind of like C++ has to deal with
regarding their handful of subtle reference types and memory management
While Go may be a good learning language, it was designed by experienced
engineers for experienced engineers: for a given language modification to
be warranted, it has to be demonstrated that it is useful to seasoned
programmers, or at least doesn't make their lives any harder. With this
being proposed as an automatic mechanism, in many cases it can make
perfectly legitimate low level uses of slices harder or less predictable
without providing anything that low level programmers aren't already
providing for themselves; if it weren't an automatic mechanism, it wouldn't
make veterans' lives' harder, but it would no longer be as beneficial to
novices. There may be a middle ground, but it has yet to be demonstrated.
but, there will always be cases, where copying is massively slower, or
even impossible, requiring custom solutions which then bar direct use with
std libs, and other code.
Do any such cases come to mind? When append needs to allocate and copy a
slice of *any* type, a highly optimized, hardware accelerated copy is
used (it doesn't matter that the slice/array element type might not be
i was thinking any type, didn't see what difference it make for this.
To confirm that copy performance is high irrespective of element type, and
doesn't need any recently invented instructions to achieve that
>. Even on an
old netbook, copies can churn 1GiB/s throughput, and with the memory on
that system being 2GiB, the largest possible copy that could remain
entirely within ram, on that machine, could occur in one second. Newer
and/or higher powered machines will have considerably faster performance in
this regard. In general, since append heuristically doubles capacity during
each growth cycle, and application will generally exhaust available ram
before before copy performance becomes problematic for non-critical
applications, and critical applications on expensive hardware where these
copies could be a problem will generally be written to preallocate the
necessary amount of space.
If there are no specific examples in which copying could be massively
slower or impossible, it sounds like this is now a hypothetical, rather
than practical or theoretical, issue.
so potentially much faster on these.
That's not what I was implying. And in the case of hardware lacking memory
virtualization capabilities, as long as tinygo, for example, is "the
operating system", then it should have absolutely no bearing on
performance; all else being equal, the lack of memory virtualization (or at
least the absence of memory traps) might actually make memory accesses
(including copying) faster.
in a common scenario, processing a stream, i can envisage only very few
slices backed by AOA, and they come and go, ie you have them for a
transition over the boundary and then go back to normal, transparently, the
implementation effectively deals with the interface between fixed length
buffers and continuous processing. (probably really should have demo code
Any kind of efficiency in stream processing really needs to operate within
a fixed (or upper-bounded) window size, or at least should eagerly process
data as it flows through in anticipation that processing will be
successful. This isn't an issue with copying (which doesn't apply to your
proposal), or allocations (which applies equally to your proposal), but
rather unbounded growth; I would recommend neither unbounded use of your
proposal, nor unbounded use of append as a solution for any kind of
continuous stream processing with variable size inputs; bounded uses of
either approach are appropriate and have no meaningful differences from
each other. As an example, if you're parsing data out of lines with
arbitrary length, with one record per line, the naive approach would be to
use something like bufio.Reader's ReadBytes('\n') to get the line, then
process the data within that line. The non-naive approach would be to
process data in fixed-sized chunks, and check each chunk for the presence
of '\n'. Usually there are more convenient ways to achieve this effect
while still maintaining both control and efficiency, such that manual
management of chunks is not an application concern.
would those custom realizations be as maintainable, reusable, and as
By definition they are not intended for reuse if they are purpose-built for
a specific task, though as mentioned before, while reusability is favored
in general contexts, it's more common in Go than in some other "large
scale" languages to favor task-specific implementations at the expense of
The maintainability and readability/explainability of such custom code
depends both on how well it was written, and how far the already-available
solution's assumptions deviate from the task at hand. The Go stdlib in many
places avoids convenient (for the user) assumptions in order to keep
functionality simple and flexible, so for many tasks, much or all of the
implementation can be found within the stdlib or similarly written
libraries even if it means the application code needs to do some of its own
initialization to make up for the lack of said assumptions. However,
existing "reusable" code which needs significant shimming to solve a
specific task may require more documentation just to explain *why all the
shims exist* than a well designed custom solution would need to document
the what the whole of the code is doing. Similarly, mis-assuming general
solutions often must be replaced with other mis-assuming general solutions
(and subsequently re-shimmed) as application requirements change, leading
to considerably more overall effort than just tweaking custom solutions.
Also remember that I claimed that there are many situations in which it is
quicker to completely write a correct custom solution than it is find,
evaluate, and integrate general solutions.
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to email@example.com.
For more options, visit https://groups.google.com/groups/opt_out.