Grokbase › Groups › Go › golang-dev › December 2012
FAQ
Reviewers: rsc, iant, r, ken2,

Message:
Hello rsc@golang.org, iant@golang.org, r@golang.org, ken@golang.org (cc:
golang-dev@googlegroups.com),

I'd like you to review this change to
https://code.google.com/p/go


Description:
spec: constant indexes and array/slice sizes must fit into int64

also:
- consistently use 'indexes' rather than 'indices'

This CL refines earlier CLs 6725053 and 6699048.
Restricting the maximum value for integer indices
to 1<<63-1 ("MaxInt64") is reasonable and permits
more compile-time checks (and a simpler implementation).

The compilers have implemented similar restrictions
early on, so this is unlikely to cause existing
programs to break.

Please review this at https://codereview.appspot.com/6899045/

Affected files:
M doc/go_spec.html


Index: doc/go_spec.html
===================================================================
--- a/doc/go_spec.html
+++ b/doc/go_spec.html
@@ -791,8 +791,8 @@
The length of a string <code>s</code> (its size in bytes) can be
discovered using
the built-in function <a href="#Length_and_capacity"><code>len</code></a>.
The length is a compile-time constant if the string is a constant.
-A string's bytes can be accessed by integer indices 0 through
-<code>len(s)-1</code> (§<a href="#Indexes">Indexes</a>).
+A string's bytes can be accessed by integer <a href="#Indexes">indexes</a>
+0 through <code>len(s)-1</code>.
It is illegal to take the address of such an element; if
<code>s[i]</code> is the <code>i</code>'th byte of a
string, <code>&amp;s[i]</code> is invalid.
@@ -815,12 +815,13 @@
</pre>

<p>
-The length is part of the array's type and must be a
+The length is part of the array's type; it must be a
<a href="#Constant_expressions">constant expression</a> that evaluates to
a non-negative
-integer value. The length of array <code>a</code> can be discovered
+integer value fitting into an int64: 0 &lt;= length &lt=
9223372036854775807.
+The length of array <code>a</code> can be discovered
using the built-in function <a
href="#Length_and_capacity"><code>len</code></a>.
-The elements can be indexed by integer
-indices 0 through <code>len(a)-1</code> (§<a href="#Indexes">Indexes</a>).
+The elements can be addressed by integer <a href="#Indexes">indexes</a>
+0 through <code>len(a)-1</code>.
Array types are always one-dimensional but may be composed to form
multi-dimensional types.
</p>
@@ -850,8 +851,8 @@
Like arrays, slices are indexable and have a length. The length of a
slice <code>s</code> can be discovered by the built-in function
<a href="#Length_and_capacity"><code>len</code></a>; unlike with arrays it
may change during
-execution. The elements can be addressed by integer indices 0
-through <code>len(s)-1</code> (§<a href="#Indexes">Indexes</a>). The
slice index of a
+execution. The elements can be addressed by integer <a
href="#Indexes">indexes</a>)
+0 through <code>len(s)-1</code>. The slice index of a
given element may be less than the index of the same element in the
underlying array.
</p>
@@ -2498,12 +2499,20 @@
</p>

<p>
-For <code>a</code> of type <code>A</code> or <code>*A</code>
+If <code>a</code> is not a map:
+</p>
+<ul>
+ <li>the index <code>x</code> must be an integer value; it is <i>in
range</i> if <code>0 &lt;= x &lt; len(a)</code>,
+ otherwise it is <i>out of range</i></li>
+ <li>a <a href="#Constants">constant</a> index must not be negative and
fit into an <code>int64</code>:
+ <code>0 &lt;= x &lt;= 9223372036854775807</code>
+</ul>
+
+<p>
+Additionally, for <code>a</code> of type <code>A</code> or <code>*A</code>
where <code>A</code> is an <a href="#Array_types">array type</a>:
</p>
<ul>
- <li><code>x</code> must be an integer value; it is <i>in range</i> if
<code>0 &lt;= x &lt; len(a)</code>,
- otherwise it is <i>out of range</i></li>
<li>a <a href="#Constants">constant</a> index must be in range</li>
<li>if <code>a</code> is <code>nil</code> or if <code>x</code> is out of
range at run time,
a <a href="#Run_time_panics">run-time panic</a> occurs</li>
@@ -2515,9 +2524,6 @@
For <code>a</code> of type <code>S</code> where <code>S</code> is a <a
href="#Slice_types">slice type</a>:
</p>
<ul>
- <li><code>x</code> must be an integer value; it is <i>in range</i> if
<code>0 &lt;= x &lt; len(a)</code>,
- otherwise it is <i>out of range</i></li>
- <li>a <a href="#Constants">constant</a> index must not be negative</li>
<li>if the slice is <code>nil</code> or if <code>x</code> is out of range
at run time,
a <a href="#Run_time_panics">run-time panic</a> occurs</li>
<li><code>a[x]</code> is the slice element at index <code>x</code> and
the type of
@@ -2529,9 +2535,7 @@
where <code>T</code> is a <a href="#String_types">string type</a>:
</p>
<ul>
- <li><code>x</code> must be an integer value; it is <i>in range</i> if
<code>0 &lt;= x &lt; len(a)</code>,
- otherwise it is <i>out of range</i></li>
- <li>a <a href="#Constants">constant</a> index must not be negative, and
it must be in range
+ <li>a <a href="#Constants">constant</a> index must be in range
if the string <code>a</code> is also constant</li>
<li>if <code>x</code> is out of range at run time,
a <a href="#Run_time_panics">run-time panic</a> occurs</li>
@@ -2598,7 +2602,7 @@
<p>
constructs a substring or slice. The index expressions <code>low</code> and
<code>high</code> select which elements appear in the result. The result
has
-indices starting at 0 and length equal to
+<a href="#Indexes">indexes</a> starting at 0 and length equal to
<code>high</code>&nbsp;-&nbsp;<code>low</code>.
After slicing the array <code>a</code>
</p>
@@ -2631,13 +2635,13 @@
</pre>

<p>
-For arrays or strings, the indices <code>low</code> and <code>high</code>
are
+For arrays or strings, the indexes <code>low</code> and <code>high</code>
are
<i>in range</i> if <code>0 &lt;= <code>low</code> &lt;= <code>high</code>
&lt;= len(a)</code>,
otherwise they are <i>out of range</i>.
For slices, the upper index bound is the slice capacity
<code>cap(a)</code> rather than the length.
-A <a href="#Constant_expressions">constant</a> index must not be negative,
and if both indices
+A <a href="#Constant_expressions">constant</a> index must not be negative,
and if both indexes
are constant, they must satisfy <code>low &lt;= high</code>. If
<code>a</code> is <code>nil</code>
-or if the indices are out of range at run time, a <a
href="#Run_time_panics">run-time panic</a> occurs.
+or if the indexes are out of range at run time, a <a
href="#Run_time_panics">run-time panic</a> occurs.
</p>

<p>
@@ -3359,7 +3363,7 @@
<pre>
t.Mv(7)
T.Mv(t, 7)
-(T).Mv(t, t)
+(T).Mv(t, 7)
f1 := T.Mv; f1(t, 7)
f2 := (T).Mv; f2(t, 7)
</pre>
@@ -4985,8 +4989,9 @@

<p>
The size arguments <code>n</code> and <code>m</code> must be integer
values.
-A <a href="#Constants">constant</a> size argument must not be negative, and
-if both <code>n</code> and <code>m</code> are provided and are constant,
then
+A <a href="#Constants">constant</a> size argument must not be negative and
+fit into an <code>int64</code>: <code>0 &lt;= size &lt;=
9223372036854775807</code>.
+If both <code>n</code> and <code>m</code> are provided and are constant,
then
<code>n</code> must be no larger than <code>m</code>.
If <code>n</code> is negative or larger than <code>m</code> at run time,
a <a href="#Run_time_panics">run-time panic</a> occurs.
@@ -4995,6 +5000,7 @@
<pre>
s := make([]int, 10, 100) // slice with len(s) == 10, cap(s) == 100
s := make([]int, 1e3) // slice with len(s) == cap(s) == 1000
+s := make([]int, 1&lt;&lt;63) // illegal: len(s) &gt;
9223372036854775807
s := make([]int, 10, 0) // illegal: len(s) > cap(s)
c := make(chan int, 10) // channel with a buffer size of 10
m := make(map[string]int, 100) // map with initial space for 100 elements

Search Discussions

  • Rob Pike at Dec 6, 2012 at 9:30 pm ⇧
    The plural of index in a mathematical context is 'indices'. The plural
    of index in a catalog context is 'indexes'. The correct plural in this
    document is 'indices'. I have said this before.

    That aside, I'm unsure about this change. First question: Why int64
    rather than int? After all, addressing only goes up to the maximum
    positive value representable in an int.

    You're missing a semicolon from one of your &lt;s.

    -rob
  • Russ Cox at Dec 6, 2012 at 9:38 pm ⇧
    We have been going back and forth on indices and indexes. Indexes is a
    common alternative spelling, matches the heading in the spec, and is
    found when you search for index.

    The rule used to be 'has to fit in int', but that means x[1<<40]
    compiles on amd64 but does not compile on 386. We were trying to make
    whether a Go program is valid not depend on the architecture (at least
    for this specific rule), so an earlier CL removed the limit entirely.
    However, we decided that allowing indexes that didn't work on _any_
    architecture was silly, so this CL re-restricts the indexes, this time
    to fit in an int64, for consistency across architectures.

    Russ
  • Rémy Oudompheng at Dec 6, 2012 at 9:43 pm ⇧

    On 2012/12/6 Russ Cox wrote:
    We have been going back and forth on indices and indexes. Indexes is a
    common alternative spelling, matches the heading in the spec, and is
    found when you search for index.

    The rule used to be 'has to fit in int', but that means x[1<<40]
    compiles on amd64 but does not compile on 386. We were trying to make
    whether a Go program is valid not depend on the architecture (at least
    for this specific rule), so an earlier CL removed the limit entirely.
    However, we decided that allowing indexes that didn't work on _any_
    architecture was silly, so this CL re-restricts the indexes, this time
    to fit in an int64, for consistency across architectures.
    But nobody is trying to make:

    package main
    import "fmt"
    func main() {
    x := 0x123456789abcdef
    fmt.Println(x)
    }

    compile regardless of architecture, right?

    Rémy.
  • Russ Cox at Dec 6, 2012 at 10:19 pm ⇧

    But nobody is trying to make:

    package main
    import "fmt"
    func main() {
    x := 0x123456789abcdef
    fmt.Println(x)
    }

    compile regardless of architecture, right?
    No.
  • Russ Cox at Dec 6, 2012 at 10:19 pm ⇧
    (Right.)
  • Rob Pike at Dec 7, 2012 at 4:45 am ⇧
    Index is a Latin-rooted word with a Latin plural. To use 'indexes'
    rather than 'indices' in this context isn't as bad as 'mediums' for
    'media' or 'semantics' as a plural, and it (all of them, really) may
    well be the way English is going, but it bothers me to use a word we
    know is, however mildly, incorrect.

    That trivium (ha!) aside, what is the right thing to do about the
    compile-time limits on an index constant? I don't see why 'int' isn't
    the right answer.

    -rob
  • Ian Lance Taylor at Dec 7, 2012 at 5:09 am ⇧

    On Thu, Dec 6, 2012 at 8:45 PM, Rob Pike wrote:
    Index is a Latin-rooted word with a Latin plural. To use 'indexes'
    rather than 'indices' in this context isn't as bad as 'mediums' for
    'media' or 'semantics' as a plural, and it (all of them, really) may
    well be the way English is going, but it bothers me to use a word we
    know is, however mildly, incorrect.
    This issue is both flammable and inflammable.
    That trivium (ha!) aside, what is the right thing to do about the
    compile-time limits on an index constant? I don't see why 'int' isn't
    the right answer.
    It's occasionally nice to be able to write code like

    var big []int
    if unsafe.Sizeof(int(1)) == 4 {
    big = make([]int, 1 << 16)
    } else {
    big = make([]int, 1 << 36)
    }

    If we restrict the constants to int, that code will fail at compile
    time, because the 1 << 36 will be rejected when building on a 32-bit
    system. Obviously there are various workarounds one can use, such as
    build constraints, or using variables rather than constants. But it
    seems reasonable to me to simply permit the simple code, and give the
    error at run time if necessary. It's unlikely that restricting the
    constant to int will ever catch any actual problem at compile time.
    Permitting an int64 constant will permit some natural code to work
    without awkward workarounds.

    Ian
  • Rob Pike at Dec 7, 2012 at 6:06 am ⇧
    I like your example. I remain nervous about the specificity of int64
    but it seems likely to be big enough for the time being, so OK.

    -rob
  • Robert Griesemer at Dec 7, 2012 at 6:20 am ⇧
    [re-sending, this time to all]

    I think the example is broken. That can be written shorter and cleaner as:

    var n int
    if unsafe.Sizeof(int(1)) == 4 {
    n = 1 << 16
    } else {
    n = 1 << 36
    }
    big := make([]int, n)

    or even:

    n := 1 << 16
    if unsafe.Sizeof(int(1)) == 8 {
    n = 1 << 36
    }
    big := make([]int, n)

    I am starting to lean the other (Rob's) way:

    len(a) will return an int and it's simply not possible to create an
    array/slice with a larger size. The places where it matters are:

    a) array declarations
    b) index/slice expressions
    c) make()

    In a) the size must be a constant and cannot be larger than int (we
    certainly don't want a runtime-error for a declaration!). So a)
    already requires the size to be within int limits. In fact my current
    CL is wrong in this respect.

    Arguably, if a) imposes this limit, we should have the same in c)
    which is in some sense the dynamic version of c). As shown, Ian's
    example can always be trivially rewritten using a variable.

    For b) any constant index for an array can impossibly be larger than
    int ever, because such arrays cannot be declared. It seems odd to
    require a run-time error in that case if the constant is an int.
    Again, in the rare case where it might matter, a variable is trivially
    solving the problem.

    Finally, this only matters when we compile code for a given platform.
    I think it makes a lot of sense for that code to not compile if it in
    fact either a) declares an array that cannot be represented (size too
    large), or make is called with a size too large. Again, if that make
    is not dynamically invoked, a variable trivially circumvents the
    issue. In some sense this is an issue of writing portable code, not of
    the language being restrictive.

    I propose:

    1) I separate the language change indices -> indexes into a separate
    CL (which can be discussed as long as we want)

    2) I change the limit to be the maximum int value for the given platform.

    - gri

    PS: 2) has also the advantage that it almost completely describes the
    implemented status quo which in fact has not caused any problems so
    far. This issue came up not because of problems with the compilers but
    because the compilers implemented something different from what the
    spec said.
    On Thu, Dec 6, 2012 at 10:06 PM, Rob Pike wrote:
    I like your example. I remain nervous about the specificity of int64
    but it seems likely to be big enough for the time being, so OK.

    -rob
  • Ian Lance Taylor at Dec 7, 2012 at 6:36 am ⇧

    On Thu, Dec 6, 2012 at 10:19 PM, Robert Griesemer wrote:
    I think the example is broken. That can be written shorter and cleaner as:

    var n int
    if unsafe.Sizeof(int(1)) == 4 {
    n = 1 << 16
    } else {
    n = 1 << 36
    }
    big := make([]int, n)
    No, you can't write it that way. Try it with a 32-bit build. You'll
    get something like

    foo.go:8: constant 68719476736 overflows int
    or even:

    n := 1 << 16
    if unsafe.Sizeof(int(1)) == 8 {
    n = 1 << 36
    }
    big := make([]int, n)
    Doesn't work either, for the same reason.

    Ian
  • Robert Griesemer at Dec 7, 2012 at 6:58 am ⇧
    It should be int64 of course. Same argument applies.
    On Dec 6, 2012 10:36 PM, "Ian Lance Taylor" wrote:
    On Thu, Dec 6, 2012 at 10:19 PM, Robert Griesemer wrote:

    I think the example is broken. That can be written shorter and cleaner as:
    var n int
    if unsafe.Sizeof(int(1)) == 4 {
    n = 1 << 16
    } else {
    n = 1 << 36
    }
    big := make([]int, n)
    No, you can't write it that way. Try it with a 32-bit build. You'll
    get something like

    foo.go:8: constant 68719476736 overflows int
    or even:

    n := 1 << 16
    if unsafe.Sizeof(int(1)) == 8 {
    n = 1 << 36
    }
    big := make([]int, n)
    Doesn't work either, for the same reason.

    Ian
  • Robert Griesemer at Dec 7, 2012 at 7:07 am ⇧
    So, what I should have written:

    var n int64 = 1 << 16
    if ...

    Otherwise its not portable code of course.

    - gri
    On Dec 6, 2012 10:58 PM, "Robert Griesemer" wrote:

    It should be int64 of course. Same argument applies.
    On Dec 6, 2012 10:36 PM, "Ian Lance Taylor" wrote:
    On Thu, Dec 6, 2012 at 10:19 PM, Robert Griesemer wrote:

    I think the example is broken. That can be written shorter and cleaner as:
    var n int
    if unsafe.Sizeof(int(1)) == 4 {
    n = 1 << 16
    } else {
    n = 1 << 36
    }
    big := make([]int, n)
    No, you can't write it that way. Try it with a 32-bit build. You'll
    get something like

    foo.go:8: constant 68719476736 overflows int
    or even:

    n := 1 << 16
    if unsafe.Sizeof(int(1)) == 8 {
    n = 1 << 36
    }
    big := make([]int, n)
    Doesn't work either, for the same reason.

    Ian
  • Russ Cox at Dec 7, 2012 at 7:29 am ⇧
    You can write it, you just have to write
    var n int64
    (not that most users would figure that out).

    Russ

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupgolang-dev @
categoriesgo
postedDec 6, '12 at 9:10p
activeDec 7, '12 at 7:29a
posts14
users6
websitegolang.org

6 users in discussion

Russ Cox: 4 posts Rob Pike: 3 posts Robert Griesemer: 3 posts Ian Lance Taylor: 2 posts Rémy Oudompheng: 1 post Robert Griesemer: 1 post

Content

People

Support

Translate

site design / logo © 2022 Grokbase