FAQ

Hi!

I try to output unicode stream via ':encoding(cp1250)' layer.
Some symbols doesn't map to this encoding, so i get warnings:

"\x{0456}" does not map to cp1250 at C:\buf\osm\osm2mp.pl line 2637,
<$_[...]> line 2460557.
"\x{043d}" does not map to cp1250 at C:\buf\osm\osm2mp.pl line 2637,
<$_[...]> line 2460557.
"\x{043d}" does not map to cp1250 at C:\buf\osm\osm2mp.pl line 2637,
<$_[...]> line 2460557.
"\x{0438}" does not map to cp1250 at C:\buf\osm\osm2mp.pl line 2637,
<$_[...]> line 2460557.
"\x{0446}" does not map to cp1250 at C:\buf\osm\osm2mp.pl line 2637,
<$_[...]> line 2460557.
and so on.

And _sometimes_ after few such warnings perl crashes with message:

panic: sv_setpvn called with negative strlen at C:\buf\osm\osm2mp.pl
line 2375, <$_[...]> line 4001961.


Maybe there are problems with octet counting in such cases?
Thanks for your report. Unfortunately, without some sample code and
data it is difficult to replicate and investigate this problem. Can you
provide a simplified test case that replicates this problem?

Steve

Search Discussions

  • Liosha at Feb 5, 2011 at 11:20 am
    Hello.

    This code causes such crash in a few seconds:

    use 5.010;
    binmode STDOUT, ':encoding(cp1250)';
    while (1) {
    my $string = join q{}, map { chr(rand 0x500) } (0..rand 1000);
    say $string;
    }
    Hi!

    I try to output unicode stream via ':encoding(cp1250)' layer.
    Some symbols doesn't map to this encoding, so i get warnings:

    "\x{0456}" does not map to cp1250 at C:\buf\osm\osm2mp.pl line 2637,
    <$_[...]> line 2460557.
    "\x{043d}" does not map to cp1250 at C:\buf\osm\osm2mp.pl line 2637,
    <$_[...]> line 2460557.
    "\x{043d}" does not map to cp1250 at C:\buf\osm\osm2mp.pl line 2637,
    <$_[...]> line 2460557.
    "\x{0438}" does not map to cp1250 at C:\buf\osm\osm2mp.pl line 2637,
    <$_[...]> line 2460557.
    "\x{0446}" does not map to cp1250 at C:\buf\osm\osm2mp.pl line 2637,
    <$_[...]> line 2460557.
    and so on.

    And _sometimes_ after few such warnings perl crashes with message:

    panic: sv_setpvn called with negative strlen at C:\buf\osm\osm2mp.pl
    line 2375, <$_[...]> line 4001961.


    Maybe there are problems with octet counting in such cases?
    Thanks for your report. Unfortunately, without some sample code and
    data it is difficult to replicate and investigate this problem. Can you
    provide a simplified test case that replicates this problem?
    Steve


    --
    С уважением,
    liosha mailto:xliosha@gmail.com
  • Eric Brine at Feb 7, 2011 at 8:12 pm

    On Sat, Feb 5, 2011 at 6:19 AM, liosha wrote:

    This code causes such crash in a few seconds:

    use 5.010;
    binmode STDOUT, ':encoding(cp1250)';
    while (1) {
    my $string = join q{}, map { chr(rand 0x500) } (0..rand 1000);
    say $string;
    }
    Reproduced with blead @g76cc22e

    #0 Perl_sv_setpvn (sv=0x8320068, ptr=0x83388e1 "", len=4294967295) at
    sv.c:4454
    #1 0xb7708793 in encode_method (enc=0xb722389c, dir=0xb7225c60,
    src=0x8320068, check=<value optimized out>, offset=0x0, term=0x0,
    retcode=0x0, fallback_cb=0x831c628) at Encode.xs:266
    #2 0xb77092cf in XS_Encode__XS_encode (cv=0x83a17b8) at Encode.xs:657
    #3 0x08138203 in Perl_pp_entersub () at pp_hot.c:2931
    #4 0x080fe127 in Perl_runops_debug () at dump.c:2267
    #5 0x08083288 in Perl_call_sv (sv=0x8320b38, flags=130) at perl.c:2614
    #6 0xb723792b in PerlIOEncode_flush (f=0x832bdd8) at encoding.xs:424
    #7 0x082492e3 in PerlIOBuf_write (f=0x832bdd8, vbuf=0x83bc2a0, count=546)
    at perlio.c:4157
    #8 0xb72391f3 in PerlIOEncode_write (f=0x832bdd8, vbuf=0x83bc2a0,
    count=1570) at encoding.xs:593
    #9 0x08211839 in Perl_do_print (sv=0x8331560, fp=0x832bdd8) at doio.c:1257
    #10 0x08145979 in Perl_pp_print () at pp_hot.c:773
    #11 0x080fe127 in Perl_runops_debug () at dump.c:2267
    #12 0x08084b7b in perl_run (my_perl=0x831d008) at perl.c:2332
    #13 0x08062f25 in main (argc=3, argv=0xbfffc9b4, env=0xbfffc9c4) at
    perlmain.c:120

    Deterministic test case:

    binmode STDOUT, ':encoding(cp1250)';
    print map chr,
    1146, 627, 46, 891, 583, 542, 507, 1169, 1162, 663, 577, 518, 223, 526,
    1016, 885, 1135, 1077, 16, 774, 802, 623, 1164, 235,
    1136, 1027, 1, 502, 1222, 132, 1127, 738, 747, 115, 315, 23, 643, 455, 815,
    1026, 140, 725, 405, 12, 208, 511, 680, 906,
    816, 392, 103, 71, 1039, 926, 1163, 953, 38, 1175, 335, 1032, 950, 865, 992,
    59, 575, 1263, 227, 216, 1265, 1036, 1189, 365,
    667, 403, 1157, 548, 150, 415, 7, 1142, 621, 630, 668, 691, 435, 176, 1152,
    396, 1015, 236, 1202, 296, 997, 1115, 1206, 910,
    997, 621, 8, 173, 455, 481, 7, 342, 448, 744, 417, 46, 19, 280, 608, 466,
    169, 1271, 195, 574, 1246, 1213, 777, 473, 169,
    806, 382, 232, 304, 1088, 473, 612, 1011, 1248, 986, 284, 1149, 427, 353,
    1110, 287, 957, 229, 378, 793, 48, 114, 1173, 767,
    673, 769, 869, 368, 348, 663, 665, 1007, 1180, 871, 561, 1267, 501, 255,
    734, 1194, 117, 317, 69, 525, 378, 391, 753, 128,
    672, 772, 675, 250, 389, 153, 1245, 1141, 419, 1214, 581, 109, 371, 1000,
    1241, 1106, 552, 163, 262, 511, 141, 240, 501,
    705, 612, 1256, 432, 4, 28, 959, 381, 196, 567, 134, 722, 4, 40, 360, 603,
    359, 518, 979, 189, 316, 1054, 1035, 161, 850,
    343, 43, 487, 210, 275, 643, 707, 514, 826, 1213, 1123, 773, 1130, 322, 679,
    203, 721, 837, 997, 140, 563, 803, 255, 890,
    163, 48, 786, 637, 1048, 110, 942, 309, 1015, 398, 603, 903, 387, 449, 814,
    700, 544, 477, 436, 794, 631, 1014, 774, 1104,
    1164, 703, 1278, 1267, 1216, 678, 88, 932, 861, 629, 669, 772, 314, 880,
    128, 263, 130, 739, 799, 790, 871, 1200, 151, 131,
    677, 237, 363, 377, 1276, 1275, 69, 1067, 165, 710, 1011, 560, 1239, 316,
    1061, 970, 1043, 1035, 241, 634, 1157, 5, 1091,
    332, 1252, 1106, 381, 837, 942, 328, 1268, 452, 892, 796, 1183, 282, 666,
    1151, 1123, 402, 1109, 1023, 804, 344, 1214, 722,
    928, 870, 721, 308, 536, 1048, 820, 217, 1028, 1252, 1054, 438, 66, 999,
    1056, 275, 742, 931, 1213, 608, 224, 697, 358, 855,
    132, 705, 477, 1222, 570, 424, 324, 28, 759, 963, 193, 150, 1098, 513, 607,
    901, 449, 411, 75, 725, 1247, 982, 274, 752, 63,
    179, 545, 617, 544, 436, 1086, 1001, 224, 149, 1054, 225, 66, 402, 364, 288,
    1156, 76, 1105, 950, 421, 203, 172, 1091, 1230,
    498, 632, 954, 296, 1067, 690, 391, 126, 251, 445, 466, 740, 843, 116, 216,
    827, 924, 1113, 406, 1211, 1094, 522, 940, 304,
    100, 286, 249, 888, 1175, 652, 184, 267, 1168, 231, 668, 323, 1087, 404,
    736, 450, 969, 693, 4, 1082, 959, 321, 1017, 892,
    16, 1162, 1166, 1271, 578, 209, 48, 913, 1116, 25, 661, 901, 854, 643, 827,
    1142, 1261, 289, 998, 45, 743, 1245, 421, 1204,
    472, 117, 345, 1013, 1239, 895, 278, 1235, 1097, 730, 539, 628, 863, 327,
    137, 1083, 490, 871, 1021, 468, 938, 1022, 553,
    903, 677, 109, 1239, 115, 627, 1188, 656, 986, 79, 730, 1270, 168, 1089,
    1086, 759, 247, 794, 1210, 340, 138, 226, 1069, 46,
    454, 447, 643, 840, 382, 493, 58, 968, 1263, 6, 1058, 567, 647, 747, 252,
    888
    ;

    - Eric
  • Eric Brine via RT at Feb 7, 2011 at 8:32 pm
    Hi,

    This appears to be a bug in Encode, and Encode is maintained separately
    from Perl. As such, I'm closing this ticket. I've already opened a
    ticket in Encode's queue for you:

    https://rt.cpan.org/Ticket/Display.html?id=65541

    Thanks,
    Eric Brine

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupperl5-porters @
categoriesperl
postedFeb 5, '11 at 1:49a
activeFeb 7, '11 at 8:32p
posts4
users4
websiteperl.org

People

Translate

site design / logo © 2022 Grokbase