FAQ
Change 25271 by rgs@oregon on 2005/08/06 22:15:14

Upgrade to Encode 2.11, plus a patch to PerlIO::encoding
by Dan Kogai to fix encoding(utf-8-strict) with partial
characters.

Affected files ...

... //depot/perl/MANIFEST#1280 edit
... //depot/perl/ext/Encode/AUTHORS#34 edit
... //depot/perl/ext/Encode/Changes#74 edit
... //depot/perl/ext/Encode/Encode.pm#157 edit
... //depot/perl/ext/Encode/Encode.xs#112 edit
... //depot/perl/ext/Encode/Encode/encode.h#6 edit
... //depot/perl/ext/Encode/MANIFEST#47 edit
... //depot/perl/ext/Encode/META.yml#18 edit
... //depot/perl/ext/Encode/lib/Encode/Alias.pm#37 edit
... //depot/perl/ext/Encode/lib/Encode/Config.pm#10 edit
... //depot/perl/ext/Encode/lib/Encode/MIME/Header.pm#11 edit
... //depot/perl/ext/PerlIO/encoding/encoding.pm#13 edit

Differences ...

==== //depot/perl/MANIFEST#1280 (text) ====
Index: perl/MANIFEST
--- perl/MANIFEST#1279~25217~ Sun Jul 24 13:52:13 2005
+++ perl/MANIFEST Sat Aug 6 15:15:14 2005
@@ -428,6 +428,7 @@
ext/Encode/lib/Encode/JP/JIS7.pm Encode extension
ext/Encode/lib/Encode/KR/2022_KR.pm Encode extension
ext/Encode/lib/Encode/MIME/Header.pm Encode extension
+ext/Encode/lib/Encode/MIME/Header/ISO_2022_JP.pm Encode extension
ext/Encode/lib/Encode/PerlIO.pod Documents for Encode & PerlIO
ext/Encode/lib/Encode/Supported.pod Documents for supported encodings
ext/Encode/lib/Encode/Unicode/UTF7.pm Encode extension
@@ -469,6 +470,7 @@
ext/Encode/t/ksc5601.enc test data
ext/Encode/t/ksc5601.utf test data
ext/Encode/t/mime-header.t test script
+ext/Encode/t/mime_header_iso2022jp.t test script
ext/Encode/t/Mod_EUCJP.pm module that t/enc_module.enc uses
ext/Encode/t/perlio.t test script
ext/Encode/t/rt.pl test script

==== //depot/perl/ext/Encode/AUTHORS#34 (text) ====
Index: perl/ext/Encode/AUTHORS
--- perl/ext/Encode/AUTHORS#33~24490~ Mon May 16 14:42:05 2005
+++ perl/ext/Encode/AUTHORS Sat Aug 6 15:15:14 2005
@@ -12,43 +12,47 @@
Andreas J. Koenig <andreas.koenig@anima.de>
Anton Tagunov <tagunov@motor.ru>
Autrijus Tang <autrijus@autrijus.org>
-Benjamin Goldberg <goldbb2@earthlink.net>
+Benjamin Goldberg <goldbb2@earthlink.net>
Bjoern Hoehrmann <derhoermi@gmx.net>
Bjoern Jacke <debianbugs@j3e.de>
-Chris Nandor <pudge@pobox.com>
+Chris Nandor <pudge@pobox.com>
Craig A. Berry <craigberry@mac.com>
Dan Kogai <dankogai@dan.co.jp>
Dave Evans <dave@rudolf.org.uk>
Deng Liu <dengliu@ntu.edu.tw>
Dominic Dunlop <domo@computer.org>
-Elizabeth Mattijsen <liz@dijkmat.nl>
+Elizabeth Mattijsen <liz@dijkmat.nl>
Gerrit P. Haase <gp@familiehaase.de>
-Graham Barr <gbarr@pobox.com>
+Graham Barr <gbarr@pobox.com>
Gurusamy Sarathy <gsar@activestate.com>
-H.Merijn Brand <h.m.brand@hccnet.nl>
+H.Merijn Brand <h.m.brand@xs4all.nl>
Hugo van der Sanden <hv@crypt.org>
Inaba Hiroto <inaba@st.rim.or.jp>
Jarkko Hietaniemi <jhi@iki.fi>
-Jungshik Shin <jshin@mailaps.org>
+Jungshik Shin <jshin@mailaps.org>
+KONNO Hiroharu <hiroharu.konno@bowneglobal.co.jp>
Laszlo Molnar <ml1050@freemail.hu>
MORIYAMA Masayuki <msyk@mtg.biglobe.ne.jp>
-Mark-Jason Dominus <mjd@plover.com>
+Makamaka <makamaka@donzoko.net>
+Mark-Jason Dominus <mjd@plover.com>
Mattia Barbon <mbarbon@dsi.unive.it>
-Michael G Schwern <schwern@pobox.com>
+Michael G Schwern <schwern@pobox.com>
+Miron Cuperman <miron@hyper.to>
Nicholas Clark <nick@ccl4.org>
Nick Ing-Simmons <nick@ing-simmons.net>
-Paul Marquess <paul_marquess@yahoo.co.uk>
+Paul Marquess <paul_marquess@yahoo.co.uk>
Peter Prymmer <pvhp@best.com>
Philip Newton <pne@cpan.org>
-Robin Barker <rmb1@cise.npl.co.uk>
+Piotr Fusik <pfusik@op.pl>
+Robin Barker <rmb1@cise.npl.co.uk>
SADAHIRO Tomoyuki <SADAHIRO@cpan.org>
SUGAWARA Hajime <sugawara@hdt.co.jp>
-SUZUKI Norio <ZAP00217@nifty.com>
+SUZUKI Norio <ZAP00217@nifty.com>
Simon Cozens <simon@netthink.co.uk>
Spider Boardman <spider@web.zk3.dec.com>
Steve Hay <steve.hay@uk.radan.com>
Steve Peters <steve@fisharerojo.org>
Tatsuhiko Miyagawa <miyagawa@edge.co.jp>
-Tels <perl_dummy@bloodgate.com>
+Tels <perl_dummy@bloodgate.com>
Vadim Konovalov <vkonovalov@peterstar.ru>
Yitzchak Scott-Thoennes <sthoenna@efn.org>

==== //depot/perl/ext/Encode/Changes#74 (text) ====
Index: perl/ext/Encode/Changes
--- perl/ext/Encode/Changes#73~24490~ Mon May 16 14:42:05 2005
+++ perl/ext/Encode/Changes Sat Aug 6 15:15:14 2005
@@ -1,8 +1,28 @@
# Revision history for Perl extension Encode.
#
-# $Id: Changes,v 2.10 2005/05/16 18:46:36 dankogai Exp dankogai $
+# $Id: Changes,v 2.11 2005/08/05 10:58:25 dankogai Exp dankogai $
#
-$Revision: 2.10 $ $Date: 2005/05/16 18:46:36 $
+$Revision: 2.11 $ $Date: 2005/08/05 10:58:25 $
+! AUTHORS CHANGES
+ To reflect changes below
+! Encode.pm encoding.pm
+ lib/Encode/Alias.pm lib/Encode/PerlIO.pod lib/Encode/Supported.pod
+ Typo fixed by Piotr Fusik in Change 25261 & 25266
+ Message-ID: <001401c595bd$dccb5d80$0bd34dd5@piec>
+! Encode.xs
+ Addresses "BUG REPORT: panic in Encode.xs".
+ Message-Id: <42EDDA97.2010608@hyper.to>
++ lib/Encode/MIME/Header/ISO_2022_JP.pm mime_header_iso2022jp.t
+! lib/Encode/MIME/Header.pm lib/Encode/Config.pm
+ Encoding 'MIME-Header-ISO_2022_JP' is introduced by Makamaka
+ Message-Id: <200507311557.j6VFvE2K034605@www231.sakura.ne.jp>
+! Encode/encode.h Encode.pm Encode.xs
+ PerlIO's "encoding(utf-8-strict)" got a problem w/ partial character.
+ Found and addressed by KONNO Hiroharu <hiroharu.konno@bowneglobal.co.jp>
+ See also ext/PerlIO/encoding/encoding.pm
+ Message-Id: <E1DineE-00068X-TB@yok-gs-workman.asia.bgsinternal.com>
+
+2.10 2005/05/16 18:46:36
! Encode.pm
fixed decode_utf8() accordingly to RT#8872
http://rt.cpan.org/NoAuth/Bug.html?id=8872

==== //depot/perl/ext/Encode/Encode.pm#157 (text) ====
Index: perl/ext/Encode/Encode.pm
--- perl/ext/Encode/Encode.pm#156~25261~ Tue Aug 2 03:39:51 2005
+++ perl/ext/Encode/Encode.pm Sat Aug 6 15:15:14 2005
@@ -1,9 +1,9 @@
#
-# $Id: Encode.pm,v 2.10 2005/05/16 18:46:36 dankogai Exp dankogai $
+# $Id: Encode.pm,v 2.11 2005/08/05 10:58:25 dankogai Exp dankogai $
#
package Encode;
use strict;
-our $VERSION = sprintf "%d.%02d", q$Revision: 2.10 $ =~ /(\d+)/g;
+our $VERSION = sprintf "%d.%02d", q$Revision: 2.11 $ =~ /(\d+)/g;
sub DEBUG () { 0 }
use XSLoader ();
XSLoader::load(__PACKAGE__, $VERSION);
@@ -19,7 +19,7 @@
);

our @FB_FLAGS = qw(DIE_ON_ERR WARN_ON_ERR RETURN_ON_ERR LEAVE_SRC
- PERLQQ HTMLCREF XMLCREF);
+ PERLQQ HTMLCREF XMLCREF STOP_AT_PARTIAL);
our @FB_CONSTS = qw(FB_DEFAULT FB_CROAK FB_QUIET FB_WARN
FB_PERLQQ FB_HTMLCREF FB_XMLCREF);


==== //depot/perl/ext/Encode/Encode.xs#112 (text) ====
Index: perl/ext/Encode/Encode.xs
--- perl/ext/Encode/Encode.xs#111~24490~ Mon May 16 14:42:05 2005
+++ perl/ext/Encode/Encode.xs Sat Aug 6 15:15:14 2005
@@ -1,5 +1,5 @@
/*
- $Id: Encode.xs,v 2.4 2005/05/16 18:46:36 dankogai Exp dankogai $
+ $Id: Encode.xs,v 2.5 2005/08/05 10:58:25 dankogai Exp dankogai $
*/

#define PERL_NO_GET_CONTEXT
@@ -151,6 +151,8 @@
UV ch =
utf8n_to_uvuni(s+slen, (SvCUR(src)-slen),
&clen, UTF8_ALLOW_ANY|UTF8_CHECK_ONLY);
+ /* if non-representable multibyte prefix at end of current buffer - break*/
+ if (clen > tlen - sdone) break;
if (check & ENCODE_DIE_ON_ERR) {
Perl_croak(aTHX_ ERR_ENCODE_NOMAP,
(UV)ch, enc->name[0]);
@@ -290,7 +292,7 @@
if ((s + skip) > e) {
/* Partial character */
/* XXX could check that rest of bytes are UTF8_IS_CONTINUATION(ch) */
- if (stop_at_partial)
+ if (stop_at_partial || (check & ENCODE_STOP_AT_PARTIAL))
break;

goto malformed_byte;
@@ -787,6 +789,13 @@
XMLCREF()
CODE:
RETVAL = ENCODE_XMLCREF;
+OUTPUT:
+ RETVAL
+
+int
+STOP_AT_PARTIAL()
+CODE:
+ RETVAL = ENCODE_STOP_AT_PARTIAL;
OUTPUT:
RETVAL


==== //depot/perl/ext/Encode/Encode/encode.h#6 (text) ====
Index: perl/ext/Encode/Encode/encode.h
--- perl/ext/Encode/Encode/encode.h#5~24231~ Tue Apr 12 08:30:23 2005
+++ perl/ext/Encode/Encode/encode.h Sat Aug 6 15:15:14 2005
@@ -98,6 +98,7 @@
#define ENCODE_PERLQQ 0x0100 /* perlqq fallback string */
#define ENCODE_HTMLCREF 0x0200 /* HTML character ref. fb mode */
#define ENCODE_XMLCREF 0x0400 /* XML character ref. fb mode */
+#define ENCODE_STOP_AT_PARTIAL 0x0800 /* stop at partial explicitly */

#define ENCODE_FB_DEFAULT 0x0000
#define ENCODE_FB_CROAK 0x0001

==== //depot/perl/ext/Encode/MANIFEST#47 (text) ====
Index: perl/ext/Encode/MANIFEST
--- perl/ext/Encode/MANIFEST#46~24231~ Tue Apr 12 08:30:23 2005
+++ perl/ext/Encode/MANIFEST Sat Aug 6 15:15:14 2005
@@ -49,6 +49,7 @@
lib/Encode/JP/JIS7.pm Encode extension
lib/Encode/KR/2022_KR.pm Encode extension
lib/Encode/MIME/Header.pm Encode extension
+lib/Encode/MIME/Header/ISO_2022_JP.pm Encode extension
lib/Encode/PerlIO.pod Documents for Encode & PerlIO
lib/Encode/Supported.pod Documents for supported encodings
lib/Encode/Unicode/UTF7.pm Encode Extension
@@ -86,6 +87,7 @@
t/ksc5601.enc test data
t/ksc5601.utf test data
t/mime-header.t test script
+t/mime_header_iso2022jp.t test script
t/perlio.t test script
t/rt.pl even more test script
t/unibench.pl benchmark script

==== //depot/perl/ext/Encode/META.yml#18 (text) ====
Index: perl/ext/Encode/META.yml
--- perl/ext/Encode/META.yml#17~24490~ Mon May 16 14:42:05 2005
+++ perl/ext/Encode/META.yml Sat Aug 6 15:15:14 2005
@@ -1,7 +1,7 @@
# http://module-build.sourceforge.net/META-spec.html
#XXXXXXX This is a prototype!!! It will change in the future!!! XXXXX#
name: Encode
-version: 2.10
+version: 2.11
version_from: Encode.pm
installdirs: perl
requires:

==== //depot/perl/ext/Encode/lib/Encode/Alias.pm#37 (text) ====
Index: perl/ext/Encode/lib/Encode/Alias.pm
--- perl/ext/Encode/lib/Encode/Alias.pm#36~25261~ Tue Aug 2 03:39:51 2005
+++ perl/ext/Encode/lib/Encode/Alias.pm Sat Aug 6 15:15:14 2005
@@ -2,7 +2,7 @@
use strict;
no warnings 'redefine';
use Encode;
-our $VERSION = do { my @r = (q$Revision: 2.3 $ =~ /\d+/g); sprintf "%d."."%02d" x $#r, @r };
+our $VERSION = do { my @r = (q$Revision: 2.4 $ =~ /\d+/g); sprintf "%d."."%02d" x $#r, @r };
sub DEBUG () { 0 }

use base qw(Exporter);

==== //depot/perl/ext/Encode/lib/Encode/Config.pm#10 (text) ====
Index: perl/ext/Encode/lib/Encode/Config.pm
--- perl/ext/Encode/lib/Encode/Config.pm#9~22835~ Thu May 20 06:30:25 2004
+++ perl/ext/Encode/lib/Encode/Config.pm Sat Aug 6 15:15:14 2005
@@ -2,7 +2,7 @@
# Demand-load module list
#
package Encode::Config;
-our $VERSION = do { my @r = (q$Revision: 2.0 $ =~ /\d+/g); sprintf "%d."."%02d" x $#r, @r };
+our $VERSION = do { my @r = (q$Revision: 2.1 $ =~ /\d+/g); sprintf "%d."."%02d" x $#r, @r };

use strict;

@@ -145,6 +145,7 @@
'MIME-B' => 'Encode::MIME::Header',
'MIME-Q' => 'Encode::MIME::Header',

+ 'MIME-Header-ISO_2022_JP' => 'Encode::MIME::Header::ISO_2022_JP',
);
}


==== //depot/perl/ext/Encode/lib/Encode/MIME/Header.pm#11 (text) ====
Index: perl/ext/Encode/lib/Encode/MIME/Header.pm
--- perl/ext/Encode/lib/Encode/MIME/Header.pm#10~22835~ Thu May 20 06:30:25 2004
+++ perl/ext/Encode/lib/Encode/MIME/Header.pm Sat Aug 6 15:15:14 2005
@@ -1,7 +1,7 @@
package Encode::MIME::Header;
use strict;
# use warnings;
-our $VERSION = do { my @r = (q$Revision: 2.0 $ =~ /\d+/g); sprintf "%d."."%02d" x $#r, @r };
+our $VERSION = do { my @r = (q$Revision: 2.1 $ =~ /\d+/g); sprintf "%d."."%02d" x $#r, @r };
use Encode qw(find_encoding encode_utf8 decode_utf8);
use MIME::Base64;
use Carp;
@@ -215,6 +215,10 @@
and =?ISO-8859-1?= but that makes the implementation too complicated.
These days major mail agents all support =?UTF-8? so I think it is
just good enough.
+
+Due to popular demand, 'MIME-Header-ISO_2022_JP' was introduced by
+Makamaka. Thre are still too many MUAs especially cellular phone
+handsets which does not grok UTF-8.

=head1 SEE ALSO


==== //depot/perl/ext/PerlIO/encoding/encoding.pm#13 (text) ====
Index: perl/ext/PerlIO/encoding/encoding.pm
--- perl/ext/PerlIO/encoding/encoding.pm#12~24248~ Tue Apr 19 02:55:51 2005
+++ perl/ext/PerlIO/encoding/encoding.pm Sat Aug 6 15:15:14 2005
@@ -1,6 +1,6 @@
package PerlIO::encoding;
use strict;
-our $VERSION = '0.08';
+our $VERSION = '0.09';
our $DEBUG = 0;
$DEBUG and warn __PACKAGE__, " called by ", join(", ", caller), "\n";

@@ -12,7 +12,8 @@
use XSLoader ();
XSLoader::load(__PACKAGE__, $VERSION);

-our $fallback = Encode::PERLQQ()|Encode::WARN_ON_ERR();
+our $fallback =
+ Encode::PERLQQ()|Encode::WARN_ON_ERR()|Encode::STOP_AT_PARTIAL();

1;
__END__
End of Patch.

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupperl5-changes @
categoriesperl
postedAug 6, '05 at 10:45p
activeAug 6, '05 at 10:45p
posts1
users1
websiteperl.org

1 user in discussion

Rafael Garcia-Suarez: 1 post

People

Translate

site design / logo © 2021 Grokbase