FAQ
Edit report at http://pear.php.net/bugs/bug.php?id=16783&edit=1

ID: 16783
Updated by: denials@gmail.com
Reported By: mschraeder at btsb dot com
Summary: Better validation when creating a field
-Status: Open
+Status: Feedback
Type: Feature/Change Request
Package: File_MARC
Operating System: N/A
Package Version: 0.4.2
PHP Version: Irrelevant
Roadmap Versions:
New Comment:

-Status: Open
+Status: Feedback
Could you please attach some example bad MARC records so that I can put
together a test and patch for your problem?


Previous Comments:
------------------------------------------------------------------------

[2009-11-12 17:42:03] frozensolid

Description:
------------
We deal with large collection files and often these files are poorly
made and do not follow proper MARC standards. Most often will be an
invalid tag (for example: "30-").

On lines 319 and 349 of MARC.php, new Data Fields are created without
validating the inputs, so the follow error is thrown:

Tag "30-" is not a valid tag. in
<b>/usr/share/php/File/MARC/Data_Field.php</b> on line <b>105</b>

Before creating the field, the Tag and Indicators should be validated,
and if invalid throw warnings and ignore the bad data.

------------------------------------------------------------------------

Search Discussions

  • Mschraeder at Nov 12, 2009 at 7:44 pm
    Edit report at http://pear.php.net/bugs/bug.php?id=16783&edit=1

    ID: 16783
    Comment by: mschraeder@btsb.com
    Reported By: mschraeder at btsb dot com
    Summary: Better validation when creating a field
    Status: Feedback
    Type: Feature/Change Request
    Package: File_MARC
    Operating System: N/A
    Package Version: 0.4.2
    PHP Version: Irrelevant
    Roadmap Versions:
    New Comment:

    I uploaded the file to http://www.mo.btsb.com/test/test.mrc

    The following test code will cause the error just by looping through
    the records.

    The last working record is:
    Everything I needed to know about being a girl I learned from Judy
    Blume / _cedited by Jennifer O'Connell.

    $testfile = 'test.mrc';

    if (file_exists($testfile)) {
    $MARCfile = new File_MARC($testfile);

    while ($marcRecord = $MARCfile->next()) {
    $field = $marcRecord->getField('245');
    echo $field->__toString().'<br>';
    }
    }

    The record obviously has a lot wrong with it, but simply commenting out
    throwing the exception in the Field constructor allows our scripts to
    handle the necessary parts of the record.


    Previous Comments:
    ------------------------------------------------------------------------

    [2009-11-12 18:36:20] dbs

    -Status: Open
    +Status: Feedback
    Could you please attach some example bad MARC records so that I can put
    together a test and patch for your problem?

    ------------------------------------------------------------------------

    [2009-11-12 17:42:03] frozensolid

    Description:
    ------------
    We deal with large collection files and often these files are poorly
    made and do not follow proper MARC standards. Most often will be an
    invalid tag (for example: "30-").

    On lines 319 and 349 of MARC.php, new Data Fields are created without
    validating the inputs, so the follow error is thrown:

    Tag "30-" is not a valid tag. in
    <b>/usr/share/php/File/MARC/Data_Field.php</b> on line <b>105</b>

    Before creating the field, the Tag and Indicators should be validated,
    and if invalid throw warnings and ignore the bad data.

    ------------------------------------------------------------------------
  • Denials at Nov 12, 2009 at 7:54 pm
    Edit report at http://pear.php.net/bugs/bug.php?id=16783&edit=1

    ID: 16783
    Updated by: denials@gmail.com
    Reported By: mschraeder at btsb dot com
    Summary: Better validation when creating a field
    Status: Feedback
    Type: Feature/Change Request
    Package: File_MARC
    Operating System: N/A
    Package Version: 0.4.2
    PHP Version: Irrelevant
    Roadmap Versions:
    New Comment:

    I can't resolve that address. Can you double-check it, please?


    Previous Comments:
    ------------------------------------------------------------------------

    [2009-11-12 20:44:00] frozensolid

    I uploaded the file to http://www.mo.btsb.com/test/test.mrc

    The following test code will cause the error just by looping through
    the records.

    The last working record is:
    Everything I needed to know about being a girl I learned from Judy
    Blume / _cedited by Jennifer O'Connell.

    $testfile = 'test.mrc';

    if (file_exists($testfile)) {
    $MARCfile = new File_MARC($testfile);

    while ($marcRecord = $MARCfile->next()) {
    $field = $marcRecord->getField('245');
    echo $field->__toString().'<br>';
    }
    }

    The record obviously has a lot wrong with it, but simply commenting out
    throwing the exception in the Field constructor allows our scripts to
    handle the necessary parts of the record.

    ------------------------------------------------------------------------

    [2009-11-12 18:36:20] dbs

    -Status: Open
    +Status: Feedback
    Could you please attach some example bad MARC records so that I can put
    together a test and patch for your problem?

    ------------------------------------------------------------------------

    [2009-11-12 17:42:03] frozensolid

    Description:
    ------------
    We deal with large collection files and often these files are poorly
    made and do not follow proper MARC standards. Most often will be an
    invalid tag (for example: "30-").

    On lines 319 and 349 of MARC.php, new Data Fields are created without
    validating the inputs, so the follow error is thrown:

    Tag "30-" is not a valid tag. in
    <b>/usr/share/php/File/MARC/Data_Field.php</b> on line <b>105</b>

    Before creating the field, the Tag and Indicators should be validated,
    and if invalid throw warnings and ignore the bad data.

    ------------------------------------------------------------------------
  • Mschraeder at Nov 12, 2009 at 8:09 pm
    Edit report at http://pear.php.net/bugs/bug.php?id=16783&edit=1

    ID: 16783
    Comment by: mschraeder@btsb.com
    Reported By: mschraeder at btsb dot com
    Summary: Better validation when creating a field
    Status: Feedback
    Type: Feature/Change Request
    Package: File_MARC
    Operating System: N/A
    Package Version: 0.4.2
    PHP Version: Irrelevant
    Roadmap Versions:
    New Comment:

    try http://www.frozen-solid.net/test.mrc


    Previous Comments:
    ------------------------------------------------------------------------

    [2009-11-12 20:53:41] dbs

    I can't resolve that address. Can you double-check it, please?

    ------------------------------------------------------------------------

    [2009-11-12 20:44:00] frozensolid

    I uploaded the file to http://www.mo.btsb.com/test/test.mrc

    The following test code will cause the error just by looping through
    the records.

    The last working record is:
    Everything I needed to know about being a girl I learned from Judy
    Blume / _cedited by Jennifer O'Connell.

    $testfile = 'test.mrc';

    if (file_exists($testfile)) {
    $MARCfile = new File_MARC($testfile);

    while ($marcRecord = $MARCfile->next()) {
    $field = $marcRecord->getField('245');
    echo $field->__toString().'<br>';
    }
    }

    The record obviously has a lot wrong with it, but simply commenting out
    throwing the exception in the Field constructor allows our scripts to
    handle the necessary parts of the record.

    ------------------------------------------------------------------------

    [2009-11-12 18:36:20] dbs

    -Status: Open
    +Status: Feedback
    Could you please attach some example bad MARC records so that I can put
    together a test and patch for your problem?

    ------------------------------------------------------------------------

    [2009-11-12 17:42:03] frozensolid

    Description:
    ------------
    We deal with large collection files and often these files are poorly
    made and do not follow proper MARC standards. Most often will be an
    invalid tag (for example: "30-").

    On lines 319 and 349 of MARC.php, new Data Fields are created without
    validating the inputs, so the follow error is thrown:

    Tag "30-" is not a valid tag. in
    <b>/usr/share/php/File/MARC/Data_Field.php</b> on line <b>105</b>

    Before creating the field, the Tag and Indicators should be validated,
    and if invalid throw warnings and ignore the bad data.

    ------------------------------------------------------------------------
  • Denials at Nov 12, 2009 at 9:28 pm
    Edit report at http://pear.php.net/bugs/bug.php?id=16783&edit=1

    ID: 16783
    Updated by: denials@gmail.com
    Reported By: mschraeder at btsb dot com
    Summary: Better validation when creating a field
    Status: Feedback
    Type: Feature/Change Request
    Package: File_MARC
    Operating System: N/A
    Package Version: 0.4.2
    PHP Version: Irrelevant
    Roadmap Versions:
    New Comment:

    Hmm. Comparing the output of File_MARC 0.4.2 to yaz-marcdump and
    pymarc, even though File_MARC throws an exception for the really
    egregiously formatted record in question (which seems to be less about
    having a '30-' tag than it is about having a horrendously corrupted
    leader), it at least continues to soldier on and produces output for the
    following record.

    In comparison, both pymarc and yaz-marcdump terminate after hitting the
    egregious record in question and don't dump the output of the following
    record.

    So, I don't think File_MARC is doing such a bad job of dealing with
    invalid input data. Throwing an exception is probably the right thing to
    do.


    Previous Comments:
    ------------------------------------------------------------------------

    [2009-11-12 21:09:18] frozensolid

    try http://www.frozen-solid.net/test.mrc

    ------------------------------------------------------------------------

    [2009-11-12 20:53:41] dbs

    I can't resolve that address. Can you double-check it, please?

    ------------------------------------------------------------------------

    [2009-11-12 20:44:00] frozensolid

    I uploaded the file to http://www.mo.btsb.com/test/test.mrc

    The following test code will cause the error just by looping through
    the records.

    The last working record is:
    Everything I needed to know about being a girl I learned from Judy
    Blume / _cedited by Jennifer O'Connell.

    $testfile = 'test.mrc';

    if (file_exists($testfile)) {
    $MARCfile = new File_MARC($testfile);

    while ($marcRecord = $MARCfile->next()) {
    $field = $marcRecord->getField('245');
    echo $field->__toString().'<br>';
    }
    }

    The record obviously has a lot wrong with it, but simply commenting out
    throwing the exception in the Field constructor allows our scripts to
    handle the necessary parts of the record.

    ------------------------------------------------------------------------

    [2009-11-12 18:36:20] dbs

    -Status: Open
    +Status: Feedback
    Could you please attach some example bad MARC records so that I can put
    together a test and patch for your problem?

    ------------------------------------------------------------------------

    [2009-11-12 17:42:03] frozensolid

    Description:
    ------------
    We deal with large collection files and often these files are poorly
    made and do not follow proper MARC standards. Most often will be an
    invalid tag (for example: "30-").

    On lines 319 and 349 of MARC.php, new Data Fields are created without
    validating the inputs, so the follow error is thrown:

    Tag "30-" is not a valid tag. in
    <b>/usr/share/php/File/MARC/Data_Field.php</b> on line <b>105</b>

    Before creating the field, the Tag and Indicators should be validated,
    and if invalid throw warnings and ignore the bad data.

    ------------------------------------------------------------------------
  • Denials at Nov 12, 2009 at 9:51 pm
    Edit report at http://pear.php.net/bugs/bug.php?id=16783&edit=1

    ID: 16783
    Updated by: denials@gmail.com
    Reported By: mschraeder at btsb dot com
    Summary: Better validation when creating a field
    -Status: Feedback
    +Status: Closed
    Type: Feature/Change Request
    Package: File_MARC
    Operating System: N/A
    Package Version: 0.4.2
    PHP Version: Irrelevant
    -Assigned To:
    +Assigned To: dbs
    Roadmap Versions:
    New Comment:

    -Status: Feedback
    +Status: Closed
    -Assigned To:
    +Assigned To: dbs
    Thank you for your bug report. This issue has been fixed
    in the latest released version of the package, which you can download
    at
    http://pear.php.net/get/

    Changed the behaviour to catch the exception that was thrown by
    MARC_Data_Field and add it as a warning to the MARC_Record object.

    Added a new test for the specific case mentioned in the bug report
    (invalid tag name). The sample provided by the bug reporter appears to
    be a different case entirely and isn't handled by other standard MARC
    parsing tools.

    Pushing out release 0.4.3 as a result.


    Previous Comments:
    ------------------------------------------------------------------------

    [2009-11-12 22:28:08] dbs

    Hmm. Comparing the output of File_MARC 0.4.2 to yaz-marcdump and
    pymarc, even though File_MARC throws an exception for the really
    egregiously formatted record in question (which seems to be less about
    having a '30-' tag than it is about having a horrendously corrupted
    leader), it at least continues to soldier on and produces output for the
    following record.

    In comparison, both pymarc and yaz-marcdump terminate after hitting the
    egregious record in question and don't dump the output of the following
    record.

    So, I don't think File_MARC is doing such a bad job of dealing with
    invalid input data. Throwing an exception is probably the right thing to
    do.

    ------------------------------------------------------------------------

    [2009-11-12 21:09:18] frozensolid

    try http://www.frozen-solid.net/test.mrc

    ------------------------------------------------------------------------

    [2009-11-12 20:53:41] dbs

    I can't resolve that address. Can you double-check it, please?

    ------------------------------------------------------------------------

    [2009-11-12 20:44:00] frozensolid

    I uploaded the file to http://www.mo.btsb.com/test/test.mrc

    The following test code will cause the error just by looping through
    the records.

    The last working record is:
    Everything I needed to know about being a girl I learned from Judy
    Blume / _cedited by Jennifer O'Connell.

    $testfile = 'test.mrc';

    if (file_exists($testfile)) {
    $MARCfile = new File_MARC($testfile);

    while ($marcRecord = $MARCfile->next()) {
    $field = $marcRecord->getField('245');
    echo $field->__toString().'<br>';
    }
    }

    The record obviously has a lot wrong with it, but simply commenting out
    throwing the exception in the Field constructor allows our scripts to
    handle the necessary parts of the record.

    ------------------------------------------------------------------------

    [2009-11-12 18:36:20] dbs

    -Status: Open
    +Status: Feedback
    Could you please attach some example bad MARC records so that I can put
    together a test and patch for your problem?

    ------------------------------------------------------------------------

    The remainder of the comments for this report are too long. To view
    the rest of the comments, please view the bug report online at
    http://pear.php.net/bugs/bug.php?id=16783
  • Mschraeder at Nov 13, 2009 at 2:19 pm
    Edit report at http://pear.php.net/bugs/bug.php?id=16783&edit=1

    ID: 16783
    Comment by: mschraeder@btsb.com
    Reported By: mschraeder at btsb dot com
    Summary: Better validation when creating a field
    Status: Closed
    Type: Feature/Change Request
    Package: File_MARC
    Operating System: N/A
    Package Version: 0.4.2
    PHP Version: Irrelevant
    Assigned To: dbs
    Roadmap Versions:
    New Comment:

    Confirmed fix.

    When File_MARC hits the file with the invalid tag it's carrying on just
    fine and otherwise seems to be working.

    I must have messed up when I was putting together the test.mrc file so
    my apologies for that. It was throwing the error I expected it to, but
    I didn't check that the records after that were formatted properly.


    Previous Comments:
    ------------------------------------------------------------------------

    [2009-11-12 22:51:05] dbs

    -Status: Feedback
    +Status: Closed
    -Assigned To:
    +Assigned To: dbs
    Thank you for your bug report. This issue has been fixed
    in the latest released version of the package, which you can download
    at
    http://pear.php.net/get/

    Changed the behaviour to catch the exception that was thrown by
    MARC_Data_Field and add it as a warning to the MARC_Record object.

    Added a new test for the specific case mentioned in the bug report
    (invalid tag name). The sample provided by the bug reporter appears to
    be a different case entirely and isn't handled by other standard MARC
    parsing tools.

    Pushing out release 0.4.3 as a result.

    ------------------------------------------------------------------------

    [2009-11-12 22:28:08] dbs

    Hmm. Comparing the output of File_MARC 0.4.2 to yaz-marcdump and
    pymarc, even though File_MARC throws an exception for the really
    egregiously formatted record in question (which seems to be less about
    having a '30-' tag than it is about having a horrendously corrupted
    leader), it at least continues to soldier on and produces output for the
    following record.

    In comparison, both pymarc and yaz-marcdump terminate after hitting the
    egregious record in question and don't dump the output of the following
    record.

    So, I don't think File_MARC is doing such a bad job of dealing with
    invalid input data. Throwing an exception is probably the right thing to
    do.

    ------------------------------------------------------------------------

    [2009-11-12 21:09:18] frozensolid

    try http://www.frozen-solid.net/test.mrc

    ------------------------------------------------------------------------

    [2009-11-12 20:53:41] dbs

    I can't resolve that address. Can you double-check it, please?

    ------------------------------------------------------------------------

    [2009-11-12 20:44:00] frozensolid

    I uploaded the file to http://www.mo.btsb.com/test/test.mrc

    The following test code will cause the error just by looping through
    the records.

    The last working record is:
    Everything I needed to know about being a girl I learned from Judy
    Blume / _cedited by Jennifer O'Connell.

    $testfile = 'test.mrc';

    if (file_exists($testfile)) {
    $MARCfile = new File_MARC($testfile);

    while ($marcRecord = $MARCfile->next()) {
    $field = $marcRecord->getField('245');
    echo $field->__toString().'<br>';
    }
    }

    The record obviously has a lot wrong with it, but simply commenting out
    throwing the exception in the Field constructor allows our scripts to
    handle the necessary parts of the record.

    ------------------------------------------------------------------------

    The remainder of the comments for this report are too long. To view
    the rest of the comments, please view the bug report online at
    http://pear.php.net/bugs/bug.php?id=16783

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouppear-bugs @
categoriesphp
postedNov 12, '09 at 5:36p
activeNov 13, '09 at 2:19p
posts7
users2
websitepear.php.net

2 users in discussion

Denials: 4 posts Mschraeder: 3 posts

People

Translate

site design / logo © 2022 Grokbase