[Bug 18757] New: Problem when importing only items in MARC records

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 18757] New: Problem when importing only items in MARC records

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18757

            Bug ID: 18757
           Summary: Problem when importing only items in MARC records
 Change sponsored?: ---
           Product: Koha
           Version: master
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P5 - low
         Component: Tools
          Assignee: [hidden email]
          Reporter: [hidden email]
        QA Contact: [hidden email]

When importing records with Stage MARC records for import, one can use matching
rules to only import items into existing records.
Those imported items are stored as XML to be staged.

The bug is that when MARC Flavour is UNIMARC the XML serialization fails
because its is looking in field 100$a which does not exist.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18757] Problem when importing only items in MARC records

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18757

Fridolin SOMERS <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|[hidden email]          |[hidden email]
                   |                            |m
           Severity|enhancement                 |normal

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18757] Problem when importing only items in MARC records

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18757

Fridolin SOMERS <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |Needs Signoff
   Patch complexity|---                         |Trivial patch

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18757] Problem when importing only items in MARC records

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18757

--- Comment #1 from Fridolin SOMERS <[hidden email]> ---
Created attachment 64104
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=64104&action=edit
Bug 18757 - Problem when importing only items in MARC records

When importing records with Stage MARC records for import, one can use matching
rules to only import items into existing records.
Those imported items are stored as XML to be staged.

The bug is that when MARC Flavour is UNIMARC the XML serialization fails
because its is looking in field 100$a which does not exist.
You see in logs the error : Unsupported UNIMARC character encoding [] for XML
output for UNIMARC; 100$a

This patch adds the format "USMARC" to XML serialization, like in
C4::Items::_get_unlinked_subfields_xml

Test plan :
- On a UNIMARC database
- Define a maching rule on title 200$a
- Select a record with items
- Export it using : Save as > MARC (Unicode/UTF-8)
- Delete all items
- Go to Tools > Stage MARC records for import
- Upload exported file
- Select title matching rule
- Select "Ingore incoming record" in "Action if matching record found :"
- Select Yes and "Always add items" in "Check for embedded item record data?"
- Click Stage for import
=> Without patch you get the error
=> With patch the import is staged
- Import into the catalog and check item is well recreated

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18757] Problem when importing only items in MARC records

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18757

Alex Buckley <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
             Status|Needs Signoff               |In Discussion

--- Comment #2 from Alex Buckley <[hidden email]> ---
Hi Fridolin

When applying your patch on a clean branch up to date with master, I find that
I am still getting the error message 'Unsupported UNIMARC character encoding []
for XML output for UNIMARC; 100$a' as I do without the patch applied.

I am getting this error when trying to both create and import records.

I have tried setting the matching record rule to both 100$a and 200$a but
continue to get the same error.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18757] Problem when importing only items in MARC records

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18757

--- Comment #3 from Fridolin SOMERS <[hidden email]> ---
Looks like this only occurs with Plack.
One must test with plack on tools/manage-marc-import.pl.
Since there are problems with background jobs with plack, test with Bug 17240.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18757] Problem when importing only items in MARC records

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18757

Fridolin SOMERS <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://bugs.koha-community
                   |                            |.org/bugzilla3/show_bug.cgi
                   |                            |?id=17240

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18757] Problem when importing only items in MARC records

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18757

--- Comment #4 from Fridolin SOMERS <[hidden email]> ---
But this patch is I think logic since it does XML serialization like in
C4::Items::_get_unlinked_subfields_xml

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18757] Problem when importing only items in MARC records

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18757

Tomás Cohen Arazi <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #5 from Tomás Cohen Arazi <[hidden email]> ---
(In reply to Alex Buckley from comment #2)

> Hi Fridolin
>
> When applying your patch on a clean branch up to date with master, I find
> that I am still getting the error message 'Unsupported UNIMARC character
> encoding [] for XML output for UNIMARC; 100$a' as I do without the patch
> applied.
>
> I am getting this error when trying to both create and import records.
>
> I have tried setting the matching record rule to both 100$a and 200$a but
> continue to get the same error.

Make sure you restart Plack after applying the patch.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18757] Problem when importing only items in MARC records

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18757

Fridolin SOMERS <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|In Discussion               |Needs Signoff

--- Comment #6 from Fridolin SOMERS <[hidden email]> ---
I set back to NSO

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18757] Problem when importing only items in MARC records

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18757

M. Tompsett <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #7 from M. Tompsett <[hidden email]> ---
https://archive.ifla.org/VI/8/projects/UNIMARC-HoldingsFormat.pdf
-- but isn't there supposed to be a 100?!

12-15 Character Set (Mandatory)
These four character positions indicate the principal graphic character sets
used in the record.
Positions 12-13 designate the G0 set and positions 14-15 designate the G1 set.
If a G1 set is
not needed, positions 14-15 contain blanks.
01 = ISO 646, IRV version (basic Latin set)
02 = ISO Registration # 37 (basic Cyrillic set)
03 = ISO 5426 (extended Latin set)
04 = ISO DIS 5427 (extended Cyrillic set)
05 = ISO 5428 (Greek set)
06 = ISO 6438 (African coded character set)
07 = ISO 10586 (Georgian set)
08 = ISO 8957 (Hebrew set) Table 1
09 = ISO 8957 (Hebrew set) Table 2
10 = [Reserved]
11 = ISO 5426-2 (Latin characters used in minor European languages and obsolete
typography)
50 = ISO 10646 Level 3 (Unicode)

Oh yuck... reading the MARC::File::XML library... it basically fakes it.

    # Does the record think it is already Unicode?
    if ($original_encoding ne 'a' && lc($format) !~ /^unimarc/o) {
        # If not, we'll make it so
        $_transcode++;
        substr($ldr,9,1,'a');
        $record->leader( $ldr );
    }

which is why the as_xml in elsewhere does the "USMARC" thing. Okay. I'll test
this, since I'm assuming Fridolin has already tested in a UNIMARC context
before submitting this patch.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18757] Problem when importing only items in MARC records

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18757

--- Comment #8 from M. Tompsett <[hidden email]> ---
BTW, it should be noted that this same problem shows itself as an "Internal
Server Error" if one tries to manually catalogue a record in UNIMARC, and uses
something clearly bad, like "KABOOM!" in the 100$a field. :)

substr outside of string at /usr/share/perl5/MARC/File/XML.pm line 561 (#2)
    (W substr)(F) You tried to reference a substr() that pointed outside of
    a string.  That is, the absolute value of the offset was larger than the
    length of the string.  See "substr" in perlfunc.  This warning is fatal if
    substr is used in an lvalue context (as the left hand side of an
    assignment or as a subroutine argument for example).

Use of uninitialized value $enc in string eq at
        /usr/share/perl5/MARC/File/XML.pm line 563 (#1)
Use of uninitialized value $enc in string eq at
        /usr/share/perl5/MARC/File/XML.pm line 565 (#1)
Use of uninitialized value $enc in concatenation (.) or string at
        /usr/share/perl5/MARC/File/XML.pm line 568 (#1)
Unsupported UNIMARC character encoding [] for XML output for UNIMARC; 100$a ->
kaboom at /usr/share/perl5/MARC/File/XML.pm line 568.

No need to make any 200 rules.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18757] Problem when importing only items in MARC records

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18757

--- Comment #9 from Fridolin SOMERS <[hidden email]> ---
(In reply to M. Tompsett from comment #7)

> which is why the as_xml in elsewhere does the "USMARC" thing. Okay. I'll
> test this, since I'm assuming Fridolin has already tested in a UNIMARC
> context before submitting this patch.

Indeed.
Several of our Koha UNIMARC have already this patch.

UNIMARC 100$a is a spaghetti dish ;-)

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/