[Bug 17791] New: On UNIMARC system, USMARC field 100 gets overwritten

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] New: On UNIMARC system, USMARC field 100 gets overwritten

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

            Bug ID: 17791
           Summary: On UNIMARC system, USMARC field 100 gets overwritten
 Change sponsored?: ---
           Product: Koha
           Version: unspecified
          Hardware: All
                OS: All
            Status: NEW
          Severity: major
          Priority: P5 - low
         Component: Z39.50 / SRU / OpenSearch Servers
          Assignee: [hidden email]
          Reporter: [hidden email]
        QA Contact: [hidden email]
                CC: [hidden email]

Overview:
On Z39.50 search some functions depend not on the remote server marc flavour,
but on current server setting. This bug is present on versions 3.22+

Steps to Reproduce:
1) On UNIMARC system, go to Cataloguing → New from Z39.50/SRU
2) Search USMARC server (Library of Congress, for example) for a book, that has
field 100 (for example, isbn 0816310653)
3) On results list, press «MARC» to see MARC fields

Actual Results:
Instead of seeing the author name in subfield A, you'll see something like
«20161219              frey50        », yet you could see other subfields
properly

Expected Results:
Name of the author in 100$a

Additional Information:
LC Z39.50/SRW/SRU Server Configuration Guidelines:
https://www.loc.gov/z3950/lcserver.html

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

Oleg Vasilenko <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|On UNIMARC system, USMARC   |On UNIMARC system, USMARC
                   |field 100 gets overwritten  |field 100$a gets
                   |                            |overwritten

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

--- Comment #1 from Oleg Vasilenko <[hidden email]> ---
Created attachment 58276
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=58276&action=edit
Patch changes the context of Z39.50 search from local to remote  server marc
flavour

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

Oleg Vasilenko <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Change sponsored?|---                         |Sponsored
             Status|NEW                         |Needs Signoff
   Patch complexity|---                         |String patch

--- Comment #2 from Oleg Vasilenko <[hidden email]> ---
Patch prepared for current master

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

Zeno Tajoli <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
             Status|Needs Signoff               |Failed QA

--- Comment #3 from Zeno Tajoli <[hidden email]> ---
Hi Oleg,

I'm not agree with this patch.
Why?
Now «20161219              frey50        » is correct default for Unimarc 100
field
If you have a Unimarc server and you want to use LOC [a MARC21 z39.50] you need
to use the field "XSLT file(s)" in z39.50 definiton.
See: http://manual.koha-community.org/3.22/en/additionaladmin.html#addztarget
In bug 6536 there is an example of XSLT for MARC21 -> Unimarc conversion.
See: https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=37264

Bye

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

Oleg Vasilenko <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Failed QA                   |Needs Signoff

--- Comment #4 from Oleg Vasilenko <[hidden email]> ---
Hi Zeno,

This change is normal if you access UNIMARC record on Unimarc server. But the
problem is that this is changed before XSLT conversion.

I do xslt conversion, but it converts this already changed string. You can
check by yourself
https://github.com/Gelow/unimarc/blob/master/marc21xml2unimarc.xsl
MARC view in Z39.50 search results shows the record as it is.

Koha code works the way, that it first converts codepage, and after that
changes the subfield where it is stated, that it is UTF-8. But the problem is
that it thinks that the record is in unimarc format, so it changes 100$a.

The patch uses server flavour, that is stated in Z39.50 servers table. That way
everythig works fine. Function, that do codepage conversion understands that
this record is USMARC (as stated in Z39.50 server table), and it changes
leader, not 100$a. Because information about codepage is located in different
places in these flavours.

You can see that this problem is present in different koha versions. But the
patch is prepared for current master

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

--- Comment #5 from Zeno Tajoli <[hidden email]> ---
Ok Oleg, you are correct.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

--- Comment #6 from Oleg Vasilenko <[hidden email]> ---
When you use XSLT conversion, you'll see the same problem

USMARC 100 is converted to UNIMARC 700. But the conversion happens after
codepage is changed. So you'll see
700$a 20161219              frey50        
700$b 1933-

but not the author name in 700$a

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

--- Comment #7 from Marcel de Rooy <[hidden email]> ---
Oleg,
This seems to me an tricky change.
What you are trying to do, is currently not fully supported in Koha.
A conversion between MARC21 and UNIMARC did not make it until now.
If you do a XSLT conversion after this statement, it will probably be fine, but
in many other cases it will not be followed by such a conversion and new
problems will probably arise.
To make this foolproof, we need more changes (and checks)?

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

--- Comment #8 from Oleg Vasilenko <[hidden email]> ---
Marcel,
I am working on the MARC21 to UNIMARC conversion right now, and now it works
good enough. I did many changes to original XSLT file published by Terry Reese
on github (bug 16488). See
https://github.com/Gelow/unimarc/blob/master/marc21xml2unimarc.xsl

The only problem is that field 100$a is corrupted on Z39.50 search. I added LOC
server as SRU and field 100$a is not corrupted that way. This is because in sub
_handle_one_result() right now SRU server is processed with it's flavour in
mind. The same should be with Z39.50

What problems will we face after this patch?

The same as we have with SRU servers right now.
When UNIMARC system connects to MARC21 server, it will check encoding stated in
100$a (thanks to checks in XML.pm called from function AddBiblioToBatch()).If
it's not set, you'll see XML encoding error («Unsupported UNIMARC character
encoding [$enc] for XML output»).

In other cases everything should be fine

If we do not apply this patch
You can see the results table, and see sourse MARC with corrupted 100$a.
But the table has empty columns because of different format, so who does need
it?

To overcome stated problems it is possible to add some check in the code for
UNIMARC and process the record some way (but record could become corrupted
again). Or add XSLT file for conversion

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

Chris Cormack <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
           Severity|major                       |new feature

--- Comment #9 from Chris Cormack <[hidden email]> ---
Changing to a new feature as this is adding support for on the fly conversion
of MARC records from Z39.50

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

Serhij Dubyk <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

Katrin Fischer <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
           Severity|new feature                 |enhancement

--- Comment #10 from Katrin Fischer <[hidden email]> ---
Changing to enh as this is just the starting point for adding the ability of
the conversion (and should then change bug title also). After this made it in
we could maybe add Oleg's XSLT file to the sample xsl files available and mark
that 'new feature'?

Signing off, was not able to detect a regression.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

Katrin Fischer <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Needs Signoff               |Signed Off

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

--- Comment #11 from Katrin Fischer <[hidden email]> ---
Created attachment 68165
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=68165&action=edit
Bug 17791 - On UNIMARC system, USMARC field 100$a gets overwritten Patch
changes the context of Z39.50 search from local to remote server

Steps to Reproduce:
1) On UNIMARC system, go to Cataloguing → New from Z39.50/SRU
2) Search USMARC server (Library of Congress, for example) for a book, that has
field 100 (for example, isbn 0816310653)
3) On results list, press «MARC» to see MARC fields

Expected Results:
Name of the author in 100$a

Signed-off-by: Katrin Fischer <[hidden email]>

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten when importing from MARC21 target

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

Katrin Fischer <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|On UNIMARC system, USMARC   |On UNIMARC system, USMARC
                   |field 100$a gets            |field 100$a gets
                   |overwritten                 |overwritten when importing
                   |                            |from MARC21 target

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten when importing from MARC21 target

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

Marcel de Rooy <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #58276|0                           |1
        is obsolete|                            |

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten when importing from MARC21 target

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

Marcel de Rooy <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|[hidden email]          |[hidden email]
         QA Contact|[hidden email]-communit |[hidden email]
                   |y.org                       |
             Status|Signed Off                  |Failed QA

--- Comment #12 from Marcel de Rooy <[hidden email]> ---
This does not work. As said earlier, we need more checks. (The records cannot
be imported without conversion. So we need more checks. For instance, only show
import when the flavours match.)

Currently btw this results in an internal server error.
I started on a clean master, chose UNIMARC and added LOC as a search target
(MARC21/USMARC).
Without this patch, I can search, but of course I do not see any titles.

With this patch, it crashes.
Unrecognized marcflavour: USMARC at /usr/share/koha/masterclone/C4/Charset.pm
line 358.
Use of uninitialized value in substr at /usr/share/perl5/MARC/File/XML.pm line
561.
substr outside of string at /usr/share/perl5/MARC/File/XML.pm line 561.
Use of uninitialized value $enc in string eq at
/usr/share/perl5/MARC/File/XML.pm line 563.
Use of uninitialized value $enc in string eq at
/usr/share/perl5/MARC/File/XML.pm line 563.
Use of uninitialized value $enc in string eq at
/usr/share/perl5/MARC/File/XML.pm line 565.
Use of uninitialized value $enc in concatenation (.) or string at
/usr/share/perl5/MARC/File/XML.pm line 568.
Use of uninitialized value in concatenation (.) or string at
/usr/share/perl5/MARC/File/XML.pm line 568.
Unsupported UNIMARC character encoding [] for XML output for UNIMARC; 100$a ->
at /usr/share/perl5/MARC/File/XML.pm line 568.

Script passes USMARC as marc flavour to MarcToUTF8Record.
This routine passes it to SetMarcUnicodeFlag which expects MARC21 or UNIMARC as
Koha does in other places too.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 17791] On UNIMARC system, USMARC field 100$a gets overwritten when importing from MARC21 target

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17791

Marcel de Rooy <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|unspecified                 |master
   Patch complexity|String patch                |Small patch

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/