[Bug 21716] New: Item Search hangs when \ exists in MARC fields

classic Classic list List threaded Threaded
22 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] New: Item Search hangs when \ exists in MARC fields

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

            Bug ID: 21716
           Summary: Item Search hangs when \ exists in MARC fields
 Change sponsored?: ---
           Product: Koha
           Version: master
          Hardware: All
                OS: All
            Status: NEW
          Severity: normal
          Priority: P5 - low
         Component: Staff Client
          Assignee: [hidden email]
          Reporter: [hidden email]
        QA Contact: [hidden email]
                CC: [hidden email]

When there are \ (backslashes) in MARC records the item search hangs when
trying to load the page the record should appear on. It gets stuck on
processing.

My specific example has a Publisher written as Narr\Francke\Attempto in 260 $b

The error is:

SyntaxError: JSON.parse: bad escaped character at line 8 column 9 of the JSON
data


Tested on 16.11 17.11 and master.

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

Martin Renvoize <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |martin.renvoize@ptfs-europe
                   |                            |.com

--- Comment #1 from Martin Renvoize <[hidden email]> ---
OMG, we're using a TT template to convert Perl to JSON.. no wonder this doesn't
work properly :'(

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

--- Comment #2 from Martin Renvoize <[hidden email]> ---
JSON.Escape and the `| json` filter is probably your friend here... or go for
the bigger job and convert it to using a json parser properly.

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

David Cook <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

Claire Gravely <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |Needs Signoff

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

--- Comment #3 from Claire Gravely <[hidden email]> ---
Created attachment 82776
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82776&action=edit
Bug 21716: Item Search hangs when \ exists in MARC fields

When there are \ (backslashes) in MARC records the item search hangs
when trying to load the page the record should appear on. It gets stuck
on processing. This patch fixes this issue.

To Test:
1. Create or edit a biblio and add in \ to the publisher field. eg:
Narr\Francke\Attempto
2. Make sure this biblio has items attached that you can search for
3. Do an item search which would include the edited biblio in the
results
    - a general search for a home library or shelving location will work
4. Try to load All results (or the page with the edited biblio)
   The page will hang :-(
5. Apply patch
6. Perform the item search again
7. Try to load All results
   The page will load :-)

Mentored by: Martin Renvoize <[hidden email]>

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

Martin Renvoize <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #82776|0                           |1
        is obsolete|                            |

--- Comment #4 from Martin Renvoize <[hidden email]> ---
Created attachment 82779
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82779&action=edit
Bug 21716: Item Search hangs when \ exists in MARC fields

When there are \ (backslashes) in MARC records the item search hangs
when trying to load the page the record should appear on. It gets stuck
on processing. This patch fixes this issue.

To Test:
1. Create or edit a biblio and add in \ to the publisher field. eg:
Narr\Francke\Attempto
2. Make sure this biblio has items attached that you can search for
3. Do an item search which would include the edited biblio in the
results
    - a general search for a home library or shelving location will work
4. Try to load All results (or the page with the edited biblio)
   The page will hang :-(
5. Apply patch
6. Perform the item search again
7. Try to load All results
   The page will load :-)

Mentored-by: Martin Renvoize <[hidden email]>
Signed-off-by: Martin Renvoize <[hidden email]>

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

Martin Renvoize <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Needs Signoff               |Signed Off

--- Comment #5 from Martin Renvoize <[hidden email]> ---
This works as expected and is a helpful addition... I've tested it and am
signing off.

QA Note.. this is sort of horrible, that's we're generating JSON by passing it
through a template.. but that's how it currently is and it would be a fairly
major refactor to undertake for such a minor fix.

We could apply the 'json' filter further too, but I believe that should be as a
followup somewhere myself, rather than confusing the issue here.

Might be nice to add regression tests too.. wonder if Jonathan or someone could
mentor writing those (I must admit, to an extent the Selenium tests are still a
mystery to me).

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

Claire Gravely <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|[hidden email]-commun |[hidden email]
                   |ity.org                     |

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

Julian Maurice <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
             Status|Signed Off                  |Failed QA

--- Comment #6 from Julian Maurice <[hidden email]> ---
(In reply to Martin Renvoize from comment #5)
> This works as expected and is a helpful addition... I've tested it and am
> signing off.
It doesn't work for me. The page loads, yes. But publisher and publication year
have disappeared.

> QA Note.. this is sort of horrible, that's we're generating JSON by passing
> it through a template.. but that's how it currently is and it would be a
> fairly major refactor to undertake for such a minor fix.
I agree this is horrible, and I'm the one responsible for it :)
When I wrote the patch, I believed it was the only way to have translatable
strings in a server-side Datatable setup.
Fortunately I was wrong, and we can make use of the datatable's 'mRender'
option to separate the data (what should be returned in the JSON response) from
the rendered html (which should contain translated strings)
It does not require a big refactor. I will submit a patch soon.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

Julian Maurice <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Failed QA                   |Needs Signoff

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

Julian Maurice <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #82779|0                           |1
        is obsolete|                            |

--- Comment #7 from Julian Maurice <[hidden email]> ---
Created attachment 82980
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=82980&action=edit
Bug 21716: Do not use TT to produce JSON for item search results

It was initially done this way to have translatable strings in the
results table, but this method has flaws and it can be done in a better
way.

This patch moves the rendering code to client-side javascript, so the
JSON returned by the server now contains only unprocessed data.

Also, this patch makes use of the new i18n macros for translation and
fixes a whitespace bug in i18n.inc

Reported-by: Claire Gravely <[hidden email]>

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

--- Comment #8 from Martin Renvoize <[hidden email]> ---
Oh nice :)

Fancy testing Claire and I'll grab it for QA?

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

Owen Leonard <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Needs Signoff               |Failed QA

--- Comment #9 from Owen Leonard <[hidden email]> ---
The QA tool says there are missing filters, for example:
  missing_filter at line 383 ( output += ' ' + [% t('by').json %]; )

it should be:

[% t('by').json | $raw %]

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

--- Comment #10 from Julian Maurice <[hidden email]> ---
Created attachment 83147
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=83147&action=edit
Bug 21716: Add missing $raw filters

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

Julian Maurice <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Failed QA                   |Needs Signoff

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

Liz Rea <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
             Status|Needs Signoff               |Failed QA

--- Comment #11 from Liz Rea <[hidden email]> ---
Hi,

I've just checked this patch, and it seems that the hang doesn't occur on
current master, but it is broken again without the patch (infinite processing).

Is this still needed? It appears to me that it isn't.

Cheers,
Liz

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

Katrin Fischer <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://bugs.koha-community
                   |                            |.org/bugzilla3/show_bug.cgi
                   |                            |?id=20891

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

--- Comment #12 from Katrin Fischer <[hidden email]> ---
Maybe the fix from bug 20891 would be helpful here.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

Liz Rea <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Failed QA                   |Needs Signoff

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

Liz Rea <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #82980|0                           |1
        is obsolete|                            |
  Attachment #83147|0                           |1
        is obsolete|                            |

--- Comment #13 from Liz Rea <[hidden email]> ---
Created attachment 88790
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=88790&action=edit
Bug 21716: Escape publisher in item search

To prevent endless processing errors.

To test:

* create a biblio with a publisher with a backslash - i.e. 260$b with
graham\lineham
* do an item search that will include that title in the results
* wait forever, because it won't work.
* apply this patch
* repeat steps, notice that you
    a. get results
    b. your result is listed with it's correct publisher
* rejoice

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 21716] Item Search hangs when \ exists in MARC fields

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21716

--- Comment #14 from Claire Gravely <[hidden email]> ---
Hi, I tried testing but with the patch applied I get no results at all (even
the pages without my dodgy biblio). The item search results page hangs on
processing and in the console I get a response 'internal server error'

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/