[Bug 19604] New: Elasticsearch Fixes for build_authorities_query for auth searching

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 19604] New: Elasticsearch Fixes for build_authorities_query for auth searching

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19604

            Bug ID: 19604
           Summary: Elasticsearch Fixes for build_authorities_query for
                    auth searching
 Change sponsored?: ---
           Product: Koha
           Version: master
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P5 - low
         Component: Searching - Elasticsearch
          Assignee: [hidden email]
          Reporter: [hidden email]

Some of the searches here were based on older syntax and caused errors

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19604] Elasticsearch Fixes for build_authorities_query for auth searching

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19604

Nick Clemens <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |Needs Signoff
         Depends on|                            |19582


Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19582
[Bug 19582] Elasticsearch: Auth-finder.pl must use search_auth_compat
--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19604] Elasticsearch Fixes for build_authorities_query for auth searching

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19604

--- Comment #1 from Nick Clemens <[hidden email]> ---
Created attachment 69072
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=69072&action=edit
Bug 19604: Elasticsearch Fixes for build_authorities_query for auth searching

To test:
1 - Do some authority searches in Zebra
2 - Switch to ES and repeat, results will vary and some may fail
3 - Apply patch and dependencies
4 - Reindex ES
5 - Repeat searches, they should suceed and results should be similar to
Zebra
6 - Slight differences are okay, but results should (mostly) meet
expectations

A few notes:
We add a 'normalizer' to ensure we get a single token from the heading
indexes, this makes 'starts with' work as expcted
We switch to 'AND' for fields searched from cataloging editor - this
matches Zebra results
We force the '__sort' fields for sorting - if sorting looks wrong try
reducing the heading field to a single subfield - this will need to be
addressed on a future bug (multiple subfields create an array, ES sorts
        those randomly)

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19604] Elasticsearch Fixes for build_authorities_query for auth searching

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19604

David Bourgault <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
             Status|Needs Signoff               |Failed QA

--- Comment #2 from David Bourgault <[hidden email]> ---
I did a search for 'me' from the OPAC's Auth search.

With Zebra I got 1411 results (some, but not all, matching on 'mé')
With ES I got a single result, in which the word 'Me' was found as-is

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19604] Elasticsearch Fixes for build_authorities_query for auth searching

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19604

Nick Clemens <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Failed QA                   |Needs Signoff

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19604] Elasticsearch Fixes for build_authorities_query for auth searching

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19604

Nick Clemens <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #69072|0                           |1
        is obsolete|                            |

--- Comment #3 from Nick Clemens <[hidden email]> ---
Created attachment 69198
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=69198&action=edit
Bug 19604: Elasticsearch Fixes for build_authorities_query for auth searching

To test:
1 - Do some authority searches in Zebra
2 - Switch to ES and repeat, results will vary and some may fail
3 - Apply patch and dependencies
4 - Reindex ES
5 - Repeat searches, they should suceed and results should be similar to
Zebra
6 - Slight differences are okay, but results should (mostly) meet
expectations

A few notes:
We add a 'normalizer' to ensure we get a single token from the heading
indexes, this makes 'starts with' work as expcted
We switch to 'AND' for fields searched from cataloging editor - this
matches Zebra results
We force the '__sort' fields for sorting - if sorting looks wrong try
reducing the heading field to a single subfield - this will need to be
addressed on a future bug (multiple subfields create an array, ES sorts
        those randomly)

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19604] Elasticsearch Fixes for build_authorities_query for auth searching

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19604

--- Comment #4 from Nick Clemens <[hidden email]> ---
Created attachment 69199
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=69199&action=edit
Bug 19604: Unit tests

1 - Apply previous patches
2 - prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t
3 - Should be green
4 - high fives!

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19604] Elasticsearch Fixes for build_authorities_query for auth searching

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19604

--- Comment #5 from Nick Clemens <[hidden email]> ---
(In reply to David Bourgault from comment #2)
> I did a search for 'me' from the OPAC's Auth search.
>
> With Zebra I got 1411 results (some, but not all, matching on 'mé')
> With ES I got a single result, in which the word 'Me' was found as-is

I altered the 'contains' search to be a wildcard and break terms - it should be
closer now but won't mirror exactly zebra

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19604] Elasticsearch Fixes for build_authorities_query for auth searching

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19604

David Bourgault <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Needs Signoff               |Failed QA

--- Comment #6 from David Bourgault <[hidden email]> ---
Tests passed, QA tools green.

However, whereas I had 1 exact 1, I now have 0 search results.

I tried reseting the mappings, I reindexed -d -a -v twice but I can't get it to
find results.

Testing in Kibana with the query `Heading:*me*`, I got multiple results
(including both 'me' and 'mé') so I don't think it's a problem specific to my
index/database.

Failed QA :(

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19604] Elasticsearch Fixes for build_authorities_query for auth searching

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19604

--- Comment #7 from Nick Clemens <[hidden email]> ---
(In reply to David Bourgault from comment #6)

> Tests passed, QA tools green.
>
> However, whereas I had 1 exact 1, I now have 0 search results.
>
> I tried reseting the mappings, I reindexed -d -a -v twice but I can't get it
> to find results.
>
> Testing in Kibana with the query `Heading:*me*`, I got multiple results
> (including both 'me' and 'mé') so I don't think it's a problem specific to
> my index/database.
>
> Failed QA :(

Can you let me know the exact search and terms you are using? and the data in
your DB? with patches applied and default data from kohadevbox I get results
for 'me' or 'mé' and all are as expected

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/