[Bug 24555] New: POC Elasticsearch - Use boolean queries instead of full text queries

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] New: POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

            Bug ID: 24555
           Summary: POC Elasticsearch - Use boolean queries instead of
                    full text queries
 Change sponsored?: ---
           Product: Koha
           Version: master
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P5 - low
         Component: Searching - Elasticsearch
          Assignee: [hidden email]
          Reporter: [hidden email]

Here is a proof of concept for replacing current use of full text queries (with
query_string) by boolean queries on biblio searches.

Advantage:
    => No search craches with special character ( "!", ")" etc...)
    => Separate query context and filter context:
        - Search scores are not altered by facets,
        - Ability to use range filter  (i.e for publication date facet)

Also, this POC adds a button "ES query" in the search results header so you can
see the query sent to ES.

Code is not beautiful, there is no unit test, it's a POC.

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

--- Comment #1 from Alex Arnaud <[hidden email]> ---
Created attachment 98242
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=98242&action=edit
Bug 24555: Replace query string with boolean query

Test plan:
 Just make simple and advanced searches

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

--- Comment #2 from Alex Arnaud <[hidden email]> ---
Created attachment 98243
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=98243&action=edit
Bug 24555: add a button that shows ES query in search result

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

Alex Arnaud <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |Needs Signoff

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

Michal Denar <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Needs Signoff               |Signed Off

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

Michal Denar <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #98242|0                           |1
        is obsolete|                            |
  Attachment #98243|0                           |1
        is obsolete|                            |

--- Comment #3 from Michal Denar <[hidden email]> ---
Created attachment 98247
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=98247&action=edit
Bug 24555: Replace query string with boolean query

Test plan:
 Just make simple and advanced searches

Signed-off-by: Michal Denar <[hidden email]>

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

--- Comment #4 from Michal Denar <[hidden email]> ---
Created attachment 98248
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=98248&action=edit
Bug 24555: add a button that shows ES query in search result

Signed-off-by: Michal Denar <[hidden email]>

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

Michal Denar <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

Katrin Fischer <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
           Assignee|[hidden email]-commun |[hidden email]
                   |ity.org                     |

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

--- Comment #5 from Julian Maurice <[hidden email]> ---
Created attachment 104419
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104419&action=edit
Bug 24555: Use 'cross_fields' type for multi_match queries

It allows to search in all fields as if it were one big field.

It also uses 'and' as operator, which means that all searched terms must
appear at least in one field for a document to match

https://www.elastic.co/guide/en/elasticsearch/reference/6.8/query-dsl-multi-match-query.html#type-cross-fields

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

Julian Maurice <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #104419|0                           |1
        is obsolete|                            |

--- Comment #6 from Julian Maurice <[hidden email]> ---
Created attachment 104422
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=104422&action=edit
Bug 24555: Use 'cross_fields' type for multi_match queries

It allows to search in all fields as if it were one big field.

It also uses 'and' as operator, which means that all searched terms must
appear at least in one field for a document to match

https://www.elastic.co/guide/en/elasticsearch/reference/6.8/query-dsl-multi-match-query.html#type-cross-fields

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

--- Comment #7 from Julian Maurice <[hidden email]> ---
Created attachment 105061
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105061&action=edit
Bug 24555: Remove 'mc-' prefix in facet field name

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

--- Comment #8 from Julian Maurice <[hidden email]> ---
Created attachment 105470
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105470&action=edit
Bug 24555: Use operator "and" for match query

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

--- Comment #9 from Julian Maurice <[hidden email]> ---
Created attachment 105471
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105471&action=edit
Bug 24555: Fix search limits

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

Julian Maurice <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #105471|0                           |1
        is obsolete|                            |

--- Comment #10 from Julian Maurice <[hidden email]> ---
Created attachment 105472
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105472&action=edit
Bug 24555: Fix search limits

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

--- Comment #11 from Julian Maurice <[hidden email]> ---
Created attachment 105744
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=105744&action=edit
Bug 24555: Fix searches like 'field-name:term'

This is used for instance in Koha::Authorities::get_usage_count
("an:<authid>")

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 24555] POC Elasticsearch - Use boolean queries instead of full text queries

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24555

Julian Maurice <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
             Status|Signed Off                  |Needs Signoff

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/