[Bug 18189] New: Elasticsearch sorting broken

classic Classic list List threaded Threaded
24 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] New: Elasticsearch sorting broken

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

            Bug ID: 18189
           Summary: Elasticsearch sorting broken
 Change sponsored?: ---
           Product: Koha
           Version: master
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P5 - low
         Component: Searching
          Assignee: [hidden email]
          Reporter: [hidden email]
        QA Contact: [hidden email]

Current setting in elasticsearch mapping use 'text' type for sort fields, this
seems like extra overhead.

If the fields are only used for sorting we can index them as 'keyword' (exact
phrase) and this has the side effect of fixing sorting

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

Nick Clemens <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |Needs Signoff
   Patch complexity|---                         |Trivial patch

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

--- Comment #1 from Nick Clemens <[hidden email]> ---
Created attachment 60780
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=60780&action=edit
Bug 18189 - Elasticsearch sorting broken

To test:
1 - Have ES enabled and biblios and authorities indexed
2 - Perform a catalog search
3 - Try to sort results by author, it will fail
4 - Perform an authority search with sorting enabled, it will fail
5 - Apply patch
6 - Reindex, deleting current indexes
    perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl
    -d -v
7 - Repeat 3&4, they should succeed

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

Nick Clemens <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|[hidden email]          |[hidden email]

--- Comment #2 from Nick Clemens <[hidden email]> ---
Note this won't fix all sorting options unless they are defined as sortable in
the config.

acqdate fails out of the box - set acqdate as sortable - reindex - it should
now work

I suppose we either don't allow altering sortability for some fields, or the
dropdown must populate based on what is defined as sortable.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

Jonathan Druart <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]-c
                   |                            |ommunity.org
         Depends on|                            |17255


Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17255
[Bug 17255] Upgrade Elastic Search code to work with version 5.1
--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

Srdjan Jankovic <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #3 from Srdjan Jankovic <[hidden email]> ---
I'm getting:
Fielddata is disabled on text fields by default. Set fielddata=true on
[author.phrase] in order to load fielddata in memory by uninverting the
inverted index. Note that this can however use significant memory.

Is something wrong with my es config?

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

--- Comment #4 from Nick Clemens <[hidden email]> ---
(In reply to Srdjan Jankovic from comment #3)
> I'm getting:
> Fielddata is disabled on text fields by default. Set fielddata=true on
> [author.phrase] in order to load fielddata in memory by uninverting the
> inverted index. Note that this can however use significant memory.
>
> Is something wrong with my es config?

Is this before or after the patch? I cannot seem to replicate the issue.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

--- Comment #5 from Srdjan Jankovic <[hidden email]> ---
(In reply to Nick Clemens from comment #4)
> (In reply to Srdjan Jankovic from comment #3)
> > I'm getting:
> > Fielddata is disabled on text fields by default. Set fielddata=true on
> > [author.phrase] in order to load fielddata in memory by uninverting the
> > inverted index. Note that this can however use significant memory.
> >
> > Is something wrong with my es config?
>
> Is this before or after the patch? I cannot seem to replicate the issue.

Both. What was the error that you were trying to get rid of?

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

--- Comment #6 from Nick Clemens <[hidden email]> ---
Before this patch passing sort options to Elastic gives an error much like the
one you mentioned. If you are getting this before and after the patch I would
suspect it is an issue with modules/versions, and or outdated mappings.

Using
admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1

should get your mappings up to date, then delete and rebuild indexes, and you
should be good

I would also check versions against kohadevbox elastic setup:
https://github.com/digibib/kohadevbox/blob/master/roles/kohadevbox/tasks/elasticsearch-2.x.yml

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

Mason James <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

--- Comment #7 from Srdjan Jankovic <[hidden email]> ---
Hm, done the reset thing, and now getting:
No mapping found for [author__sort.phrase] in order to sort on, with:
{"index":"koha_biblios","index_uuid":"fehRdZHCRlGe1FxHYiKbMA"}

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

Srdjan Jankovic <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Needs Signoff               |Signed Off

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

Srdjan Jankovic <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #60780|0                           |1
        is obsolete|                            |

--- Comment #8 from Srdjan Jankovic <[hidden email]> ---
Created attachment 61494
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=61494&action=edit
[SIGNED-OFF] Bug 18189 - Elasticsearch sorting broken

To test:
1 - Have ES enabled and biblios and authorities indexed
2 - Perform a catalog search
3 - Try to sort results by author, it will fail
4 - Perform an authority search with sorting enabled, it will fail
5 - Apply patch
6 - Reindex, deleting current indexes
    perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl
    -d -v
7 - Repeat 3&4, they should succeed

Signed-off-by: Srdjan <[hidden email]>

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

--- Comment #9 from Srdjan Jankovic <[hidden email]> ---
Sorry, my fault, omitted -d when reindexing.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

J Schmidt <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

--- Comment #10 from Jonathan Druart <[hidden email]> ---
What about integers and dates?

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

--- Comment #11 from Jonathan Druart <[hidden email]> ---
Does only work with author, our other fields are not marked as sortable by
default (?!?)

Moreover "Ó Cadhain, Máirtín" is sorted after "Yerby, Frank Garvin",
terrible...

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

Jonathan Druart <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Signed Off                  |Passed QA

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

Jonathan Druart <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #61494|0                           |1
        is obsolete|                            |

--- Comment #12 from Jonathan Druart <[hidden email]> ---
Created attachment 61689
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=61689&action=edit
Bug 18189 - Elasticsearch sorting broken

To test:
1 - Have ES enabled and biblios and authorities indexed
2 - Perform a catalog search
3 - Try to sort results by author, it will fail
4 - Perform an authority search with sorting enabled, it will fail
5 - Apply patch
6 - Reindex, deleting current indexes
    perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl
    -d -v
7 - Repeat 3&4, they should succeed

Signed-off-by: Srdjan <[hidden email]>

Signed-off-by: Jonathan Druart <[hidden email]>

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

Kyle M Hall <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Passed QA                   |Pushed to Master
                 CC|                            |[hidden email]

--- Comment #13 from Kyle M Hall <[hidden email]> ---
Pushed to master for 17.05, thanks Nick!

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

Katrin Fischer <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #14 from Katrin Fischer <[hidden email]> ---
Is enh correct here or should this not be a bug?

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

Jonathan Druart <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|enhancement                 |normal

--- Comment #15 from Jonathan Druart <[hidden email]> ---
Sort is broken, it's a bug.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

--- Comment #16 from Katrin Fischer <[hidden email]> ---
Ok, missed the dependency on bug 17255 earlier - should we still push this to
stable?

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18189] Elasticsearch sorting broken

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189

Katrin Fischer <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|Pushed to Master            |RESOLVED

--- Comment #17 from Katrin Fischer <[hidden email]> ---
3 months - still no answer here. Closing.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/