[Bug 19581] New: Elasticsearch - Catmandu split option adds extra null fields to indexes

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 19581] New: Elasticsearch - Catmandu split option adds extra null fields to indexes

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19581

            Bug ID: 19581
           Summary: Elasticsearch - Catmandu split option adds extra null
                    fields to indexes
 Change sponsored?: ---
           Product: Koha
           Version: master
          Hardware: All
                OS: All
            Status: NEW
          Severity: normal
          Priority: P5 - low
         Component: Searching - Elasticsearch
          Assignee: [hidden email]
          Reporter: [hidden email]

This affects sorting, when encountering a null field the title in question will
sort out of place.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19581] Elasticsearch - Catmandu split option adds extra null fields to indexes

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19581

Nick Clemens <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |Needs Signoff
   Patch complexity|---                         |Small patch

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19581] Elasticsearch - Catmandu split option adds extra null fields to indexes

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19581

--- Comment #1 from Nick Clemens <[hidden email]> ---
Created attachment 68947
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=68947&action=edit
Bug 19581: Unit tests

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19581] Elasticsearch - Catmandu split option adds extra null fields to indexes

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19581

--- Comment #2 from Nick Clemens <[hidden email]> ---
Created attachment 68948
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=68948&action=edit
Bug 19581: Remove 'split' option from Catmandu fixes for marc

This option in combination with append seems to add null entries in the indexes
for fields that don't exist. These fields then affect sorting. Removing it
solves
the issue

To test:
1 - Have an ES setup
2 - Catalog and index some records
3 - View a record in ES:
    curl 'localhost:9200/koha_kohadev_biblios/data/4?pretty=yes'
4 - Note there are 'null' entries in the indexes, save this result for
comparison
5 - Apply patch
6 - Reindex, amking sure to delete existing indexes
    perl misc/search_tools/rebuild_elastic_search.pl -d
7 - View the record again
8 - Note the null entries are removed
9 - Note indices are otherwise intact

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19581] Elasticsearch - Catmandu split option adds extra null fields to indexes

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19581

Nick Clemens <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #68947|0                           |1
        is obsolete|                            |

--- Comment #3 from Nick Clemens <[hidden email]> ---
Created attachment 68949
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=68949&action=edit
Bug 19581: Unit tests

This patch makes a few changes to the tests to take into account the
change to indexing.

To test:
prove t/db_dependent/Koha_Elasticsearch_Indexer.t
Should return green

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19581] Elasticsearch - Catmandu split option adds extra null fields to indexes

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19581

David Bourgault <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #68948|0                           |1
        is obsolete|                            |

--- Comment #4 from David Bourgault <[hidden email]> ---
Created attachment 69092
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=69092&action=edit
Bug 19581: Remove 'split' option from Catmandu fixes for marc

This option in combination with append seems to add null entries in the indexes
for fields that don't exist. These fields then affect sorting. Removing it
solves
the issue

To test:
1 - Have an ES setup
2 - Catalog and index some records
3 - View a record in ES:
    curl 'localhost:9200/koha_kohadev_biblios/data/4?pretty=yes'
4 - Note there are 'null' entries in the indexes, save this result for
comparison
5 - Apply patch
6 - Reindex, amking sure to delete existing indexes
    perl misc/search_tools/rebuild_elastic_search.pl -d
7 - View the record again
8 - Note the null entries are removed
9 - Note indices are otherwise intact

Signed-off-by: David Bourgault <[hidden email]>

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19581] Elasticsearch - Catmandu split option adds extra null fields to indexes

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19581

David Bourgault <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #68949|0                           |1
        is obsolete|                            |

--- Comment #5 from David Bourgault <[hidden email]> ---
Created attachment 69093
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=69093&action=edit
Bug 19581: Unit tests

This patch makes a few changes to the tests to take into account the
change to indexing.

To test:
prove t/db_dependent/Koha_Elasticsearch_Indexer.t
Should return green

Signed-off-by: David Bourgault <[hidden email]>

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19581] Elasticsearch - Catmandu split option adds extra null fields to indexes

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19581

David Bourgault <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Needs Signoff               |Signed Off
                 CC|                            |[hidden email]

--- Comment #6 from David Bourgault <[hidden email]> ---
QA tools green
Tests passed
Work as described.

I could still values being set to null, but only as indicators of 00X fields,
which I assume is on purpose.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 19581] Elasticsearch - Catmandu split option adds extra null fields to indexes

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19581

Séverine Queune <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/