[Bug 22690] New: Merging problem with records with many items (ElasticSearch)

classic Classic list List threaded Threaded
73 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] New: Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

            Bug ID: 22690
           Summary: Merging problem with records with many items
                    (ElasticSearch)
 Change sponsored?: ---
           Product: Koha
           Version: unspecified
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P5 - low
         Component: Cataloging
          Assignee: [hidden email]
          Reporter: [hidden email]
        QA Contact: [hidden email]
                CC: [hidden email]
  Target Milestone: ---

When merging records with many items, Koha reindexes the origin record and the
destination record for each item moving (check MoveItemFromBiblio in
C4::Items).
If the record being eliminated has 1000 items, Koha will make 2000 reindexes.
Because of this behaviour, the merging is too slow.

Test plan:

- Create 2 records (one with 1000 items for example)
- Add the records to one list and start the merging process of those records.
- Choose the record with many items as the one to be eliminated.
- Start the merging
- After a while the web server should give you a timeout error (the merging
process continues)

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

Owen Leonard <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|enhancement                 |normal
            Version|unspecified                 |master

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

Katrin Fischer <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|Cataloging                  |Searching - Elasticsearch
         QA Contact|[hidden email]-communit |
                   |y.org                       |
                 CC|                            |[hidden email]

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

axel Amghar <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #1 from axel Amghar <[hidden email]> ---
hello,
when i try to merge records, i have an issue :
 Following required fields are missing:
-801


https://snag.gy/76Fh9R.jpg
this is a mandatory tag but i can't add it
Do someone know why?

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

--- Comment #2 from Vitor Fernandes <[hidden email]> ---
(In reply to axel Amghar from comment #1)
> hello,
> when i try to merge records, i have an issue :
>  Following required fields are missing:
> -801
>
>
> https://snag.gy/76Fh9R.jpg
> this is a mandatory tag but i can't add it
> Do someone know why?

Axel that happens because the framework chosen for merging has 801 as mandatory
field (that's my guess).

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

--- Comment #3 from axel Amghar <[hidden email]> ---
(In reply to Vitor Fernandes from comment #2)

> Axel that happens because the framework chosen for merging has 801 as
> mandatory field (that's my guess).

I think is the same thing with "000" and "001".
They are mandatory but we can't add them.
So..
Do we have to fix it ?

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

--- Comment #4 from axel Amghar <[hidden email]> ---
Created attachment 90061
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=90061&action=edit
Bug 22690 - Merging records with many items (ElasticSearch)

This patch allow us to merged many items without timeout

Test plan :

Without the patch :

> - Create 2 records (one with 1000 items for example)
> - Add the records to one list and start the merging process of those records.
> - Choose the record with many items as the one to be eliminated.
> - Start the merging
> - After a while the web server should give you a timeout error (the merging process continues)

With the patch :
- make the same
- this time verify that all items have been merged without time out

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

axel Amghar <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|[hidden email]-commun |[hidden email]
                   |ity.org                     |

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

--- Comment #5 from axel Amghar <[hidden email]> ---
Created attachment 90115
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=90115&action=edit
Bug 22690: Merging records with many items (ElasticSearch)

This patch allow us to merged many items without timeout

Test plan :

Without the patch :

> - Create 2 records (one with 1000 items for example)
> - Add the records to one list and start the merging process of those records.
> - Choose the record with many items as the one to be eliminated.
> - Start the merging
> - After a while the web server should give you a timeout error (the merging process continues)

With the patch :
- make the same
- this time verify that all items have been merged without time out

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

axel Amghar <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #90061|0                           |1
        is obsolete|                            |

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

axel Amghar <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #90115|0                           |1
        is obsolete|                            |

--- Comment #6 from axel Amghar <[hidden email]> ---
Created attachment 90116
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=90116&action=edit
Bug 22690: Merging records with many items (ElasticSearch)

This patch allow us to merged many items without timeout

Test plan :

Without the patch :

> - Create 2 records (one with 1000 items for example)
> - Add the records to one list and start the merging process of those records.
> - Choose the record with many items as the one to be eliminated.
> - Start the merging
> - After a while the web server should give you a timeout error (the merging process continues)

With the patch :
- make the same
- this time verify that all items have been merged without time out

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

--- Comment #7 from axel Amghar <[hidden email]> ---
Hello,
I found an issue when you click on merge selected, and then you click instant
on merge,
the destination record doesn't have the time to load.
And when you look your record, it lose his MARC record.
Someone else succeed to reproduce this bug?

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

axel Amghar <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |Needs Signoff

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

Michal Denar <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

Alex Arnaud <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

Ere Maijala <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Blocks|                            |20447


Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20447
[Bug 20447] Add support for MARC holdings records
--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging problem with records with many items (ElasticSearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

Ere Maijala <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Needs Signoff               |Failed QA

--- Comment #8 from Ere Maijala <[hidden email]> ---
I don't get the logic of the $items_number parameter. As far as I can see the
first UPDATE moves all items regardless of the parameter, but the subsequent
processing of acquisitions etc. only goes through items in the $items_number
parameter. If MoveItemsFromBiblio is supposed to move all items, I'd remove the
$items_number parameter and do everything necessary inside MoveItemsFromBiblio.

Also, I believe the 801 issue should be handled in a separate bug since it's
not related to Elasticsearch.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging records with many items too slow (Elasticsearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

Ere Maijala <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Merging problem with        |Merging records with many
                   |records with many items     |items too slow
                   |(ElasticSearch)             |(Elasticsearch)

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging records with many items too slow (Elasticsearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

Ere Maijala <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|[hidden email]    |[hidden email]
             Status|Failed QA                   |ASSIGNED

--- Comment #9 from Ere Maijala <[hidden email]> ---
Taking this since I need this for bug 20447.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging records with many items too slow (Elasticsearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

Ere Maijala <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #90116|0                           |1
        is obsolete|                            |

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging records with many items too slow (Elasticsearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

Ere Maijala <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |Needs Signoff

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging records with many items too slow (Elasticsearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

--- Comment #10 from Ere Maijala <[hidden email]> ---
Created attachment 92828
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=92828&action=edit
Bug 22690: Refactor merging of records to improve performance (Elasticsearch)

This patch allows merging of records with many items without the web server
timing out.

Test plan:

Without the patch:

- Create 2 records (one with e.g. 1000 items).
- Do a cataloguing search that displays both records, select them and click
"Merge selected".
- Choose the record with many items as the one to be eliminated.
- Start the merging.
- After a while the web server should give you a timeout error (the merging
process may still continue)

With the patch:
- Do the same as above
- This time verify that the records are merged without timeout
- Create a new biblio with an item
- Add with the item:
  * acquisition order
  * hold (reserve)
- Merge the biblio to another one
- Verify that the item and its related data was moved
- Verify that tests pass:
  prove -v t/db_dependent/Koha/Item.t
  prove -v t/db_dependent/Items/MoveItemFromBiblio.t

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging records with many items too slow (Elasticsearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

Ere Maijala <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #92828|0                           |1
        is obsolete|                            |

--- Comment #11 from Ere Maijala <[hidden email]> ---
Created attachment 94505
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=94505&action=edit
Bug 22690: Refactor merging of records to improve performance (Elasticsearch)

This patch allows merging of records with many items without the web server
timing out.

Test plan:

Without the patch:

- Create 2 records (one with e.g. 1000 items).
- Do a cataloguing search that displays both records, select them and click
"Merge selected".
- Choose the record with many items as the one to be eliminated.
- Start the merging.
- After a while the web server should give you a timeout error (the merging
process may still continue)

With the patch:
- Do the same as above
- This time verify that the records are merged without timeout
- Create a new biblio with an item
- Add with the item:
  * acquisition order
  * hold (reserve)
- Merge the biblio to another one
- Verify that the item and its related data was moved
- Verify that tests pass:
  prove -v t/db_dependent/Koha/Item.t
  prove -v t/db_dependent/Items/MoveItemFromBiblio.t

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging records with many items too slow (Elasticsearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

Joonas Kylmälä <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Needs Signoff               |Patch doesn't apply
                 CC|                            |[hidden email]

--- Comment #12 from Joonas Kylmälä <[hidden email]> ---
Doesn't apply anymore:

Apply? [(y)es, (n)o, (i)nteractive] y
Applying: Bug 22690: Refactor merging of records to improve performance
(Elasticsearch)
Using index info to reconstruct a base tree...
M       C4/Items.pm
M       Koha/Biblio.pm
M       Koha/Item.pm
M       t/db_dependent/Koha/Item.t
Falling back to patching base and 3-way merge...
Auto-merging t/db_dependent/Koha/Item.t
Auto-merging Koha/Item.pm
CONFLICT (content): Merge conflict in Koha/Item.pm
Auto-merging Koha/Biblio.pm
CONFLICT (content): Merge conflict in Koha/Biblio.pm
Auto-merging C4/Items.pm
error: Failed to merge in the changes.
Patch failed at 0001 Bug 22690: Refactor merging of records to improve
performance (Elasticsearch)

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging records with many items too slow (Elasticsearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

Ere Maijala <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Patch doesn't apply         |Needs Signoff

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging records with many items too slow (Elasticsearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

Ere Maijala <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #94505|0                           |1
        is obsolete|                            |

--- Comment #13 from Ere Maijala <[hidden email]> ---
Created attachment 97696
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=97696&action=edit
Bug 22690: Refactor merging of records to improve performance (Elasticsearch)

This patch allows merging of records with many items without the web server
timing out.

Test plan:

Without the patch:

- Create 2 records (one with e.g. 1000 items).
- Do a cataloguing search that displays both records, select them and click
"Merge selected".
- Choose the record with many items as the one to be eliminated.
- Start the merging.
- After a while the web server should give you a timeout error (the merging
process may still continue)

With the patch:
- Do the same as above
- This time verify that the records are merged without timeout
- Create a new biblio with an item
- Add with the item:
  * acquisition order
  * hold (reserve)
- Merge the biblio to another one
- Verify that the item and its related data was moved
- Verify that tests pass:
  prove -v t/db_dependent/Koha/Item.t
  prove -v t/db_dependent/Items/MoveItemFromBiblio.t

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging records with many items too slow (Elasticsearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

--- Comment #14 from Ere Maijala <[hidden email]> ---
Rebased.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging records with many items too slow (Elasticsearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

Michal Denar <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Needs Signoff               |Failed QA

--- Comment #15 from Michal Denar <[hidden email]> ---
Hi Ere,
patch working, but one test fails:
Test Summary Report
-------------------
t/db_dependent/Koha/Item.t (Wstat: 256 Tests: 5 Failed: 1)
  Failed test:  3
  Non-zero exit status: 1

Second test was OK.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging records with many items too slow (Elasticsearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

--- Comment #16 from Ere Maijala <[hidden email]> ---
Mike, which test is failing? I'm seeing some errors from an unrelated test that
tries to clean up the database in wrong order (remove patrons before their
checkouts).

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 22690] Merging records with many items too slow (Elasticsearch)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22690

--- Comment #17 from Michal Denar <[hidden email]> ---
Hi Ere,
prove t/db_dependent/Koha/Item.t:

    not ok 3 - Value is mapped correctly for column biblionumber

    #   Failed test 'Value is mapped correctly for column biblionumber'
    #   at t/db_dependent/Koha/Item.t line 109.
    #          got: undef
    #     expected: '462'

    not ok 4 - Value is mapped correctly for column biblioitemnumber

    #   Failed test 'Value is mapped correctly for column biblioitemnumber'
    #   at t/db_dependent/Koha/Item.t line 109.
    #          got: undef
    #     expected: '461'

    not ok 28 - Value is mapped correctly for column timestamp

    #   Failed test 'Value is mapped correctly for column timestamp'
    #   at t/db_dependent/Koha/Item.t line 109.
    #          got: undef
    #     expected: '2020-01-24 22:23:22'

   not ok 42 - Value is mapped correctly for column biblionumber

    #   Failed test 'Value is mapped correctly for column biblionumber'
    #   at t/db_dependent/Koha/Item.t line 124.
    #          got: undef
    #     expected: '462'

  not ok 67 - Value is mapped correctly for column timestamp

    #   Failed test 'Value is mapped correctly for column timestamp'
    #   at t/db_dependent/Koha/Item.t line 124.
    #          got: undef
    #     expected: '2020-01-24 22:23:22'


    # Looks like you failed 6 tests of 79.
not ok 3 - as_marc_field() tests


It's problem with my mappings?

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
123