[Bug 18948] New: Elasticsearch - Reindexes should use aliases to avoid down time while reindexing

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Bug 18948] New: Elasticsearch - Reindexes should use aliases to avoid down time while reindexing

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18948

            Bug ID: 18948
           Summary: Elasticsearch - Reindexes should use aliases to avoid
                    down time while reindexing
 Change sponsored?: ---
           Product: Koha
           Version: master
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P5 - low
         Component: Architecture, internals, and plumbing
          Assignee: [hidden email]
          Reporter: [hidden email]
        QA Contact: [hidden email]

This patchset aims to add the ability to leverage the ES alias feature during a
reindex - when doing a full reindex or a deletion the indexer will look for an
existing index under the alias - if not found it will create a new one, if
found it will index into a new index and point the alias there when done and
then delete the old index.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Bug 18948] Elasticsearch - Reindexes should use aliases to avoid down time while reindexing

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18948

--- Comment #1 from Nick Clemens <[hidden email]> ---
Created attachment 65086
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65086&action=edit
Bug 18948 - Elasticsearch - Reindexes should use aliases to avoid down time
while reindexing

To test:
1  - Delete your current indices for ES (if any)
2  - Reindex your records
3  - curl -XGET 'localhost:9200/koha_kohadev?pretty'
4  - Note indices are now of form koha_kohadev_##########
5  - The numbers there are a result of appending 'time' command to name
6  - Ensure searching works as before the patches
7  - Reindex your records with a commit setting of 1 (to slow things
        down) you may need a large numebr fo records
8  - Ensure searching works during reindex
9  - Ensure reindexing completes successfully
10 - curl -XGET 'localhost:9200/koha_kohadev?pretty'
11 - There should only be one index each for biblios and authorities
** - Note number in above
12 - Perform a partial reindex by passing a biblionumber to the command
13 - Curl the indices again and ensure number/name has not changed

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Bug 18948] Elasticsearch - Reindexes should use aliases to avoid down time while reindexing

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18948

Nick Clemens <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]-c
                   |                            |ommunity.org,
                   |                            |[hidden email],
                   |                            |[hidden email],
                   |                            |[hidden email]

--- Comment #2 from Nick Clemens <[hidden email]> ---
TODO:
1 - Ensure if records are indexed incrementally before first rebuild that an
alias is used
2 - Unit tests

Other comments welcome, this was first round proof of concept

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Bug 18948] Elasticsearch - Reindexes should use aliases to avoid down time while reindexing

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18948

--- Comment #3 from Nick Clemens <[hidden email]> ---
Created attachment 65147
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65147&action=edit
Bug 18948 - Elasticsearch - Reindexes should use aliases to avoid down time
while reindexing

Refactoring of rebuild_elastic_search.pl to move code into
Koha::SearchEngine::Elasticsearch::Indexer

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Bug 18948] Elasticsearch - Reindexes should use aliases to avoid down time while reindexing

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18948

Nick Clemens <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |Needs Signoff

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Bug 18948] Elasticsearch - Reindexes should use aliases to avoid down time while reindexing

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18948

Nick Clemens <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #65086|0                           |1
        is obsolete|                            |

--- Comment #4 from Nick Clemens <[hidden email]> ---
Created attachment 65159
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65159&action=edit
Bug 18948 - Elasticsearch - Reindexes should use aliases to avoid down time
while reindexing

To test:
1  - Delete your current indices for ES (if any)
2  - Reindex your records
3  - curl 'localhost:9200/_cat/indices?v'
4  - Note indices are now of form koha_kohadev_##########
5  - The numbers there are a result of appending 'time' command to name
6  - Ensure searching works as before the patches
7  - Reindex your records with a commit setting of 1 (to slow things
        down) you may need a large numebr fo records
8  - Ensure searching works during reindex
9  - Ensure reindexing completes successfully
10 - curl 'localhost:9200/_cat/indices?v'
11 - There should only be one index each for biblios and authorities
** - Note number in above
12 - Perform a partial reindex by passing a biblionumber to the command
13 - Curl the indices again and ensure number/name has not changed
14 - Delete all indexes
    curl -XDELETE 'localhost:9200/koha_kohadev_*?pretty'
15 - Edit a known biblio:
    http://localhost:8081/cgi-bin/koha/cataloguing/addbiblio.pl?biblionumber=1
16 - Add a word to the title "potato"
17 - Save the record
18 - Search for potato
19 - You should arrive at the detail page for that record
20 - Confirm your index is of form koha_kohadev_##########

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Bug 18948] Elasticsearch - Reindexes should use aliases to avoid down time while reindexing

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18948

--- Comment #5 from Nick Clemens <[hidden email]> ---
Needs unit tests

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Bug 18948] Elasticsearch - Reindexes should use aliases to avoid down time while reindexing

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18948

Nick Clemens <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Blocks|                            |18950


Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18950
[Bug 18950] Elasticsearch - Add password access for admin functions
--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Loading...