[Bug 25292] New: L1 cache too long in Z3950 server (z3950-responder)

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 25292] New: L1 cache too long in Z3950 server (z3950-responder)

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25292

            Bug ID: 25292
           Summary: L1 cache too long in Z3950 server (z3950-responder)
 Change sponsored?: ---
           Product: Koha
           Version: unspecified
          Hardware: All
                OS: All
            Status: ASSIGNED
          Severity: normal
          Priority: P5 - low
         Component: Architecture, internals, and plumbing
          Assignee: [hidden email]
          Reporter: [hidden email]
        QA Contact: [hidden email]

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 25292] L1 cache too long in Z3950 server (z3950-responder)

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25292

Jonathan Druart <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Blocks|                            |24761


Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24761
[Bug 24761] Caching issues in scripts running in daemon mode [OMNIBUS]
--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 25292] L1 cache too long in Z3950 server (z3950-responder)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25292

Jonathan Druart <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |Needs Signoff

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 25292] L1 cache too long in Z3950 server (z3950-responder)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25292

--- Comment #1 from Jonathan Druart <[hidden email]> ---
Created attachment 103795
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=103795&action=edit
Bug 25292: Flushing L1 on every Z3950 server search

This patch flushes the L1 cache on each Z3950 server search.

There is no need to flush it in init_handler (when the connection is
made), the L1 cache is not accessed (this behaviour goes against what I
expected).

This means each search will fetch values from the
L2 cache (ie memcached) and persist the L1 cache (in Perl
memory cache) until the result are returned (or longer?).

Test plan:
% perl misc/z3950_responder.pl --config-dir=/kohadevbox/koha/etc/z3950
% zoomsh
ZOOM>connect localhost:2100
ZOOM>set databaseName biblios
ZOOM>search x
ZOOM>search x

Between the 2 searches you can modify the value of SearchEngine, you
will notice that the number of hits is different

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 25292] L1 cache too long in Z3950 server (z3950-responder)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25292

Jonathan Druart <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
         Depends on|                            |13937


Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13937
[Bug 13937] Add an Elasticsearch-compatible Z39.50/SRU daemon
--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 25292] L1 cache too long in Z3950 server (z3950-responder)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25292

Frédéric Demians <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #2 from Frédéric Demians <[hidden email]> ---
I can reproduce the test plan on a dual installation (ES/Zebra), and see the
result variation depending on selected search engine. So it works. But won't it
impact performance?

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 25292] L1 cache too long in Z3950 server (z3950-responder)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25292

--- Comment #3 from Jonathan Druart <[hidden email]> ---
Of course there will be a difference, but you will hit L2 (memcached) anyway.
It won't be worst than other search (UI).

I am not expecting a huge difference here but I have not compared.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 25292] L1 cache too long in Z3950 server (z3950-responder)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25292

Victor Grousset/tuxayo <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #4 from Victor Grousset/tuxayo <[hidden email]> ---
(In reply to Jonathan Druart from comment #1)
> ZOOM>search x
> ZOOM>search x
>
> Between the 2 searches you can modify the value of SearchEngine, you
> will notice that the number of hits is different

I don't get what should I do to see a difference in the number of hits.

I've searched for few things before the patch.
And after the patch (after restarting the responder). Same number of hits.
Then I did restart_all.

In this session, I switched at some points from ES to Zebra.
I'm confused and can't invent a list of steps that leads to something
consistently good (after the patch) or bad (before the patch).


kohadev-koha@6c8a568fb951:/kohadevbox/koha$ zoomsh
ZOOM>connect localhost:2100
ZOOM>set databaseName biblios
ZOOM>search shock
localhost:2100: 2 hits
ZOOM>search the
localhost:2100: 192 hits
ZOOM>search the
localhost:2100: 192 hits
ZOOM>search the
localhost:2100: 192 hits
# I think I switched from
ZOOM>search the
localhost:2100: 0 hits
ZOOM>search the
localhost:2100: 0 hits
ZOOM>search the
localhost:2100: 0 hits
ZOOM>search les
localhost:2100: 0 hits
ZOOM>search yes
localhost:2100: 0 hits
ZOOM>search shoch
localhost:2100: 0 hits
ZOOM>search shodck
localhost:2100: 0 hits
ZOOM>search shock
localhost:2100: 0 hits
ZOOM>search the
localhost:2100: 0 hits


second try with more details.

kohadev-koha@6c8a568fb951:/kohadevbox/koha$ zoomsh
ZOOM>connect localhost:2100
ZOOM>set databaseName biblios
ZOOM>search shock
localhost:2100: 2 hits
ZOOM>search the
localhost:2100: 192 hits
ZOOM>
kohadev-koha@6c8a568fb951:/kohadevbox/koha$ # switch toZebra
kohadev-koha@6c8a568fb951:/kohadevbox/koha$ zoomsh                              
ZOOM>connect localhost:2100
ZOOM>set databaseName biblios
ZOOM>search shock
localhost:2100 error: Temporary system error (Bib-1:2) Cannot connect to
upstream server                                                      
ZOOM>search the
localhost:2100 error: Temporary system error (Bib-1:2) Cannot connect to
upstream server                                                      
ZOOM>
kohadev-koha@6c8a568fb951:/kohadevbox/koha$ zoomsh
ZOOM>
kohadev-koha@6c8a568fb951:/kohadevbox/koha$ #restart responder
kohadev-koha@6c8a568fb951:/kohadevbox/koha$ zoomsh
ZOOM>connect localhost:2100
ZOOM>set databaseName biblios
ZOOM>search shock
localhost:2100 error: Temporary system error (Bib-1:2) Cannot connect to
upstream server
ZOOM>search the
localhost:2100 error: Temporary system error (Bib-1:2) Cannot connect to
upstream server
ZOOM>

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 25292] L1 cache too long in Z3950 server (z3950-responder)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25292

--- Comment #5 from Victor Grousset/tuxayo <[hidden email]> ---
The last two logs are with the patch applied.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 25292] L1 cache too long in Z3950 server (z3950-responder)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25292

--- Comment #6 from Jonathan Druart <[hidden email]> ---
This is all what I do:

ZOOM>connect localhost:2100
ZOOM>set databaseName biblios
ZOOM>search x
localhost:2100: 18 hits

# Change the syspref SearchEngine

ZOOM>search x
localhost:2100: 230 hits

This is a ugly way of confirming that the correct value has been retrieve, but
you could add a warn statement in the code, in search_handler, to check which
value from cache (so easy to test is a syspref) is retrieved.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 25292] L1 cache too long in Z3950 server (z3950-responder)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25292

--- Comment #7 from Victor Grousset/tuxayo <[hidden email]> ---
After changing the syspref, I must restart ZOOM to have a different number of
hits.

Is that expected?

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 25292] L1 cache too long in Z3950 server (z3950-responder)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25292

--- Comment #8 from Jonathan Druart <[hidden email]> ---
(In reply to Victor Grousset/tuxayo from comment #7)
> After changing the syspref, I must restart ZOOM to have a different number
> of hits.
>
> Is that expected?

No, please detail what you tried and the result.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 25292] L1 cache too long in Z3950 server (z3950-responder)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25292

--- Comment #9 from Jonathan Druart <[hidden email]> ---
(In reply to Jonathan Druart from comment #8)
> (In reply to Victor Grousset/tuxayo from comment #7)
> > After changing the syspref, I must restart ZOOM to have a different number
> > of hits.
> >
> > Is that expected?
>
> No, please detail what you tried and the result.

Hum forget that, you detailed above.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 25292] L1 cache too long in Z3950 server (z3950-responder)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25292

Jonathan Druart <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #103795|0                           |1
        is obsolete|                            |

--- Comment #10 from Jonathan Druart <[hidden email]> ---
Created attachment 107621
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=107621&action=edit
Bug 25292: Flushing L1 on every Z3950 server search

This patch flushes the L1 cache on each Z3950 server search.

There is no need to flush it in init_handler (when the connection is
made), the L1 cache is not accessed (this behaviour goes against what I
expected).

This means each search will fetch values from the
L2 cache (ie memcached) and persist the L1 cache (in Perl
memory cache) until the result are returned (or longer?).

Test plan:
% perl misc/z3950_responder.pl --config-dir=/kohadevbox/koha/etc/z3950
% zoomsh
ZOOM>connect localhost:2100
ZOOM>set databaseName biblios
ZOOM>search x
ZOOM>search x

Between the 2 searches you can modify the value of SearchEngine, you
will notice that the number of hits is different

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 25292] L1 cache too long in Z3950 server (z3950-responder)

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25292

--- Comment #11 from Jonathan Druart <[hidden email]> ---
I am really confused, I am pretty sure the original patch was working when I
wrote it but... reading the code it couldn't have worked.

This new patch should work accordingly to what is advertised in the commit
message.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/