[Bug 18725] New: Process_message_queue sends duplicate emails if message_queue is not writable.

classic Classic list List threaded Threaded
54 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] New: Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

            Bug ID: 18725
           Summary: Process_message_queue sends duplicate emails if
                    message_queue is not writable.
 Change sponsored?: ---
           Product: Koha
           Version: master
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P5 - low
         Component: Notices
          Assignee: [hidden email]
          Reporter: [hidden email]
        QA Contact: [hidden email]

Last week, we had a database server whose disk filled, causing database writes
to fail.

This meant that messages in message_queue marked 'pending' were not marked as
'sent' when they were added to the postfix mail queue; messages were sent every
15 minutes (as specified in the cron job) until the disk space issues were
cleared.

I would suggest adding a token write to the start of process_message_queue.pl
as a 'canary in the coal mine'. If the database write fails,
process_message_queue should stop, because it's not safe to proceed sending
emails that may not be marked 'sent'.

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Barton Chittenden <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|enhancement                 |normal

--
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Kyle M Hall <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
           Severity|normal                      |major

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Kyle M Hall <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|[hidden email]-commun |[hidden email]
                   |ity.org                     |

--
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Kyle M Hall <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |Needs Signoff

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

--- Comment #1 from Kyle M Hall <[hidden email]> ---
Created attachment 64803
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=64803&action=edit
Bug 18725 - Process_message_queue sends duplicate emails if message_queue is
not writable.

Last week, we had a database server whose disk filled, causing database writes
to fail.

This meant that messages in message_queue marked 'pending' were not marked as
'sent' when they were added to the postfix mail queue; messages were sent every
15 minutes (as specified in the cron job) until the disk space issues were
cleared.

I would suggest adding a token write to the start of process_message_queue.pl
as a 'canary in the coal mine'. If the database write fails,
process_message_queue should stop, because it's not safe to proceed sending
emails that may not be marked 'sent'.

I have no good idea on how to test this patch so this is what I've got:
Test Plan:
1) Apply this patch
2) prove t/db_dependent/Letters.t

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Tomás Cohen Arazi <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Marcel de Rooy <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Katrin Fischer <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Needs Signoff               |Signed Off
                 CC|                            |[hidden email]

--- Comment #2 from Katrin Fischer <[hidden email]> ---
I am not sure if it's the right fix, but everything still seems to work
alright.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Katrin Fischer <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Patch complexity|---                         |Small patch

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Katrin Fischer <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #64803|0                           |1
        is obsolete|                            |

--- Comment #3 from Katrin Fischer <[hidden email]> ---
Created attachment 65625
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=65625&action=edit
[SIGNED OFF] Bug 18725 - Process_message_queue sends duplicate emails if
message_queue is not writable.

Last week, we had a database server whose disk filled, causing database writes
to fail.

This meant that messages in message_queue marked 'pending' were not marked as
'sent' when they were added to the postfix mail queue; messages were sent every
15 minutes (as specified in the cron job) until the disk space issues were
cleared.

I would suggest adding a token write to the start of process_message_queue.pl
as a 'canary in the coal mine'. If the database write fails,
process_message_queue should stop, because it's not safe to proceed sending
emails that may not be marked 'sent'.

I have no good idea on how to test this patch so this is what I've got:
Test Plan:
1) Apply this patch
2) prove t/db_dependent/Letters.t

Signed-off-by: Katrin Fischer <[hidden email]>

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Jonathan Druart <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]-c
                   |                            |ommunity.org

--- Comment #4 from Jonathan Druart <[hidden email]> ---
I am not sure this fix is the best one.

I would suggest to test this kind of issue at the beginning of the cronjob
instead. What do you think?

By the way, why do you try 10 times?

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Marcel de Rooy <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Signed Off                  |Failed QA

--- Comment #5 from Marcel de Rooy <[hidden email]> ---
+    my $lock_sth = $dbh->prepare('LOCK TABLES message_queue WRITE');
+    my $attempts = 0;
+    until ( $sth->execute() ) {
+        $attempts++;
+        die 'Unable to lock message_queue to update message status: ' .
$dbh->errstr() if $attempts > 10;
+        sleep 1;
+    }
+
     my $result = $sth->execute( $params->{'status'},
                                 $params->{'message_id'} );

Hey, you are executing on $sth while you probably meant $lock_sth. You only
prepare the lock, and start fake updating with two undefs.

Overall, this seems not a good idea to me. We had problems with locking tables
already and you should not lock when we run Letters.t since the lock does a
commit. Note that we had a test for prove and an env variable
KOHA_NO_TABLE_LOCKS.

If the disk is full, a lot of other jobs must fail too btw. Doesnt the job
crash on trying the update btw, or did it only send the same one mail each
time? If you cant write to disk, perhaps you could try to save this error
condition to the cache in memory ??
Note that processing mail will probably soon fail too without much disk space..

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Kyle M Hall <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #65625|0                           |1
        is obsolete|                            |

--- Comment #6 from Kyle M Hall <[hidden email]> ---
Created attachment 68602
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=68602&action=edit
Bug 18725 - Process_message_queue sends duplicate emails if message_queue is
not writable.

Last week, we had a database server whose disk filled, causing database writes
to fail.

This meant that messages in message_queue marked 'pending' were not marked as
'sent' when they were added to the postfix mail queue; messages were sent every
15 minutes (as specified in the cron job) until the disk space issues were
cleared.

I would suggest adding a token write to the start of process_message_queue.pl
as a 'canary in the coal mine'. If the database write fails,
process_message_queue should stop, because it's not safe to proceed sending
emails that may not be marked 'sent'.

I have no good idea on how to test this patch so this is what I've got:
Test Plan:
1) Apply this patch
2) prove t/db_dependent/Letters.t

Signed-off-by: Katrin Fischer <[hidden email]>

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Kyle M Hall <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Failed QA                   |Needs Signoff

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Kyle M Hall <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #68602|0                           |1
        is obsolete|                            |

--- Comment #7 from Kyle M Hall <[hidden email]> ---
Created attachment 68603
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=68603&action=edit
Bug 18725 - Process_message_queue sends duplicate emails if message_queue is
not writable.

Last week, we had a database server whose disk filled, causing database writes
to fail.

This meant that messages in message_queue marked 'pending' were not marked as
'sent' when they were added to the postfix mail queue; messages were sent every
15 minutes (as specified in the cron job) until the disk space issues were
cleared.

I would suggest adding a token write to the start of process_message_queue.pl
as a 'canary in the coal mine'. If the database write fails,
process_message_queue should stop, because it's not safe to proceed sending
emails that may not be marked 'sent'.

Test Plan:
1) Apply this patch
2) Make the message_queue table unwriteable somehow
3) Run process_message_queue.pl
4) Script should die with error

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

David Bourgault <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #8 from David Bourgault <[hidden email]> ---
It would be easier if you could provide maybe a hint on how to block access to
the message_queue table? Not sure how to test this otherwise.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

M. Tompsett <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Needs Signoff               |Patch doesn't apply
                 CC|                            |[hidden email]

--- Comment #9 from M. Tompsett <[hidden email]> ---
Bug 18894 broke this easily applying... I'll get around to fixing it later, if
Kyle doesn't beat me to it.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

--- Comment #10 from M. Tompsett <[hidden email]> ---
While the rebase was easy enough to do, I still couldn't figure out how to get
step 2 to happen.

When I locked the table, the canary code paused the script execution until I
unlocked it.

I did try filling the message_queue table. Somewhere around 1.5 million
messages, and it complained the table was full in sql. However, that doesn't
prevent an update. And the number of records was a function of how much space
was left.

I attempted to 'dd' the drive full, and yet I was still able to run the script.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Kyle M Hall <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Patch doesn't apply         |Needs Signoff

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Kyle M Hall <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #68603|0                           |1
        is obsolete|                            |

--- Comment #11 from Kyle M Hall <[hidden email]> ---
Created attachment 70805
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70805&action=edit
Bug 18725 - Process_message_queue sends duplicate emails if message_queue is
not writable.

Last week, we had a database server whose disk filled, causing database writes
to fail.

This meant that messages in message_queue marked 'pending' were not marked as
'sent' when they were added to the postfix mail queue; messages were sent every
15 minutes (as specified in the cron job) until the disk space issues were
cleared.

I would suggest adding a token write to the start of process_message_queue.pl
as a 'canary in the coal mine'. If the database write fails,
process_message_queue should stop, because it's not safe to proceed sending
emails that may not be marked 'sent'.

Test Plan:
1) Apply this patch
2) Make the message_queue table unwriteable somehow
3) Run process_message_queue.pl
4) Script should die with error

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Kyle M Hall <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Needs Signoff               |Signed Off

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Kyle M Hall <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #70805|0                           |1
        is obsolete|                            |

--- Comment #12 from Kyle M Hall <[hidden email]> ---
Created attachment 70826
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70826&action=edit
Bug 18725 - Process_message_queue sends duplicate emails if message_queue is
not writable.

Last week, we had a database server whose disk filled, causing database writes
to fail.

This meant that messages in message_queue marked 'pending' were not marked as
'sent' when they were added to the postfix mail queue; messages were sent every
15 minutes (as specified in the cron job) until the disk space issues were
cleared.

I would suggest adding a token write to the start of process_message_queue.pl
as a 'canary in the coal mine'. If the database write fails,
process_message_queue should stop, because it's not safe to proceed sending
emails that may not be marked 'sent'.

Test Plan:
1) Apply this patch
2) Make the message_queue table unwriteable somehow
3) Run process_message_queue.pl
4) Script should die with error

Signed-off-by: Katrin Fischer <[hidden email]>

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

--- Comment #13 from Kyle M Hall <[hidden email]> ---
(In reply to M. Tompsett from comment #10)

> While the rebase was easy enough to do, I still couldn't figure out how to
> get step 2 to happen.
>
> When I locked the table, the canary code paused the script execution until I
> unlocked it.
>
> I did try filling the message_queue table. Somewhere around 1.5 million
> messages, and it complained the table was full in sql. However, that doesn't
> prevent an update. And the number of records was a function of how much
> space was left.
>
> I attempted to 'dd' the drive full, and yet I was still able to run the
> script.

Hmm. Perhaps make the binary db file readonly?

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Kyle M Hall <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #70826|0                           |1
        is obsolete|                            |

--- Comment #14 from Kyle M Hall <[hidden email]> ---
Created attachment 70853
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70853&action=edit
Bug 18725 - check for writability before sending each message

This will mean that if we fill the disk in the middle of
processing the message queue we will not continue to send messages
even if the database was writable at the time of launch.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Kyle M Hall <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|Signed Off                  |Needs Signoff

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

Kyle M Hall <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #70853|0                           |1
        is obsolete|                            |

--- Comment #15 from Kyle M Hall <[hidden email]> ---
Created attachment 70876
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70876&action=edit
Bug 18725 - Process_message_queue sends duplicate emails if message_queue is
not writable.

Last week, we had a database server whose disk filled, causing database writes
to fail.

This meant that messages in message_queue marked 'pending' were not marked as
'sent' when they were added to the postfix mail queue; messages were sent every
15 minutes (as specified in the cron job) until the disk space issues were
cleared.

I would suggest adding a token write to the start of process_message_queue.pl
as a 'canary in the coal mine'. If the database write fails,
process_message_queue should stop, because it's not safe to proceed sending
emails that may not be marked 'sent'.

Test Plan:
1) Apply this patch
2) Make the message_queue table unwriteable somehow
3) Run process_message_queue.pl
4) Script should die with error

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

--- Comment #16 from M. Tompsett <[hidden email]> ---
Oh, I just thought of one way to try to get it to fail as described: apparmor.
Changing apparmor settings might all creation of records, and then crashing
horribly on update attempts. But I have such an aversion to apparmor, that I am
not even going to try.

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

M. Tompsett <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #70876|0                           |1
        is obsolete|                            |

--- Comment #17 from M. Tompsett <[hidden email]> ---
Created attachment 73277
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=73277&action=edit
Bug 18725 - Process_message_queue sends duplicate emails if message_queue is
not writable.

Last week, we had a database server whose disk filled, causing database writes
to fail.

This meant that messages in message_queue marked 'pending' were not marked as
'sent' when they were added to the postfix mail queue; messages were sent every
15 minutes (as specified in the cron job) until the disk space issues were
cleared.

I would suggest adding a token write to the start of process_message_queue.pl
as a 'canary in the coal mine'. If the database write fails,
process_message_queue should stop, because it's not safe to proceed sending
emails that may not be marked 'sent'.

Test Plan:
1) Apply this patch
2) Make the message_queue table unwriteable somehow
3) Run process_message_queue.pl
4) Script should die with error

Signed-off-by: Mark Tompsett <[hidden email]>

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

[Bug 18725] Process_message_queue sends duplicate emails if message_queue is not writable.

bugzilla-daemon
In reply to this post by bugzilla-daemon
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18725

--- Comment #18 from M. Tompsett <[hidden email]> ---
Created attachment 73278
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=73278&action=edit
Bug 18725: Tests to trigger canary case

Since replicating the conditions necessary for the canary case
proved difficult, tests were written to simulate it.

prove t/db_dependent/Letters.t

Signed-off-by: Mark Tompsett <[hidden email]>

--
You are receiving this mail because:
You are watching all bug changes.
_______________________________________________
Koha-bugs mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
12