Difference between revisions of "2018/06/22"
Line 8: | Line 8: | ||
...and rebooted (which wasn't strictly necessary, but the system was asking for one because of a kernel upgrade some days ago). | ...and rebooted (which wasn't strictly necessary, but the system was asking for one because of a kernel upgrade some days ago). | ||
+ | |||
+ | ...and then it turned out I had forgotten to save my changes before rebooting, so I had to do this: | ||
+ | <pre> | ||
+ | root@tootcat2:~# systemctl restart mastodon-sidekiq.service | ||
+ | Warning: mastodon-sidekiq.service changed on disk. Run 'systemctl daemon-reload' to reload units. | ||
+ | sroot@tootcat2:~# systemctl daemon-reload | ||
+ | </pre> | ||
+ | |||
+ | After finding [https://discourse.joinmastodon.org/t/lots-of-activerecord-connectiontimeouterror/994/3 this explanation], I understood two things: | ||
+ | * <code>DB_POOL</code> should be the same as the <code>sidekiq -c</code> value | ||
+ | * I actually have plenty of CPU headroom, and can boost this quite a bit more. | ||
+ | |||
+ | I first tried 40, and that helped a bit -- but the "busy" queue was still getting stuck around 3500. | ||
+ | |||
+ | So then I bumped it up to 100, and within a minute or two the queue was down under 100. |
Revision as of 00:37, 23 June 2018
This isn't the first time this has happened... outgoing toots are being delayed by something like 1.5-2 hours. Last time, we upped the number of threads from 15 to 25 (according to my memory; apparently we didn't document it), and that seemed to fix the blockage.
So this time, I'm making notes.
Editing /etc/systemd/system/mastodon-sidekiq.service
:
- changing
DB_POOL=5
toDB_POOL=20
- changing
sidekiq -c 25
tosidekiq -c 40
...and rebooted (which wasn't strictly necessary, but the system was asking for one because of a kernel upgrade some days ago).
...and then it turned out I had forgotten to save my changes before rebooting, so I had to do this:
root@tootcat2:~# systemctl restart mastodon-sidekiq.service Warning: mastodon-sidekiq.service changed on disk. Run 'systemctl daemon-reload' to reload units. sroot@tootcat2:~# systemctl daemon-reload
After finding this explanation, I understood two things:
DB_POOL
should be the same as thesidekiq -c
value- I actually have plenty of CPU headroom, and can boost this quite a bit more.
I first tried 40, and that helped a bit -- but the "busy" queue was still getting stuck around 3500.
So then I bumped it up to 100, and within a minute or two the queue was down under 100.