Difference between revisions of "CleanErrorLogs"

We need to monitor our error logs from time to time and clean out the errors. Now seems especially important because our users have been getting the owls so much. We're organizing the work here: http://www.aboutus.org/CleanErrorLogs

The Logs

Our two most important logs may be followed on the squals using the following:

tail -f /var/log/httpd/aboutus.org-error_log

tail -f /www/aboutus/compostus/compost/log/production.log

http://www.aboutus.org/au_web_services/system_information/log_counts is also a very helpful analysis tool.

Inconsistent DB

Couldn't find Wikitext with ID=15624778 http://www3.aboutus.org/au_web_services/system_information/log?start=1427149336 Couldn't find Wikitext with ID=15624987 http://www3.aboutus.org/au_web_services/system_information/log?start=1427251164

Full table scan of revision, find the one with the missing wikitext ... what are we going to do when we find it?

PageCache Race

ActiveRecord::StatementInvalid (Mysql::Error: Duplicate entry '1278324' for key 2: INSERT INTO `page_caches`

Catch this error and reload the page cache ... perhaps try the load after the create fails and only fail if the load after fails

Duplicate User gives Owls

Account Created Race

http://www3.aboutus.org/au_web_services/system_information/log?start=1428851935

JSON Errors associated with action catchup

utf8 issue?

ignore for now

http://www3.aboutus.org/au_web_services/system_information/log?start=1428214801

Duplicate Anonymous Users

Clean up at some point in time

inconsistent AuPageLookup (no page)

http://www3.aboutus.org/au_web_services/system_information/log?start=1428651435

go through and remove all AuPageLookup that don't have pages

create logger message to tell us which pages

No revision for page

http://www3.aboutus.org/au_web_services/system_information/log?start=1428696571

Does and not short-circuit like && does?

Feeds don't work

http://www3.aboutus.org/au_web_services/system_information/log?start=1428680724

Connection with MW produces unexpected results

http://www3.aboutus.org/au_web_services/system_information/log?start=1427762953