Of course we cannot always share details about our work with customers, but nevertheless it is nice to show our technical achievements and share some of our implemented solutions.
Atlassian's Confluence application has a couple of neat features, including collaborative editing and a REST API to do (automated) updates to a page.
The latter feature, REST API, has been successfully used for almost two years to automatically update Confluence pages based on SLA reports generated by Thruk. See the article automatic SLA reporting from Icinga and push into Confluence page for more information.
As the SLA calculations happen on the first of the month for the prior month, they run once a month. Around the end of 2019 we noticed that all of a sudden most of the pages were not updated anymore. PDF reports were successfully uploaded as attachments to the relevant Confluence pages, but page updates (content update) did not work anymore. At first a temporary issue in the report script was suspected but when the same issue happened again one month later, it got our heads scratching for real.
Debugging the output created from the cron job could maybe show why the updates suddenly fail. Instead of using /dev/null as script output, a temporary log file should be created.
18 09 6 * * /opt/scripts/thruk-sla-reporting.sh -T "Atlassian Confluence" -U "wiki.example.com" -W 87707968 -P "SLA report Confluence" -R 13 > /tmp/13.log
And indeed, once all the automated reports were fired up again, most of the temporary log files contained the following error:
root@thruk:~# cat /tmp/13.log
{
"message": "Unable to save changes to unreconciled page ContentId{id=87707968}. Refreshing the page should fix this.",
"reason": "Not Implemented",
"statusCode": 501
}
Researching this error led to a discussion in Atlassian's community forum: Some synchrony pages are unreconciled. Synchrony, of course, is the backend for the collaborative editing feature. Following this discussion led to an official Confluence bug CONFSERVER-59196 which describes this error message in detail.
The bug seems to only affect Confluence 7.x versions and in 7.0 there are two workarounds:
As the previous report 13 failed, I did exactly this and created a new cronjob entry to run the same job again. And, surprise surprise!, this time it worked.
On Confluence 7.1 there is no workaround available.
The bug will be fixed in Confluence 7.4 (not GA yet) according to the bug ticket.
Now that the cause of the statistics update is known (upgrade to Confluence 7.x) the exact moment can be pinpointed. We upgraded Confluence on October 30, 2019. And this matches perfectly with our perception that some of the the reports failed at the end of 2019.