On the twelfth day after christmas, my true love said to me, “This wordpress theme won't let me save any customisations. Can you take a look at it?”
The theme customisation menu in WordPress displays various options in the left sidebar, and a live preview of the changes on the right. You can edit things in the menu and see what they look like, and there's a "Save and Publish" button at the top. But the button remained stuck at "Saved" (greyed-out), and never detected any changes. Nor was the menu properly styled, and many other controls were inoperative.
We found other reports of the problem, but no definitive solution. Disabling certain plugins fixed the problem for some people, but that didn't help us—hardly any plugins were active anyway, and none of the problematic ones were installed.
We looked at network requests for the page in the Chrome developer console, and saw a series of 404 responses for local CSS and JS resources within the Minamaze theme. Here's one of the failing URLs:
That /var/lib/wordpress certainly didn't belong in the URL, so we went grepping in the code to see how the URL was being generated. It took us quite some time to figure it out, but we eventually found this code that was used to convert a filesystem path to the static resources into a URL (slightly edited here for clarity):
site_url(str_replace(ABSPATH, '/', $_extension_dir))
(Summary: Take /a/b/c ($_extension_dir), replace /a/b/ (ABSPATH) with a single /, and use the resulting /c as a relative URL.)
ABSPATH was set to /usr/share/wordpress/, but the extension dir was under /var/lib/wordpress/, so it's no surprise that stripping ABSPATH from it didn't result in a valid URL. Not that doing search-and-replace on filesystem paths is the most robust way to do URL generation in the first place, but at least we could see why it was failing.
The Debian package of WordPress is… clever. It places the code under /usr/share/wordpress (owned by root, not writable by www-data), but overlays /var/lib/wordpress/wp-content for variable data.
Alias /wp /usr/share/wordpress Alias /wp/wp-content /var/lib/wordpress/wp-content
This is a fine scheme in principle, but it is unfortunately at odds with WordPress standard practice, and the Debian README mentions that liberal applications of chown www-data may be required to soothe the itch.
Unfortunately, it also means that themes may not be installed under ABSPATH, which usually doesn't matter… until some theme code makes lazy and conflicting assumptions.
The eventual solution was to ditch /usr/share/wordpress and use only /var/lib/wordpress for everything. Then ABSPATH was set correctly, and the URL generation worked. (We tried to override the definition of ABSPATH in wp-config.php, but it's a constant apparently set by the PHP interpreter.)
In the end, however, I couldn't quite make up my mind whether to blame the Debian maintainers of Wordpress for introducing this overlay scheme, or the theme developers for generating URLs by doing string manipulation on filesystem paths, or the Wordpress developers for leaving static file inclusion up to theme developers in the first place.
Well, why not all three?