Improving the WordPress Theme Review Experiment

A disclaimer
I would have posted this on WP’s official site if it allowed me to post, but since it doesn’t, I am publishing it on my own blog. This is meant as an appraisal of the new theme approval process on WordPress.

On 9th June 2010 WordPress made an announcement that it was expanding the theme review process. There was a call for volunteers to support theme reviews. This was indeed good news, particularly if you were a theme developer. For a long time Joseph Scott had been the sole approver of themes on WP, and though his work was impeccable and his reviews very good and accurate, if he was traveling or unwell or on holiday theme approvals tended to get delayed. This was by no means a deal-breaker, but having more people looking at themes definitely meant that things would get better.

Soon after the announcement a few things happened that were moves in the right direction:

1. Quite a few folks signed up in response to Joseph’s post, so there are more volunteers now.
2. A dedicated Trac project was setup for approvals.
3. Every time a theme got submitted the corresponding Trac ticket got emailed to the theme author so that he/she could follow what was going on with his theme.
4. There was much better focus on the quality of code in a theme. Actually the focus was more on writing code that didn’t generate “silent errors” or PHP NOTICE messages, but nonetheless adhering to this is something that keeps down the size of logs on your server.

But that is where the goodness stopped. One of the key things which I had hoped would be addressed, speed of approvals took a nosedive. To cite an example earlier when I submitted a theme version it would take around 1-3 days to get approved. It took 1 day if I submitted it on a weekday and typically 3 days if I submitted it on a Friday. In some cases if it was vacation time then the versions took about a week to 10 days to get approved, but this was more of an exception to the rule. Normally on a day you would see 2-3 versions of different themes getting approved, and on occasion the number of approvals even ran into double digits.

However, with the new experiment somewhere something went wrong. Now you have around 1 theme being approved in 2-3 days, which doesn’t seem right. To drive my point home, version 3.5.7 of Suffusion got approved on 10th July. Today is 28th July and only nine more versions of different themes have been approved (including a version of the WP default theme Twenty Ten). This is a far cry from how things happened earlier. I normally used to wait for about a week before pinging the folks to check the status of a version, so for 3.5.4 I sent out a message asking what was up, and this is what I received:

Thanks for the note.  Unfortunately there is currently a backlog of themes in the review queue.  We are working on it.  Themes are being reviewed in the order in which they were received.  Thank you for your continued patience.

I didn’t know at that time how much the backlog was or how much longer the next version of Suffusion would take for approval. I found out later how you can estimate the potential wait time. The backlog, as I know now, is well over 100 theme versions at present. So somehow adding more people slowed down the process.

The delay also brings in a bunch of associated problems. Currently if you have a theme version in the queue and you submit a new version, the older version gets deleted and the newer version goes to the back of the queue. So as a theme developer:

1. If I have submitted a bug-fix for the theme 3 weeks back and that hasn’t been approved, how do I ensure that my users get the bug-fix? Distribute it from my site (unapproved)? Doesn’t sound safe.
2. If I have submitted a version 3 weeks back with some bug-fixes, then I discover some more bugs in the submitted version what do I do with them? If I resubmit a new version I go to the bottom. So I add another month or so to something that has already been in for 3 weeks? Ouch. Incidentally I have had this happen to me quite often, most recently with version 3.5.8 and 3.5.9.
3. If I choose not to submit the bug-fix for the older version but let the bugs be approved (these are niche bugs that cannot be caught in the approval process, mind you) and then I submit the new version, I am risking users staying with a buggy version for an indeterminate number of extra weeks.

So somehow this review process discourages active theme development, which I feel is counter-productive to the whole experiment.

How can the experiment be improved? I have a few suggestions:

1. Improve targets. Set a target saying that each approver should aim to disposition a new theme for approval every weekday (or some such figure). If you have 10 approvers, this will take care of tackling 10 themes everyday and bring down the backlog much faster.
   I can almost hear people say, “What does he know?”, or “He is speaking like a consultant”, or “He is oversimplifying”. Well, I have been doing free support for Suffusion since late last September and that is how I keep my backlog down. Every morning I go through all outstanding support requests, take a crack at answering them if they haven’t been answered by someone else, then carry on with my regular job for the rest of the day. If a request comes in during the day, if I am not doing much then I respond to it (or if I think others can respond, I skip it). This has been a largely successful way of staying on top of things. I have responded to more than 3000 posts on my support forum at around 10 posts per day.
2. Don’t de-prioritize existing themes in the queue upon newer submissions. I love putting out new functionality in the theme, and anyone with any experience whatsoever with Suffusion will agree with me in the fact what I provide is not just cosmetic, but hard features. Sometimes a new version is required to patch a bug. Getting pushed to the back of the queue is very off-putting, because it simply adds so much more in terms of wait time for approvals.
   This simply encourages parallel avenues of theme distribution because developers know that their work will not be approved soon on WP’s official site, so they distribute it from their site. And that is where they can start putting in stuff like tracking code etc into a theme. Mind you, quicker approvals will make this a non-issue, but the current wait of a few weeks is a bit too much.

As a concluding remark, the topic of this post and the issues in the post are not mission-critical and I am sure people have more interesting things to do. However I am sure that the free theme developers would be quite happy with these modifications, particularly since some of them invest a lot of energy in building their themes.