Checking if Plugins Pass Basic “Good Programming” Test

On http://wordpress.org/ideas/topic/plugins-and-theme-repository-thorough-bug-update-and-security-review/ there is discussion about adding to WordPress a “system of testing and making sure that plugins and themes work, are up to date (no plugins over a year old, say) and are bulletproof secure. The plugin and theme repository should demand that every single plugin and theme should be rigorously reviewed and screened for security.”

My suggestions for a Partial solution, that could Easily be checked for by the Plugin repository and by the WordPress Core:

1) /wp-admin/network/plugins.php shows if are using the latest version of each plugin. But it doesn’t indicate the Date of the latest version, or if the latest version date is a long time ago. Adding the date would be extremely easy.

2) On plugins.php display the version of WordPress the plugin was written for, just like the plugin repository page does. Display the user rating from the repository, for your installed version of WordPress with your installed version of the plugin.

3) Plugin repository could list results of installing the plugin in a default configuration, with full error reporting on, testing for a) syntax errors, b) obsolete function calls, c) database errors, d) direct MySQL calls (e.g. bypassing WordPress security and roles), e) multi-site initialization errors (e.g. are tables created for the plugin when create a site? Does the plugin put data in wp_options or in wp_SITEID_options? ).

The test site would be restored to default state after each plugin test. The test would be run at each plugin update posted to the repository. This sounds like something that could be easily automated.

Not a guaranteed security or quality test, but a good check of the basics.

@Ipstenu replied “We have yet to manage to do it in a way that didn’t net a bajillion false positives that we have to look at them all manually anyway :/ ”

Nope. Purely objective reports.

Have PHP error logging set to maximum, use software to count each category of error, report the findings. Already includes obsolete WordPress functions, obsolete PHP functions, syntax errors, mismatched variable types, uninitialized variables, and more.

Look for occurrences of the most common MySQL function calls (instead of WordPress database functions), report the number.

Get a list of tables after adding a new site (single MySQL query). Software subtract the list of tables started with. Report the difference (or maybe a little more secure, the # tables).

Another test: Any plugin using wp_ instead of the actual table prefix, isn’t programmed well.

Similarly, # rows in [tableprefix]_options before and after (shouldn’t change, plugins should use [tableprefix]_[siteid]_options).

Show the whole list in a new tab in the Repository, for each version, and watch plugin authors clean up their act!