so clearly it has been decided that the most important thing is not that a number is accurate but that an experienced person can see HOW accurate it is.
That's a wise vision. The value of a database lies as much in its size as in the accuracy of its data: the ideal situation would be a very large database with very accurate data; sticking too much to accuracy would in reality lead to a very small database -- the practical compromise is a sufficiently large database with sufficiently accurate data.
I see no good reasons why the standards for TCOD should be more stringent than they are for COD ;-).
I think that we should calibrate our mindsets to this "intermediately liberal" attitude and would prefer a rather liberal attitude when it comes to the pure convergence parameters as long as the most important ones are given. That would be (in addition to which code was used...) a description of the basis set (e.g. something like "400 eV" for a plane wave pseudopotential code or perhaps "tier 1" if you use FHI-AIMS), k-point set and the force/cell convergence criterion. Then we could have a rough classification resulting in a little green, yellow or red label for well, intermediately and poorly converged calculations.
I support this. As the number of codes out there is still countable, it would be doable to make such a decision table for every individual code.
A major problem is that the choice of functional may not be as easily tagged as good or bad.
That's not necessarily a problem. Unlike the numerical parameters that you listed above, the functional is an uncontrolled approximation. The former can all be tested for convergence, the functional cannot. What matters most is to be reassured that a particular calculation is numerically converged. Such a database would then be useful to check against COD, in order to find out the performance of a particular functional over a wide class of materials (some work along this lines has been done by MaterialsProject.org, where materials for which GGA was overbinding could be flagged as interesting for further study: either the experiments were not good, or there was special physics at work).
P.S. OK, so maybe someone has to go first and get complained at by everoyne else for his sloppy standards... Here a suggestion for marking up the two most important convergence parameters: Max residual force on atoms < 0.01 eV/Å = good, 0.1-0.01 = intermediate,
0.1 = low K-point resolution < 0.2 Å^{-1} = good, 0.75-0.2 =
intermediate, > 0.75 = low
In order to judge the k-mesh, one would preferably have also the information whether the material is a metal or not.
Stefaan