Generic Proposal Problems
Themes Commonly Found
in Less-than-Enthusiastic Reviews
No testable hypothesis
or clearly defined questions
Some pose "hypotheses,"
but they cannot be falsified (tested) with data proposed.
Many pose questions, but questions are considered generic
or vague. Those which fail to pose either questions
or hypotheses usually state "goals" or "objectives"
which are considered fuzzy, as in "to determine
the relationship of," "to quantify the processes
data collection over problem solving
Proposal gives impression of "first we'll generate
the data, then we'll figure out what it means."
No evidence of strategic thinking: first posing question,
then identifying best data to answer it, thinking through
how the data will be interpreted and what the results
might mean. Frequently comes across as using material
that is most easily available, or cruise that is happening
anyway. Proposal to use large variety of analyses, failing
to explain how all the different types of data will
fit together. Might reviewers use words like "fishing
trip" or "shotgun" to describe this proposal?
seen as secondary, not worth doing
Frequently the project is seen as a local study without
broader relevance to other regions, or the question
has been addressed in other places and nothing new will
emerge from one more study. Sometimes the locale is
said to be so unique that it provides no insight into
typical cases. Other times even the broader question
is said to be "not exciting," "not firstorder."
Wrong tools for
Problem posed is fine, but techniques or strategy inappropriate.
Precision is often an issue. Or there are too many ambiguities
in the tools being used; too many assumptions required
for strategy to work. Solution will be nonunique.
Data to be generated is not tightly coupled to problem
Variety of reasons: proposal plans to develop new technique
and then apply it (implying no problems will occur during
development); ditto with using technique only very recently
developed; firstorder work on recent material
not yet completed, too soon to propose secondorder
(frequently seen for post-cruise proposals); trying
to answer too many different questions (don't promise
more than you can deliver); preliminary data not sufficiently
convincing to begin major new activity. Usual recommendation
is to scale way back, do pilot study first.
PI appears illinformed
Often case of person changing research directions or
trying to apply skills from one field to another. Failure
to cite relevant papers implies ignorance of work by
others; misstates or ignores concepts/facts generally
known; misinterprets preliminary data presented in proposal;
reasoning doesn't make sense, appears to be based on
Existing data disproves or already answers question
being posed; error in logic (circular reasoning)
Problems with modeling
Proposals with primary focus on modeling tend to get
same types of comments, regardless of discipline: realworld
boundary conditions too poorly known to constrain or
apply model; no attempt to compare model results to
realworld data (model not to be verified); too
many assumptions required/underdetermined; simplifications
needed for computational efficiency make model too general
to be of use; scale problem (mostly for lab models:
can't assume linear extrapolation to realworld
orders of magnitude); no apparent use/need for model
perceived by databased colleagues; model can reproduce
reality but so what? (does not give insight into processes
and relationships, "just tweaking the knobs Ill
you get a fit").
Last updated: May 4, 2011