Generic Proposal Problems   Email     Print   PDF

Themes Commonly Found in Less-than-Enthusiastic Reviews

No testable hypothesis or clearly ­defined questions
Some pose "hypotheses," but they cannot be falsified (tested) with data proposed. Many pose questions, but questions are considered generic or vague. Those which fail to pose either questions or hypotheses usually state "goals" or "objectives" which are considered fuzzy, as in "to determine the relationship of," "to quantify the processes affecting."

Proposal emphasizes data collection over problem solving
Proposal gives impression of "first we'll generate the data, then we'll figure out what it means." No evidence of strategic thinking: first posing question, then identifying best data to answer it, thinking through how the data will be interpreted and what the results might mean. Frequently comes across as using material that is most easily available, or cruise that is happening anyway. Proposal to use large variety of analyses, failing to explain how all the different types of data will fit together. Might reviewers use words like "fishing trip" or "shotgun" to describe this proposal?

Problem/question/hypothesis seen as secondary, not worth doing
Frequently the project is seen as a local study without broader relevance to other regions, or the question has been addressed in other places and nothing new will emerge from one more study. Sometimes the locale is said to be so unique that it provides no insight into typical cases. Other times even the broader question is said to be "not exciting," "not first­order."

Wrong tools for the job
Problem posed is fine, but techniques or strategy inappropriate. Precision is often an issue. Or there are too many ambiguities in the tools being used; too many assumptions required for strategy to work. Solution will be non­unique. Data to be generated is not tightly coupled to problem posed.

Too ambitious, premature
Variety of reasons: proposal plans to develop new technique and then apply it (implying no problems will occur during development); ditto with using technique only very recently developed; first­order work on recent material not yet completed, too soon to propose second­order (frequently seen for post-cruise proposals); trying to answer too many different questions (don't promise more than you can deliver); preliminary data not sufficiently convincing to begin major new activity. Usual recommendation is to scale way back, do pilot study first.

PI appears ill­informed on topic
Often case of person changing research directions or trying to apply skills from one field to another. Failure to cite relevant papers implies ignorance of work by others; misstates or ignores concepts/facts generally known; misinterprets preliminary data presented in proposal; reasoning doesn't make sense, appears to be based on fuzzy concepts.

Fatal flaw
Existing data disproves or already answers question being posed; error in logic (circular reasoning)

Problems with modeling
Proposals with primary focus on modeling tend to get same types of comments, regardless of discipline: real­world boundary conditions too poorly known to constrain or apply model; no attempt to compare model results to real­world data (model not to be verified); too many assumptions required/under­determined; simplifications needed for computational efficiency make model too general to be of use; scale problem (mostly for lab models: can't assume linear extrapolation to real­world orders of magnitude); no apparent use/need for model perceived by data­based colleagues; model can reproduce reality but so what? (does not give insight into processes and relationships, "just tweaking the knobs Ill you get a fit").