From an Idea to a High-Impact Research Question

A research question can be new, technically interesting and publishable without addressing a problem that matters much beyond the study itself. The difficult decision often comes before the experiments, simulations or data collection: Is this the right question to investigate?

The Research Question Assessment Framework is a practical attempt to make that decision more systematic. It evaluates a proposed question against seven criteria, each scored from 0 to 10, and provides an overall Research Impact Potential Score out of 70. The aim is not to predict citations or guarantee a breakthrough. It is to expose weaknesses early, distinguish a consequential scientific problem from a convenient literature gap, and help researchers decide whether to pursue, refine or abandon an idea.

Why develop this framework?

I began by using large language models (LLMs) to examine the research profiles and selected contributions of 45 pioneering scientists. I wanted to look beyond citation totals and ask why some scientific contributions remain useful long after publication, why other researchers build upon them, and what this might teach us about choosing research questions.

Some profiles contain extraordinary citation numbers. But citation counts alone do not explain scientific influence. They are affected by field size, publication age, collaboration patterns and many other factors. Nor do the profiles establish why a particular paper became influential. The exercise is exploratory, not a controlled bibliometric study or proof of causation.

Figure 1. Illustrative citation histories from two researcher profiles examined during the exploratory analysis. They motivate the question of lasting influence; they do not validate the scoring framework or demonstrate the impact of individual publications. Image credit: Google Scholar.

The exercise suggested a useful distinction: an unstudied topic is not necessarily an important scientific problem. A stronger question identifies an unresolved bottleneck, offers a distinct advance, can be examined rigorously, and may give others knowledge or tools on which to build. I developed these considerations into a transparent scoring rubric that can be applied before committing to a full research programme.

The seven-criterion assessment

Infographic of the Research Question Assessment Framework: seven criteria scored 0–10 each (scientific bottleneck prevalence; novelty and distinct contribution; generalisability and downstream use; scientific testability and rigour; timing and historical relevance; enduring value and delayed impact; feasibility and opportunity cost), with their scoring formulas and a total score bands from very low potential (0–19) to exceptional potential (60–70). — **Figure 2.** Research Question Assessment Framework. Seven criteria contribute up to ten points each, producing a total score between 0 and 70. Image credit: Yusuf Radad.

Each criterion asks a different question. Some components can be estimated quantitatively from the literature; others require evidence-informed scientific judgement. The equations below define a structured assessment index, not an empirically calibrated model of future research impact.

1. Scientific bottleneck prevalence (0–10)

How widely does the relevant literature recognise the proposed problem as an unresolved bottleneck?

\boxed{S_1=10\frac{N_b}{N_r}}

Here, $N_b$ is the number of relevant publications explicitly identifying the bottleneck, and $N_r$ is the total number of publications in a representative, predefined literature sample. Each publication is counted at most once. For example, if 24 of 80 relevant papers identify the limitation, $S_1=3.0/10$ .

The search must cover the scientific domain without selecting only papers that already mention the proposed bottleneck. The score measures recognised prevalence, not the intrinsic importance of solving the problem. A neglected but foundational question may score low.

2. Novelty and distinct contribution (0–10)

How substantially would the proposed contribution depart from the closest established work?

\boxed{S_2=L(C\mid P)}

$C$ denotes the proposed contribution, $P$ the closest prior work, and $L$ an evidence-based novelty level:

Score	Anticipated contribution
10	Foundational discovery or new scientific principle
9	Previously unknown mechanism or scientific explanation
8	Genuinely new theory or method
7	Major advancement of an existing theory or method
6	Significant modification resolving an established limitation
5	Novel integration or substantial adaptation of existing methods
4	Established methods applied to a new problem with distinct scientific insight
3	Incremental extension of established knowledge
2	Another geometry, site, dataset or configuration
1	Replication or minor refinement
0	No identifiable novelty

The scale assesses the contribution, not the sophistication of the technology used. Applying CFD, machine learning or a complex experiment is not inherently original. Novelty must be assessed against the closest credible predecessors, including contradictory findings.

3. Generalisability and downstream use (0–10)

How far could the result travel beyond the original study, and what might others do with it?

\boxed{S_3=G+U}

$G\in[0,5]$ measures transferability: from a single case (0), through related configurations (2), multiple systems (3) and a general physical class (4), to defensible cross-disciplinary application (5).
$U=\min(5,N_u)$ , where $N_u$ is the number of distinct, literature-supported downstream application categories that the expected result could enable.

Examples of downstream use include a predictive model, design rule, benchmark, measurement method or reusable dataset. Multiple papers describing essentially the same application should not be counted as multiple uses. Broad relevance is valuable, but a narrowly applicable result can still be consequential.

4. Scientific testability and rigour (0–10)

Can the proposed investigation distinguish explanations and produce trustworthy evidence?

\boxed{S_4=H+T+V+Q+C}

Each component receives 0 (absent), 1 (partly addressed) or 2 (convincingly addressed):

Symbol	Component	Maximum
$H$	Clear hypotheses or discriminating predictions	2
$T$	Decisive scientific test	2
$V$	Independent validation	2
$Q$	Uncertainty quantification and robustness	2
$C$	Comparison against the strongest relevant existing explanation or method	2

A useful test should be able to challenge the underlying premise, not merely generate results. A negative finding can be scientifically valuable if it rules out a credible explanation. The rubric applies to theoretical as well as experimental, computational and data-driven research; relevant forms of proof and validation will differ.

5. Timing and historical relevance (0–10)

Why investigate this question now?

\boxed{S_5=R+E+W}

Symbol	Component	Maximum
$R$	Relevance to contemporary scientific challenges	4
$E$	New enabling developments in theory, methods, instrumentation or data	3
$W$	Time sensitivity: value that could be lost by delaying the study	3

Recent reviews, research roadmaps, and changes in recognised bottlenecks may inform the score. Popularity alone is not evidence of scientific importance. Timeless questions can be valuable even without immediate urgency.

6. Enduring value and delayed impact (0–10)

Could the contribution remain useful 10–20 years from now?

\boxed{S_6=L+R+F}

Symbol	Component	Maximum
$L$	Longevity of the underlying scientific contribution	4
$R$	Long-term reusability of theories, methods, models, benchmarks or data	3
$F$	Future relevance, including plausible delayed uptake	3

The question is not whether a paper will accumulate citations for decades, but whether its knowledge or capability could remain useful. Historical examples of comparable methods or findings may support this judgement. Potential delayed recognition is a possibility, not a bonus in itself.

7. Feasibility and opportunity cost (0–10)

Can this team investigate the question credibly, and is doing so a justified use of its resources?

\boxed{S_7=A+E+D+O}

Symbol	Component	Maximum
$A$	Availability of necessary data, expertise and resources	3
$E$	Technical and practical execution feasibility	3
$D$	Control of critical dependencies and risks	2
$O$	Opportunity cost compared with alternative research questions	2

Feasibility is researcher- and team-specific. A question may have great scientific potential but be impractical for the present team. Evaluate the smallest decisive pilot separately from the full research programme; difficulty should not be confused with lack of intellectual merit.

From seven scores to a decision

The total is an unweighted sum:

\boxed{S_{\mathrm{total}}=\sum_{i=1}^{7} S_i,\qquad 0\le S_{\mathrm{total}}\le 70}

Total score	Classification	Suggested response
$60\le S\le70$	Exceptional potential	Strong candidate for a major research programme
$50\le S<60$	High-impact potential	Prioritise serious investigation
$40\le S<50$	Promising research	Pursue selectively or sharpen weak dimensions
$30\le S<40$	Moderate / incremental	Redesign unless clear practical benefits justify it
$20\le S<30$	Low potential	Usually reconsider or substantially reformulate
$0\le S<20$	Very low potential	Generally drop or return to problem formulation

These categories are provisional decision aids, not probabilities or validated thresholds for future citations, publication outcomes or scientific breakthroughs. In particular, a high total should not conceal weak originality or a lack of a discriminating test.

For a more informative comparison, the two dimensions can also be kept visible:

S_{\mathrm{scientific}}=\frac{1}{6}\sum_{i=1}^{6}S_i,\qquad S_{\mathrm{execution}}=S_7.

This separates the question's scientific promise from the current team's ability to execute it, while retaining the agreed total out of 70.

How to use the AI assessment skill

The accompanying Research Question Assessment AI skill applies the rubric to a proposed research question. It asks for a literature-grounded comparison against the closest existing work and returns seven scores, each with a critical explanation of no more than 50 words, followed by the total and its classification.

A researcher only needs to describe the question and the intended investigation. For example:

I am considering investigating how front and back wall geometries affect the hydrodynamic performance of caisson-integrated oscillating water columns, particularly the role of turbulent energy dissipation. I plan to use physical experiments and CFD simulations to study the mechanisms, performance differences and scale effects. Use the research-question-assessment skill to critically evaluate this question against the existing literature.

Download the SKILL.md file from the GitHub repository

The Research Question Assessment AI skill is open source: github.com/YusufRaddad/research-question-assessment.

The tool is intended for decisions before a study starts. Evaluating a published paper with its results already known invites hindsight bias. The skill should also distinguish verified literature evidence from inference, mark unverified novelty as provisional and refrain from inventing publication counts merely to calculate a score.

What the score can—and cannot—tell us

The framework deliberately combines one literature-frequency measure, several explicit scoring rubrics, and informed judgement. This makes the assessment transparent and contestable, not mechanically objective.

Four limitations matter:

Recognition is not importance. Frequently repeated bottlenecks may reflect research fashion; emerging problems may be absent from the literature.
Novelty and future value are uncertain. Even a careful search may miss relevant predecessors, and no rubric can establish the future significance of an untested result.
Scores are ordinal and partly overlapping. Adding them with equal weights is a useful convention, not a demonstrated statistical law. Results should be checked for sensitivity to weights, definitions and evaluator disagreement.
Evidence quality matters. Literature boundaries, sample sizes, source dates, key assumptions and the confidence of each judgement should be recorded. Missing evidence should be labelled unknown, not silently scored zero.

Established guidance on research assessment similarly stresses that numerical indicators should support, rather than replace, expert judgement. This framework is also related in spirit to structured research-priority exercises, while its seven criteria and thresholds are specific to the present proposal.

The current version is experimental and has not yet been validated against independent expert rankings or historical research outcomes. A useful next step is to test it prospectively on competing research questions and retrospectively on historical questions using only the literature available before their results were known.

The question worth asking first

The framework does not promise to identify the next landmark discovery. Its purpose is more practical: to make the intellectual decision preceding a research project more demanding.

Before asking how to conduct a study, ask whether its central question is worth answering.