Start with the problem
For example, formulate low CTR on a major query segment, poor stock positions, or a recurring relevance error.
Query segments →Academy · Advanced
Improve ranking without optimizing on feeling.
A ranking change can help and damage strong search routes. Therefore, work with an explicit problem, a hypothesis, a defined change and pre-selected evaluation criteria.
Make the hypothesis explicit
Describe what change you make, why it should help and what outcome it should yield.
Experimental template →Use a query set
Check known strong, weak and critical queries before and after the change.
Query test set →Save regressions
An improvement on one segment is insufficient when important exact or commercial queries deteriorate.
Regression tests →Choose appropriate KPIs
Use judgments in addition to behaviors such as CTR and conversion. One metric should not make the rest invisible.
KPI Framework →Capture learning points
Also, document experiments that don't work so that the same change doesn't come back without context later.
Change log →Release discipline
From experiment to safe change.
Use a release gate when the test set and key KPIs demonstrate that the change remains within agreed limits. When in doubt, turning back is better than accepting an unexplained regression.
StepQuestionEvidence
ProblemWhat's going wrong?Data/query set
HypothesisWhy does this help?Expected outcome
EvaluationWhat changed?Judgements + KPIs
ReleaseIs it safe?Gate/regression check