Don’t just test the queries that happen to be demonstrating well.

Volume

Include important top queries.

Diversity

Add long-tail, SKU, brand, category, attribute and natural language.

Problem cases

Keep known zero-result searches and regressions.

Business criticism

Highlight queries whose errors have a lot of impact.

Maintenance

Refresh the set periodically based on real search behavior.