Skip to content
DataStoa

Playbook · 5 min read

How to evaluate a data provider: 12 questions to ask

Twelve questions that separate reliable data providers from persuasive ones, covering sources and rights, coverage, accuracy, freshness, delivery, pricing and what happens when you stop.

By DataStoa Editorial · · Updated 11 Nov 2026

How to evaluate a data provider: 12 questions to ask
In this guide

Two providers can describe the same data in almost identical words and deliver very different results. The only reliable way to tell them apart is to ask specific questions, compare the answers and verify the claims with a sample. This playbook gives you twelve questions, grouped by theme, with what a good answer looks like and the red flags to watch for.

Before you start: decide what passes

Write down your pass and fail thresholds first. For example: "at least 85% of our 500 target accounts present, at least 90% accuracy on headquarters country, updated at least monthly, licence must allow use in our product". Thresholds turn a subjective sales conversation into an objective comparison, and they stop the most persuasive presenter from winning by default.

Source and rights

1. Where exactly does the data come from?

A good answer names the source types (public registries, licensed partner feeds, web collection, panels, surveys, first-party platform data) and explains how they are combined. "Proprietary sources" with no further detail is not an answer.

2. Do you have the right to license this data for my use?

The provider should explain the chain of rights: whether they own the data, license it from third parties, or derive it from public sources under terms that allow commercial reuse. If the data is resold from another supplier, ask whether that supplier permits it.

3. How do you handle personal data?

For any data about people, ask about legal basis, transparency notices, opt-out handling and how often suppression lists are applied. For aggregated data, such as foot traffic or card spend, ask how small groups are protected from re-identification.

Coverage and quality

4. What is covered, and where are the gaps?

Ask for coverage broken down by your dimensions: country, industry, company size, time period. Good providers volunteer their weak spots. A provider that claims complete coverage everywhere is either very new or not being candid.

5. How do you measure accuracy?

Look for a method, not just a number: what was checked, against which ground truth, on how large a sample and how recently. Accuracy claimed without a method cannot be compared across providers.

6. How fresh is the data?

Separate collection frequency from delivery frequency. A file delivered daily may contain records that were last verified a year ago. Ask for the distribution of last-verified dates, not just the update schedule.

7. How far back does the history go, and is it point-in-time?

For modelling and backtesting, you need to know what the data looked like at each past date, not today's values copied backwards. Ask whether historical records are snapshots or restated, and whether revisions are flagged.

Delivery and operations

8. How is the data delivered, and how stable is the schema?

Confirm the delivery methods and formats you need, how schema changes are announced and how much notice you get. A changelog and versioned documentation are good signs.

9. What happens when something breaks?

Ask about support hours, response times, how late or faulty deliveries are handled and whether there is a service level for API uptime. Ask for an example of a recent incident and how it was communicated.

Commercials and trust

10. How is it priced, and what changes the price?

Understand the price model and the levers: coverage, history, refresh rate, users, affiliates, use in products. Ask for the renewal policy too, including whether price increases are capped.

11. Who else uses it, and for what?

References from customers with a similar use case are worth more than logos. If references are not possible, ask for anonymised case studies with concrete outcomes.

12. What happens when we stop?

Clarify deletion obligations, whether derived data (scores, models, aggregates) can be kept, and whether historical deliveries can be retained for audit. This question often reveals restrictive terms early.

Scoring the answers

Use a simple scorecard so the comparison is visible to everyone involved:

Area Weight Provider A Provider B Provider C
Source and rights 25%
Coverage of our scope 25%
Accuracy and freshness (from sample) 25%
Delivery and support 10%
Price and licence fit 15%

Score each area from 1 to 5, using sample results wherever possible instead of claims. Adjust the weights to your use case: an investment team will weight point-in-time history heavily, while a sales team will weight coverage and accuracy of contacts.

Red flags

  • The provider cannot or will not describe its sources.
  • Samples are only available as a hand-picked set and never on your entities.
  • Accuracy figures have no method or date.
  • The licence is not available until after you commit.
  • Every question gets a yes, including contradictory ones.
  • Pressure to sign quickly with a discount that expires within days.

None of these alone proves a provider is bad, but each deserves a direct follow-up question.

Putting it into practice

Send the twelve questions in writing, ask for written answers and keep them with the contract. They become a useful reference at renewal time, when you can check whether the data matched what was promised. On DataStoa you can send the same questions to several providers from their listings, or include them in a data request so every proposal answers them up front. Start with a search by data type to build your long list.

Frequently asked questions

How many providers should I evaluate?

Three is usually the right number for a detailed evaluation. Start with a longer list of five to eight, then cut it using the first few questions and the listing details.

Should I trust quality metrics shown on a listing?

Treat them as a starting point. Ask how they were measured, and confirm them with your own sample test on records you know well.

Is a verified provider guaranteed to have good data?

No. Verification confirms that the business is real and has passed a review, which reduces risk. It does not replace testing the data for your specific use.

What if the best provider is also the most expensive?

Ask what scope reduction would bring the price within budget, such as fewer countries, a shorter history or fewer refreshes. Compare cost per useful record, not list price.

Ready to find the data?

Search verified providers, or describe what you need once and let providers come to you with a price.