The best way to improve the usability of your product is directly observe actual end-users attempting realistic tasks, so that should always be the first option. In real life all sorts of things happen, so if budget issues, time constraints, or some other issue prevents you from being able to conduct true usability testing of your product, you may want to try a survey. From Jeff Sauro at Measuring Usability:
The perceptions of the user experience in general, and usability in particular are, of course, not a full substitute for data collected in a usability test or from directly observing users with a product.
However, the two types of data—test data and survey data—are related. Surveys won’t provide the rich behavioral data that helps diagnose the root causes of problems, but carefully crafted surveys can provide similar data to usability tests, along with additional benefits. For example:
- Attitudes toward usability and task-based usability metrics (like task completion) have a reasonably strong correlation. Therefore, measuring SUS will provide some idea about task-completion rates, even if you don’t collect task data.
- Attitudes toward usability and likelihood to recommend are strongly correlated. You can collect both in a survey and have measures that are tied more closely to company revenue.
- Users self-reported usability problems often uncover up to half of the same issues uncovered in a traditional moderated usability test.
Source: Using Surveys to Measure the User Experience: MeasuringU