Every startup begins with the founders' vision for how the world could be, with an idea about how to satisfy a lingering need. But not all ideas are born equal. Because startups operate with limited resources and in extreme uncertainty, they are very likely to fail. Therefore, the only sensible startup ideas are ones that have a very high potential payoff in order to compensate for the risk incurred due to the tiny likelihood of success. 
How do you recognize such a high-potential startup idea? I believe it has at least three main characteristics:
- Satisfies a core customer need in a good market. A core need could be a problem that a significant percentage of the market believes to be a major issue or an unrealized desire of the target customer. A good market is either a quickly growing new market or a big market that can be disrupted successfully.   Picking a small problem to solve or a small market to pursue effectively limits the potential size of the startup, which immediately raises the question "is this idea worth it" given the high likelihood of failure, as noted above.
- Provides a solution that is disruptively better than the status quo. Creating a product that is not 10% or 20% better, but 10x or 20x better than what exists today is difficult and rare but key to winning against incumbents or creating substantial value to establish a new market. Without such a drastically better product, the existing momentum is very hard to change, because most customers are naturally risk averse.
- Provides a solution that is scalable. It is fairly trivial to create a hugely better solution at a correspondingly very high cost. However, disruptive products that scale well and remain profitable are rather rare. Often, the key to scalability is some new technology, which pushes what's possible on the cost-quality frontier in some substantial way. That is why technology is often a central component of successful startups.
Coming up with an idea that has all three crucial characteristics is a very challenging task, requiring an extraordinary and rare ability to envision the future that you will be building.  Worse, entrepreneurship is inherently contrarian, which makes distinguishing between visionary ideas and wishful thinking incredibly difficult. That is the leap of faith that all entrepreneurs take when starting a new venture.
In practice, it is very easy to go down the wrong path and waste time working on ideas that are not worth your time. That's why I think it is very important to spend time thinking whether the proposed idea has the characteristics outlined above. One particularly useful question to ask yourself in order to tease that out is "why now?" - what is different about the world today that allows the disruptive and scalable solution that you are dreaming about to exist even though previous attempts failed? Answering "why now" helps evaluate ideas by highlighting assumptions, which might otherwise remain hidden.  Once assumptions are known, you can focus on validating them through good startup management - which I discuss next.
 Picking a high-potential idea carries many additional benefits. For example, big ideas make it easier to attract investors, recruit employees, and get media attention. Conversely, picking a smaller idea makes all of the above harder but reduces risk marginally, if at all, while still requiring an extraordinary time investment.
 "Predicting" what markets will be good in the near future is incredibly difficult and a major topic in this excellent speech by Don Valentine, founder of Sequoia Capital.
 To give you a sense of how rare such ideas are, I reference Marc Andreessen and Andy Rachleff who have said that out of the thousands of new companies started every year there are only about 10-15 VC-funded companies that will ultimately reach $100 million in annual revenues.
 Avichal Garg has an excellent blog post that expands on the importance of "why now."