I am interested in how to select interesting yet reasonable problems for students to work on, either at Honours (that is, a research-based single year immediately after a degree) or PhD. 

By this I mean a problem that is unsolved but for which there is a good chance that a student can solve it either completely or partially and come out with a thesis either way.

There are a number of possible strategies that I see some of my colleagues use, but which are sadly not available to me:

(1) Be sufficiently brilliant that you already know roughly how something unsolved should be solved and guide the student accordingly, modifying the strategy on the fly.

(2) Have a major project, say classifying a big class of structures, that is amenable to attack with a big general theorem with many well-defined sub-cases that can be assigned individually to students. 

Personally I often work on problems that lead nowhere - I don't solve it, it's too hard, I only rediscover known examples etc - but provided at least some of the problems work out - it doesn't matter.

But for students, its more of a "one-shot" affair - they can't afford to work for a year or, worse, three years, and get nothing.

Are there any general principles that will help in the selection of problems? Or is just a situation where you've either "got the knack" or you haven't?

Let me first answer a slightly different question, how to organize one's thoughts about such problems. I simply maintain a list of suitable projects, with ideas on how to approach them, and put them in a file "Dissertation Problems.tex". Some of these are projects that I might like to carry out myself, but many are projects that would be suitable for a math PhD dissertation. I have another similar file called "Math Ideas" that I maintain on my mobile phone, so that I can write into it when I am traveling.

It often happens that when I am working on one problem, I have an idea for a related project, or a side question, or a generalization to a method that arises, or an idea for a counterexample to such a generalization. Sometimes, of course, these side problems can be solved or incorporated into the original project. But just as often, they are not directly necessary for the original project, but still interesting, and so I save them for a later project or for a student. Of course, not all ideas pan out, and in many cases, these ideas turn out to be uninteresting or wrong in some way. But often, they have turned into very interesting questions whose resolution forms a dissertation or a paper or joint paper.

In this way, in time I accumulate more problems and questions than I can work on myself, and when my PhD students are ready for a problem, I can suggest several that may be to their liking. Often the student already has ideas for a problem, and in these cases I help them to focus their questions and efforts. These files also work for joint projects with colleagues looking to do a joint project with me. 

The most important thing, however, in this method, is to remember to write the idea down. Surely we all have great ideas from time to time, but then, after we become absorbed in another task or project, the fleeting idea is regrettably forgotten. So it is important to be systematic about recording it. Create a file on your laptop (and on your mobile phone!) to which you add suitable mathematical ideas when they occur to you. Perhaps you want two files, one for student projects, and one for projects you want to reserve for yourself. 

This question I have answered---how to organize one's thoughts about mathematical ideas--- may seem to be a trival matter, but I believe that many mathematical ideas are held in mind only briefly before being forgotton forever, and so I find it to be a critically important issue, an important key to successful mathematical practice.

Let me now turn to the question you actually asked, namely, how to select problems in the first place. This I find to be an intensely personal matter, having to do with one's style of mathematics. We all likely have different mathematical styles, with some of us interested in easy-to-state or sweeping questions about fundamental issues, others preferring to understand motivating principal examples very deeply, and others interested in aspects of some big-machinery construction method, and so on. These mathematical styles will lead to very different sorts of questions and problems, and will naturally affect the kinds of problems that we would find suitable for students. It is of course much easier to find suitable problems for students that arise as part of a big program with many examples or cases that need to be worked out, and these kinds of problems often carry with them the opportunity to learn a part of that machinery. (My own style surely tends more toward quirky but fundamental questions, perhaps underlying or alongside the well-beaten path but not directly on it, and less towards big machinery, but in others eyes I am likely merely presupposing a certain amount of big machinery, such as forcing or large cardinals.) The sweeping-questions problems, however, are more dangerous for students because they can often be harder or of unknown difficulty. But occasionally, I find that an interesting special case or aspect of an interesting new sweeping question, which arises during my own investigations, and when these occur I have found these smaller projects very satisfying mathematically, both for myself and my students. I would never give a problem to a PhD student that I didn't myself find compelling and interesting.
