Theodore B. Taylor, one of the pionneers in the ORION project, feared later in his life the proliferation of potential bomb material. He thought we should stop using nuclear power, and presented a talk about "putting back the nuclear genie into the bottle". http://www.nci.org/new/ib5196.htm
Indeed, nuclear power resembles a genie, it can accomplish miracles and at the same time has a great destruction potential. The question is: once the genie is released, is it preferable, or is it even possible, to put it back into the bottle? Or do we have to learn how to live with it?
It made me reflect about our prehistoric ancestors discovering fire. For them, fire had probably all the attributes of a genie: a wonderful instrument for making things that couldn't be done earlier, a formidable weapon, and the potential of destroyingtheir entire known world. There must have been people at that time proclamating that manipulating fire was dangerous, immoral, unnatural, forbidden. However, those who couragously learned how to control its power (probably burning themselves more than once) were the final winners. And we are their descendents, the descendants of those who have the impudence of being curious experimenting with dangerous things.
The argument can be made against Orion in that regard, but all other nuclear power doesn't use material that could be easily used in a nuclear weapon. It would be easier for someone to mine and enrich their own uranium than to try to use any material from our fuel cycle.