You and Your Research

notes from Richard Hamming’s “You and Your Research” speech

https://www.cs.virginia.edu/~robins/YouAndYourResearch.html

This talk is about “great research” and is about how you individually do research, not how to manage projects.

note: “Nobel-Prize work” used as a proxy for “great research”

This talk may be generalized: ” I will talk mainly about science because that is what I have studied. But so far as I know, and I’ve been told by others, much of what I say applies to many fields. Outstanding work is characterized very much the same way in most fields, but I will confine myself to science.”

Why did Hamming study math/CS?

• “At Los Alamos I was brought in to run the computing machines which other people had got going, so those scientists and physicists could get back to business. I saw I was a stooge. I saw that although physically I was the same, they were different. And to put the thing bluntly, I was envious. I wanted to know why they were so different from me. I saw Feynman up close. I saw Fermi and Teller. I saw Oppenheimer. I saw Hans Bethe: he was my boss. I saw quite a few very capable people. I became very interested in the difference between those who do and those who might have done.”
• Interested in why the researchers felt so different from him
• Steps:
  • read lots of biographies
  • ask researchers “how did you come to do this?”
  • drop modesty and say “I would like to do something significant” to himself
    • but “society frowns on people who set out to do really good work. You’re not supposed to; luck is supposed to descend on you and you do great things by chance.”
  • talk in first-person about things he’s seen, done, heard

Pasteur “luck favors the prepared mind”

• the prepared mind sooner or later finds something important and does it → what you end up finding is due to luck, but the act of deciding to do something is not luck
• example: being in the right place at the right time to meet Claude Shannon

Newton said, “If others would think as hard as I did, then they would get similar results.”

• One of the characteristics you see, and many people have it including great scientists, is that usually when they were young they had independent thoughts and had the courage to pursue them.
• Einstein example

One success can bring you confidence and courage (Clogston example)

• One of the characteristics of successful scientists is having courage. Once you get your courage up and believe that you can do important problems, then you can. If you think you can’t, almost surely you are not going to.

Age is dependent on field (math/physicists tend to weight their early findings more; music/politics/literature on older works)

• Nobel Prize Effect: “When you are famous it is hard to work on small problems”

Having fewer resources can make you a more effective researcher

• Example of Institute for Advanced Study in Princeton “ruining” great scientists; best physics came from “shacks” in Cambridge
• Bell Labs not having programmers available → forced Hamming to think about great programming problems
• If you can’t solve a problem you start thinking “why not?”
• “ideal working conditions are very strange. The ones you want aren’t always the best ones for you.”

Drive and compounding interest

• “John was a genius and I clearly was not. Well I went storming into Bode’s office and said, “How can anybody my age know as much as John Tukey does?” He leaned back in his chair, put his hands behind his head, grinned slightly, and said, “You would be surprised Hamming, how much you would know if you worked as hard as he did that many years.” I simply slunk out of the office!”
• What Bode was saying was this: “Knowledge and productivity are like compound interest.” Given two people of approximately the same ability and one person who works ten percent more than the other, the latter will more than twice outproduce the former. The more you know, the more you learn; the more you learn, the more you can do; the more you can do, the more the opportunity - it is very much like compound interest.”
• “You have to neglect things if you intend to get what you want done”
• “Drive, misapplied, doesn’t get you anywhere… Just hard work is not enough - it must be applied sensibly.”

Ambiguity

• “Great scientists tolerate ambiguity very well. They believe the theory enough to go ahead; they doubt it enough to notice the errors and faults so they can step forward and create the new replacement theory.”
• “Great contributions are rarely done by adding another decimal place. It comes down to an emotional commitment.”
• “emotional commitment is not enough. It is a necessary condition apparently.”
  • requirement is to force your subconscious to continually think about your problem
  • “Keep your subconscious starved so it has to work on your problem, so you can sleep peacefully and get the answer in the morning, free.”

Working on important problems (lunch table anecdote)

Over on the other side of the dining hall was a chemistry table. I had worked with one of the fellows, Dave McCall; furthermore he was courting our secretary at the time. I went over and said, ``Do you mind if I join you?'' They can't say no, so I started eating with them for a while. And I started asking, ``What are the important problems of your field?'' And after a week or so, ``What important problems are you working on?'' And after some more time I came in one day and said, ``If what you are doing is not important, and if you don't think it is going to lead to something important, why are you at Bell Labs working on it?'' I wasn't welcomed after that; I had to find somebody else to eat with! That was in the spring.

In the fall, Dave McCall stopped me in the hall and said, ``Hamming, that remark of yours got underneath my skin. I thought about it all summer, i.e. what were the important problems in my field. I haven't changed my research,'' he says, ``but I think it was well worthwhile.'' And I said, ``Thank you Dave,'' and went on. I noticed a couple of months later he was made the head of the department. I noticed the other day he was a Member of the National Academy of Engineering. I noticed he has succeeded. I have never heard the names of any of the other fellows at that table mentioned in science and scientific circles. They were unable to ask themselves, ``What are the important problems in my field?''

If you do not work on an important problem, it's unlikely you'll do important work. It's perfectly obvious. Great scientists have thought through, in a careful way, a number of important problems in their field, and they keep an eye on wondering how to attack them. Let me warn you, `important problem' must be phrased carefully. The three outstanding problems in physics, in a certain sense, were never worked on while I was at Bell Labs. By important I mean guaranteed a Nobel Prize and any sum of money you want to mention. We didn't work on (1) time travel, (2) teleportation, and (3) antigravity. They are not important problems because we do not have an attack. It's not the consequence that makes a problem important, it is that you have a reasonable attack. That is what makes a problem important. When I say that most scientists don't work on important problems, I mean it in that sense. The average scientist, so far as I can make out, spends almost all his time working on problems which he believes will not be important and he also doesn't believe that they will lead to important problems.

• Looking for attacks
  • Most great scientists know many important problems. They have something between 10 and 20 important problems for which they are looking for an attack.
  • The great scientists, when an opportunity opens up, get after it and they pursue it. They drop all other things. They get rid of other things and they get after an idea because they had already thought the thing through.

Working with the door open

• “I notice that if you have the door to your office closed, you get more work done today and tomorrow, and you are more productive than most. But 10 years later somehow you don’t know quite know what problems are worth working on; all the hard work you do is sort of tangential in importance. He who works with the door open gets all kinds of interruptions, but he also occasionally gets clues as to what the world is and what might be important.”
• The people who work with their door close worked on slightly the wrong thing

Compounding important work

• Reframe work such that future you and others can build on top of it. “The essence of science is cumulative”
• Do not work on isolated problems. File away methods and use them for future problems
• Building cumulatively is a deliberate choice: the easy short-term path forces others to duplicate what you’ve already done
• “It’s just as easy to do a broad, general job as one very special case. And it’s much more satisfying and rewarding!”

Selling

• Everyone is busy. You need to present so well that they’ll put their work aside and look at your stuff
• Need to learn how to give both formal and informal talks (don’t be a “back-room scientist” who keeps quiet during conferences and then file a report afterwards)
• Offer to give speeches to a limited audience for practice and feedback
• Authors want to give technical talks and audiences want broad talks. Solution: first say why it’s important, then explain how it’s done
• Goal: audience should come away thinking “that was a good talk” and then be able to understand what the author did
• Many talks are filled with far too much information
• “I believed, in my early days, that you should spend at least as much time in the polish and presentation as you did in the original research. Now at least 50% of the time must go for the presentation. It’s a big, big number.”

Friday afternoon alignment

• Commit 10% of time to “only thinking great thoughts” - ability to re-align every week

Control over work

• In the beginning you won’t have much, but once you’re moderately successful, there are more people asking for results than you can deliver and you have some power of choice, but not completely.
• Educating your boss
  • Clearly explain why your ideas matter
  • Demand that deadlines are reasonable (example of being forced to deliver work by Friday instead of Monday, and then the demander not using it over the weekend)
  • “Thank your programmers” → make sure managers acknowledge important work
  • Selling your ideas also applies here

Is it worth it?

• The struggle to make something of yourself seems to be worthwhile in itself. The success and fame are sort of dividends, in my opinion.
• It’s obvious that those who did great research want to do it again

Personality defects

• thinking you need to fight the system rather than learning to work with the system and take advantage of what it has to offer
  • If you want to do something then go do it instead of asking if you can do it
  • It’s easy to get a “no” if you want it
• Dress appropriately enough so that you fit in and your manners don’t get in the way of your work
  • don’t be super casual in important places, it’s wasted effort to convince people that your thoughts matter
• “change the system” is often a foolish side project
  • Very few of you have the ability to both reform the system and become a first-class scientist.
• Anger is a fault
• Look for the positive instead of the negative

If you really want to be a first-class scientist you need to know yourself, your weaknesses, your strengths, and your bad faults, like my egotism. How can you convert a fault to an asset? How can you convert a situation where you haven’t got enough manpower to move into a direction when that’s exactly what you need to do?