President & Co-Founder @OpenAI
Joined July 2010
- Tweets 4,815
- Following 65
- Followers 495,334
- Likes 1,386
One very simple technique which helps me move faster: when printf debugging, print out full log messages rather than just bare quantities: e.g. print(f"Reached speed: {velocity=}") rather than print(velocity). More typing but saves puzzling out meaning of each line at runtime.
19
8
1
219
For example, a few years ago we made the painful decision to shut down our robotics project. It was producing great results, but we'd discovered we could move even faster in data-rich virtual domains. Most of that team started working on code, which led to Codex & Copilot.
3
4
62
The health of an organization’s vision is better measured by its ability to cancel the right projects than to start new ones.
Saying yes to a new project is the easy part, what’s hard is making tough & visionary decisions in the service of focus. Superpower if you can do it.
4
18
1
146
Security is the art of sufficiently raising the cost of attack such that you either (a) will yield a net negative return on investment to adversaries, or (b) are a much less attractive target than alternatives.
2
12
2
68
Management technique we learned early on: short-term machine learning deadlines can be set based on inputs (e.g. high-quality execution on a set of experiments) but not outputs (e.g. reaching some level of performance). Science does not bend easily to the wishes of managers.
“Are there any software engineers that switched into a machine learning role and found it a lot more stressful due to deadlines combined with the uncertainty of research?”
Discussion: old.reddit.com/comments/ulsuzn
21
136
21
908
Trick to avoid all timezone bugs: represent times as Unix timestamps (i.e. a number) rather than your language's Time or Date object.
Additionally makes time arithmetic much clearer, and is especially friendly when building an API where remote machines will parse the timestamp.
10
7
3
128
Such an indescribable feeling when one variant is showing great improvement relative to the baseline... and then upon more careful inspection you realize you had a bug and both runs were actually identical except for different seeds.
9
4
2
118
I fell in love with programming right before college. I remember surprise & delight when I realized that college was prepared to give me credits for getting better at my newfound hobby. Even today I am amazed that it’s possible to be employed to program. The joy has not faded.
11
12
2
274
The file system makes for a great key-value store but a poor general database — writing and reading a key is very fast, but it's slow to list (with 100k+ entries), and many tools choke on giant directories. Still, sometimes can be a great decision to use it to back your app.
4
4
2
74
Modern feature that has most changed my use of Python: dataclasses. Not just useful for data structures — they let you separate state from implementation of any class. Also dataclasses tend to be more useful by default, since they come with repr, eq, etc.
6
21
2
275
First rule of system administration — a service tends towards one of three states: manually watched, monitored, or broken.
1
4
83
A software skill people not often appreciated outside security: probing the emergent semantics of a complex system. First understood this at Stripe while integrating various financial backends. Turns out to be extremely useful for building systems for training neural networks.
4
11
110
A small taste of the kinds of content that you will be able to produce personally in the not-too-distant future:
Back in 2018 at OpenAI, a few of us wrote a story with gpt as an AI "co-author". We didn't have an AI illustrator back then, but now we sort of do, so I tried plugging the text into #dalle.
Here is the result! “The Bees”, a short story by humans & AIs: web.mit.edu/phillipi/www/the…
12
12
93
Any technology comes with benefits and commensurate risks. AI is unique only in the unprecedented magnitude we expect from both.
Navigating these risks & distributing these benefits is daunting a priori, but each day of progress & experience makes the puzzle a bit more clear.
7
14
1
76
Tending to an ML training run is like gardening.
Need to sprinkle your budding ML model with fresh data, shine energy on it in the form of compute, and nip off any unpromising growths in the form of divergences.
Main difference is that plants at maturity don't speak English.
9
6
111
GPS maps made getting lost largely obsolete. Search engines made recall of specific facts far less important. Very curious what new capabilities AI systems will give us that we will soon take for granted.
45
27
5
293