President & Co-Founder @OpenAI
Joined July 2010
- Tweets 4,815
- Following 65
- Followers 487,944
- Likes 1,386
Fun article on a Ruby DSL to interpret natural language-like syntax.
Also interesting to realize that these days, it's actually easier to just use full natural language to generate Ruby code.
Making the Ruby interpreter run a program written in a natural language: dmitrytsepelev.dev/natural-l… Comments: news.ycombinator.com/item?id…
2
3
18
My wife & I just said goodbye to our foster dog. He was rescued from a shelter hours before being put to sleep; we cared for him for a month but were unable to save him. He was incredibly loving; a gentle giant who loved our rose garden and the quiet life. Sad but glad we tried.
14
4
1
225
Most stressful part of college was being told it would be the best four years of my life. Completed only two good-not-great years and was always worried I’d passed peak happiness & forgotten to enjoy it.
A decade later, am far happier and still yet to peak. Chart your own path.
11
22
1
326
Every so often I stumble across some emails I sent a decade ago, and it strikes me how little I knew back then. Very exciting to think I might be able to learn so much in the next decade that in retrospect I will feel the same way about my emails from today.
4
4
159
Building software is an iterative game of increasingly deeply understanding a specific problem and balancing the fundamental tradeoffs you discover.
4
41
2
284
ML bugs are so much trickier than bugs in traditional software because rather than getting an error, you get degraded performance (and it's not obvious a priori what ideal performance is).
So ML debugging works by continual sanity checking, e.g. comparing to various baselines.
46
229
39
1,903
Biggest mental shift while switching from classic to scientific programming: explicit “for” loops are now extremely expensive.
Instead, you express as much as you can by chaining hyperoptimized lower-level “for” loop primitives, eg matrix multiplies.
Fun & new way of thinking.
18
18
2
288
Worst part about using an internal-only library: can't search online for documentation/usage & usually not fully polished.
Best part about using an internal-only library: someone at your company knows every corner of it & any change is on the table.
Not an easy tradeoff.
8
5
115
One very simple technique which helps me move faster: when printf debugging, print out full log messages rather than just bare quantities: e.g. print(f"Reached speed: {velocity=}") rather than print(velocity). More typing but saves puzzling out meaning of each line at runtime.
19
8
1
219
For example, a few years ago we made the painful decision to shut down our robotics project. It was producing great results, but we'd discovered we could move even faster in data-rich virtual domains. Most of that team started working on code, which led to Codex & Copilot.
3
4
62
The health of an organization’s vision is better measured by its ability to cancel the right projects than to start new ones.
Saying yes to a new project is the easy part, what’s hard is making tough & visionary decisions in the service of focus. Superpower if you can do it.
4
18
1
146
Security is the art of sufficiently raising the cost of attack such that you either (a) will yield a net negative return on investment to adversaries, or (b) are a much less attractive target than alternatives.
2
12
2
68
Management technique we learned early on: short-term machine learning deadlines can be set based on inputs (e.g. high-quality execution on a set of experiments) but not outputs (e.g. reaching some level of performance). Science does not bend easily to the wishes of managers.
“Are there any software engineers that switched into a machine learning role and found it a lot more stressful due to deadlines combined with the uncertainty of research?”
Discussion: old.reddit.com/comments/ulsuzn
21
136
21
908
Trick to avoid all timezone bugs: represent times as Unix timestamps (i.e. a number) rather than your language's Time or Date object.
Additionally makes time arithmetic much clearer, and is especially friendly when building an API where remote machines will parse the timestamp.
10
7
3
128
Such an indescribable feeling when one variant is showing great improvement relative to the baseline... and then upon more careful inspection you realize you had a bug and both runs were actually identical except for different seeds.
9
4
2
118
I fell in love with programming right before college. I remember surprise & delight when I realized that college was prepared to give me credits for getting better at my newfound hobby. Even today I am amazed that it’s possible to be employed to program. The joy has not faded.
11
12
2
274
The file system makes for a great key-value store but a poor general database — writing and reading a key is very fast, but it's slow to list (with 100k+ entries), and many tools choke on giant directories. Still, sometimes can be a great decision to use it to back your app.
4
4
2
74
Modern feature that has most changed my use of Python: dataclasses. Not just useful for data structures — they let you separate state from implementation of any class. Also dataclasses tend to be more useful by default, since they come with repr, eq, etc.
6
21
2
275