If San Francisco had the zoned capacity of New York I doubt it would have New York rents. SF's population limits give it the appearance of a more sought-after place to live than it is.

Also, it's impossible to deny that despite having many sui generis issues, the NYC region is much more sustainably built than the Bay Area. Substantially better transit and even the low-density suburbs tend to be higher-density than their Bay Area analogues.

The Bay Area has a natural advantage in the amount of energy needed for climate control, though. Also, I'm not sure about the low-density suburbs thing. census.gov/dataviz/visualiza… is hard to read, but I think it shows pop-weighted density higher in SF at 25-40 miles out.