Tesla Dojo 2, Teslabot News from All Hands

Elon Musk spoke about many breakthroughs at a Tesla All Hands meeting yesterday.

Elon said that the Tesla Dojo 2 chip will be in volume production in a few months. The Tesla Dojo 2 chip has ten times the performance of the Dojo 1.

If Tesla Dojo 2 is ten times better than the Dojo 1 training chip then it would be 3.6 petaflops for fp8. This would be several time better than the Nvidia H100 and maybe 40% of the B200. What needs to be known is how does the Dojo 2 compare for power and cost. It could compare with B200 when factoring in cost. The B200 is sold for about 75% margin. If Nvidia sells the B200 for $40,000 but it costs $10,000 to Nvidia. If it costs Tesla $10,000 for Dojo 2 with the power of 40% of a B200, then it would be worth it. $25,000 worth of Dojo 2 would be equal to a $40,000 B200.

Dojo 1 per-chip performance: 0.362 PFLOPS (362 TFLOPS) at BF16/CFP8.
Dojo 2 projected per-chip performance*: 0.362 PFLOPS x 10 = 3.62 PFLOPS.

Nvidia B200 (Blackwell Architecture)
Process: TSMC 4nm (custom).
Compute:
– FP4 (with sparsity): 9,000 TFLOPS (9 PFLOPS), per Nvidia’s GTC 2024 keynote.
– FP8 (with sparsity): ~4,500 TFLOPS (4.5 PFLOPS, estimated based on scaling trends).
Power: ~1,000W TDP.
Use case: Next-gen AI training, targeting extreme performance.

Nvidia H100
Process: TSMC 4nm.
Compute:
– FP16 (with sparsity): 989 TFLOPS (~0.99 PFLOPS).
– FP8 (with sparsity): 1,979 TFLOPS (~1.98 PFLOPS).
Power: ~700W TDP.
Use case: General-purpose AI training, widely adopted across industries.

Tesla Highlights

– Already produced over 7 million vehicles globally.
– will surpass 10 million next year.
– at times there are rocky times, a little bit of stormy weather but the future is bright.
– Tesla’s work related injury rate has declined over time.
– where does AI and robots fit into the sustainability picture?
– it’s about sustainable abundance for all.
– on track to complete the Semi factory – will make millions of the Tesla Semi – will also have the ability to go autonomous down the road.
– the future is autonomous.
– five years from now, autonomous Teslas will be everywhere – regulatory globally.
– almost the entire fleet – which will surpass 10 million vehicles next year – is capable of autonomy.
– an autonomous car could have the usefulness that’s 5x to 10x a regular car.
– with a software update, we could turn 10 million cars into the usefulness of 50 to 100 million overnight.
Megapack can more than double the total power output of a given grid and excellent for stablizing the grid.
– Long-term more than 90% of all power on earth will be solar + batteries.
– Cell manufacturing – we’re making the most efficient cell in the world – lowest cost per kWh.
– built the first Optimus on the production line in Fremont.
– Should make 5,000 to 6000 Optimus Teslabots this year and parts for twice as many. Ten times as many in 2026.
– Cortex 1 – used for AI training. Over 50,000 active GPUs – soon to be 100,000 GPUs – making it top five in the world.
– Have Dojo 1 active now – handling 5% to 10% of the training load. Dojo 2 – will be 10x better than Dojo 1. Optimistic about Dojo – “have a real shot at a breakthrough.”
– have the cars doing useful work for the first time with no one in them – driving from end of line and park themselves in Frement and now in Austin.
– Tesla self-driving will be 10x better than human drivers.
– Optimus is going into production this year – the new 22 DOF hand is now in production.
– “most sophisticated humanoid robot on earth”
– “our robot has a real brain”
– “Tesla is the leader in real-world AI – what we learned from the cars we transferred to the bot.”
– Tesla is the only company that can make intelligent humanoid robots at scale.
– “Optimus will be the biggest product of all time by far – nothing will be even close – I think it will be 10x bigger than any other product ever made.”
– the Cybercab production line looks like a high speed consumer electronics line – moving so fast that people can’t get close to it – cars coming off the line in 5 seconds.
– we need even bigger casting machines – 50,000 tons – “how big can a casting machine be – what are the limits of physics? – let’s find out!”
– hope make about 5,000 Optimus (parts for 10k to 12k) this year.
An employee asked: is a robot going to steal my job? Elon: people will manage a flock of robots. Same will be true for self-driving cars – people will manage a fleet of cars.
– interesting opportunity to make an electric VTOL jet – “maybe at some point we’ll do that.”
– Hyperloop – vacuum tunnels – high speed autonomous pods from city center to city center.
– Master Plan 3 + abundance for all = Master Plan 4
– Optimus production in Fremont + an even bigger production line in Austin. “We’ll be making tens of millions of robots a year – serious volume – maybe one hundred million robots a year.”