A Month With Tesla FSD (supervised) v12.3
May 2024
In late March 2024, after an overnight software update, my Tesla Model 3 greeted me with this geek-oriented message on its screen: “FSD (supervised) v12 upgrades the city-street driving to a single end-to-end neural network trained on millions of video clips, replacing 300k lines of explicit C++ code.”
Tesla offered all U.S. owners a free one-month subscription to the recently released Full Self Driving or FSD (supervised) v12 — FSD is a ADAS Level 2+ solution. For me, this was an opportunity to assess this drastically revamped version against my experience with FSD v10. I tested the system extensively, on city streets, on highways and freeways, at night, and in the rain in different parts of California. I tried to expose the system to a variety of test conditions to identify both strong points and flaws.
Previously, Elon Musk has promised repeatedly the imminent delivery of a feature-complete solution. It would eventually allow owners to earn a significant return on their investment by putting their EV on Tesla’s “future” autonomous ride-hailing platform. Musk now aims to unveil a robotaxi August 8 and wants to bet the future of the company on autonomous driving.
Tesla clearly intends to leverage this recent testing campaign to vastly expend its FSD userbase and prepare a business transformation. But is FSD close to performing at the level required to power a fleet of robotaxis? What works well and what still needs improvement to provide the level of safety and confidence required to launch and scale the autonomous ride-hailing platform Musk has been promising?
What is The FSD (supervised) I Tested?
Initially priced at $5k for unlimited access when introduced in 2020, FSD’s one-time activation fee increased step-by-step to $10k then $15k in 2022. The fee came down to $12k in 2023, then to $8k this April — note that the service is not transferable to a new owner. Simultaneously, subscription fees dropped from $199 to $99 a month — a better deal unless you keep your vehicle more than 6 years. Both options can be activated from the Tesla app.
A couple of years ago, I subscribed to FSD v10 for $199 for one month on my first Model 3, built in 2020. At the time, I tested it on freeways exclusively, between San Francisco and Los Angeles and in the Bay Area. I was not impressed. I felt neither the monthly fee nor the one-time $15k fee were justified vs. Autopilot, Tesla’s standard Level 2 equipment, i.e., essentially adaptive cruise control combined with automatic lane keeping system (ALKS, or Autosteer in Tesla jargon).
During my month-long test this April, my Model 3 (bought new in Dec 2023) was running FSD (supervised) v12.3.4. The major upgrade of v12 vs. versions up to v11 consists in replacing hard-coded instructions with neural networks for the path planning portion of the stack, leading to an end-to-end deployment of neural networks.
As a reminder, perception of the surroundings is performed by 8 cameras — the radar I had on my previous Model 3 is gone. A camera is also fitted just above the rearview mirror to monitor the driver. All the hardware required to run FSD comes as standard equipment.
FSD (supervised) remains a Level 2+ solution. It features automatic lane change, navigation on autopilot (on freeways, highways and city streets incl. unprotected left turn, traffic lights & signs recognition), intelligent speed control (operating speed can be adjusted manually about the speed limit), auto-park and smart summon. The solution operates at up to 85 mph (135 km/h), at night, and in a light rain. It offers 3 behavior modes — chill, average, assertive — as well as an option to minimize lane changing (see above).
V12 is Much Better Overall than My Previous Experience with v10
FSD v12 is significantly better than the v10 I previously tested. Improvements include the overall smoothness of the drive (most of the time), the system’s ability to better anticipate and avoid obstacles, and the perception feedback within the UI. But it remains far from perfect — more on this later.
I was able to test the system for the first time on city streets, which enables extended journeys from home to final destinations. I once activated FSD right outside my garage in San Francisco only to turn it off 50 km later after exiting the freeway in Palo Alto. There were long portions of the drive (up to 10 min) during which I did not even have to touch the steering wheel — though I kept my eyes on the road as well as my hands and feet ready to take over. I never felt the need to take over during that journey.
Navigation on autopilot anticipates changes in direction with (mostly) appropriate lanes changes. The car passes slower vehicles on the freeway — though it tends to remain in a left lane. When passing heavy duty trucks, FSD positions the car to the left portion of the lane to increase safety (see above). Trajectories are overall much smoother than with v10 which can likely be attributed to the switch to neural nets for path planning.
The car handles roundabouts without issues — although most have stop signs in my area. It drives safely around construction zones and re-routes accordingly. It identifies the opened door on a parked vehicle (visible on the screen) and moves away from it. It also slows down to about 10 mph (16 km/h) when approaching speed bumps for increased comfort. A nice touch. When arriving at destination, FSD finds a spot to park and lets you take over.
The revamped representation of the system’s perception provides increased confidence. It shows the relevant obstacles and is capable to identify and track a large number of vehicles, cyclists or pedestrians all around, including beyond other vehicles. (See below)
I found the Driver Monitoring System (DMS) to strike a good balance between safety and annoyance. It takes about five seconds of looking away from the road (e.g., to the side as a test) before the system warns me with an audio signal and requests that I nudge the steering wheel.
A Number of Safety-Related Issues Remain
There were several occurrences when I had to take over from FSD, including situations where I clearly did not feel safe.
This included the near miss of an extended side mirror on a large pick-up truck which the camera-based perception failed to assess correctly. One time in SF, the car hesitated over about 100 m to move to the right lane where a bike was cruising; I had to take over to stop the swerving and protect the cyclist. There was no real need to change lanes.
A couple of times, the system hesitated when approaching a yellow light; the car first slowed down then accelerated to get through the intersection, which did not feel safe. On several occasions, the car started to depart from the turning lane in which it was rightfully located; it seems FSD identified there was less traffic in the other lane, but I would have missed my turn had I not taken over.
There were also instances when the car drove steadily 5 of 10 mph below the 25 mph stated limit — which the system had correctly identified — without clear reasons. This is not a safety concern but rather an annoyance.
And I experienced many occurrences of the well-known shadow braking, i.e., sudden braking for no visible reason.
Finally, I must mention NHTSA’s investigation of hundreds of crashes and 13 fatalities involving Tesla’s Autopilot in the USA. Among other conclusions, the report shows “evidence that Tesla’s weak driver engagement system was not appropriate for Autopilot’s permissive operating capabilities”. Despite the fact that FSD is not Autopilot, the DMS has been adjusted to increase driver engagement. In any case, it remains critical for drivers to keep their eyes on the road.
FSD Not Yet Ready to Go Unsupervised
In my view, FSD v12 as tested is far from being ready to graduate to Level 4 for a robotaxi application for the soon-to-be unveiled product. I would not ride in the back of a vehicle operated by this solution with no one behind the wheel. There remain major gaps vs. what I have experienced with Waymo after multiple rides — or even vs. Cruise although the gap is smaller.
There is no doubt FSD performance will continue to improve. The Tesla fleet has accumulated over 1.3 billion FSD miles since the feature was introduced including 300 million with v12. The OEM has benefited immensely from the trove of data they have collected which helps further train their models. And I suspect the one-month trial in the USA has further opened the flood gate.
There are potentially changes coming in the sensor suite (cameras only today). Rumor has it in Silicon Valley that Tesla is developing a radar. Moreover, Lidar startup Luminar just disclosed in their latest financial report that Tesla was its largest customer in Q1 2024, representing 10% of their revenue of $21 million. Has Musk’s vision-only position evolved?
By the way, FSD is only available in the USA today. Late April, Musk received a tentative approval to deploy supervised FSD in China, in partnership with Baidu for mapping and navigation services. Furthermore, Musk announced during the company Q1 2024 webcast that it was discussing the possibility of licensing FSD to a major OEM. Development costs must be amortized.
In the end, let’s see what Musk announces in August regarding a robotaxi project, a potential evolution of the sensor suite, and his new prediction for the release of FSD (unsupervised). In the meantime, I will continue to subscribe to FSD (supervised) for $99 a month when I drive long distances as I feel to enhances safety, in particular for boring stretches of freeway.
Marc Amblard
Managing Director, Orsay Consulting