We Set The First Tesla FSD Cannonball Run Record
The Cannonball Run Is The Nurburgring of Autonomy. So We Set A Record In A Tesla.
By Alex Roy, General Partner, NIVC
Eight years ago, Elon Musk claimed Teslas would be capable of driving coast-to-coast without human intervention by the end of 2017. Skeptics focus on missed deadlines, but on a long enough timeline, the success rate of almost all technology is 100%. The only question in my mind has been how far Musk is from fulfilling that promise. With Tesla Full Self-Driving (Supervised) v13 finally rolling out to the public, there was only one way to find out.
The first step was to set a Full Self-Driving (FSD) record coast-to-coast using the last, best version of Tesla’s prior generation software: FSD v12.5.6.4, then drive cross country using FSD v13.2.2, twice, back-to-back. Why three runs in rapid succession? Because progress can’t be measured without a baseline, and I like gathering as much data as reasonably possible before drawing conclusions.
The first drive is complete, and the first coast-to-coast FSD record has been set. The results of the second and third drives will be shared in January 2025.
Clockwise from Top Left: V1 Gen 2, Stinger Laser/Radar UI, Tesla IR Driver Monitoring Illuminators, GoPro Hero 13, Teledyne/FLIR Thermal Camera Display
For the sake of clarity, I will call cross-country driving records in vehicles with supervised automation surpassing 98% Semi-Autonomous, such as Jay Robert’s impressive Comma AI record. If the vehicle is a production car with native software like Tesla’s FSD, the sub-category is Production Autonomy.
The key metrics for these records are Engagement Rate (%) and # of Disengagements. Elapsed time is tertiary until Engagement Rates reach 100%, which will mean an important threshold will have been crossed: unsupervised/driverless coast-to-coast operation will be possible, and Musk’s vision achieved.
Why not just test FSD locally like so many Tesla fans and critics?
Extraordinary claims require extraordinary testing. If one claims either “FSD is almost here” or “FSD can never work”, then current testing — YouTube videos shot in one neighborhood, city, or even state/region — doesn’t cut it. The promise of Tesla’s FSD is that it’s generalizable; it should work everywhere. If the entire world is your intended domain, you’ve got to test across continents, maximizing complexity of weather, traffic, and road conditions.
Anything less is unserious.
Musk promised cross-country autonomous driving, and that means adding the Production Autonomy category to the most infamous, legendary, and technologically underappreciated of American driving traditions: the Cannonball Run.
The Cannonball Run Is The Nurburgring of Autonomy
For decades car companies have tested their performance cars on Germany’s Nurburgring, the longest and most challenging track in the world. The Ring is a known quantity, allowing repeatable apples-to-apples testing. Ring times mean bragging rights, despite having no relationship to how 99% of performance cars are used by their owners. But in a human-driven world, where the potentiality of power is a mere foot press away, and drivers buy cars based on the projection of that potential, a Ring time is a technical metric with priceless emotional value.
For now a Ring time has little relevance to marketing an autonomous vehicle (AV), whose initial form must be driverless, safe, comfortable, convenient, and reliable. Musk’s vision of a coast-to-coast drive dovetails perfectly with testing and demonstrating all of the criteria AV passengers and buyers require.
Musk knows the future doesn’t sell unless it captures the imagination. Autonomy needs its Nurburgring, and the Cannonball Run is now the Nurburgring of autonomy: a grand, repeatable, public spectacle anyone can understand.
Sadly, no car company today wants to embrace the Cannonball brand and extend its legacy to include autonomy, which is ironic given the notorious race’s origin story.
Erwin "Cannonball" Baker in his factory-sponsored Stutz, ready to set another cross-country driving record.
The Cannonball Run has sometimes been maligned as a rolling cantina of scum and villainy, but the naysayers are wrong about its present and ignorant of its past. Cannonballing — the unsanctioned sport of driving across the United States non-stop except for fuel (and recently, recharging) — was wildly popular for most of the 20th century, and has been the best test of personal transportation technologies since its 1915 inception, when car maker Stutz hired Erwin “Cannonball” Baker to drive a Bearcat from LA to NY.
Baker’s drive took 11 days, 7 hours and 15 minutes, and was heralded as a proof that personally owned internal combustion (ICE) cars were safer, cleaner, and more convenient than horses or trains.
As ICE vehicles improved, automakers like Ford, Cadillac, Graham-Paige, and Gardner sponsored Baker to demonstrate their innovations in reliability, fuel economy, speed, safety, and comfort—all to great fanfare. As gas stations and road infrastructure improved, so did Baker’s times and popularity, inspiring a nation to get out, buy their first cars, and explore new roads.
One hundred years later the Cannonball Run is more relevant than ever, as modern Cannonballers are not merely early adopters of new technologies, but often the first to combine them across disciplines, extending innovation far beyond the car. The world’s most disciplined Cannonballers don’t speak of horsepower or top speed; their language is situational awareness, fuel economy, battery efficiency, powertrain software hacking, OTA updates, radar detection, laser jamming, NOAA, Google Earth, real-time maps, FAA data integration, real-time weather & traffic, crowd-sourced data, apps, mission control, synchronizing external spotters, thermal cameras, image stabilization, ambient temperatures, power management, charger handshakes, voltage regulation, thermal management, range optimization, weight savings, redundant communications, and Starlink.
The best Cannonballers know as much as (or more) about innovation than most SAE committee members or OEM futurists, and would be right at home at a defense tech conference or VC partner meeting.
The future of the Cannonball Run is autonomous, not to the exclusion of human driving, but as an inevitable and obvious supplement. People will always want to drive cross country as fast as possible, and the advent of autonomous driving technology poses seductive questions:
What if Cannonballing isn’t only about speed, but also about safety? How much faster could AI safely make a human driver? Once AVs are indisputably safer at current speed limits than human drivers, will future Cannonballers be passenger-engineers competing to raise maximum AV speeds because their customized AI can safely go faster?
How To Classify Cannonball Run Records In The Future
Human-driven Cannonball Run records have traditionally been measured by elapsed time. Were Tesla FSD capable of unsupervised/driverless operation today, elapsed time would be the sole metric, and one could directly compare human-driven and unsupervised/autonomous Cannonball times.
Someday.
Because FSD is not yet capable of Tesla’s stated goal of unsupervised/driverless operation coast-to-coast, and because our intent is to compare the progress of subsequent generations of Tesla FSD (and potential competitors) toward unsupervised/driverless operation, the key metrics are Engagement Rate (ER) and Disengagements (DE), followed by the less relevant (for now) elapsed time.
This requires adapting classes of Cannonball Records into:
Human Driven, ranked by elapsed time
Driverless, ranked by elapsed time
Semi-autonomous, ranked by ER & DE
Production Autonomy Cannonball Record co-driver C. Fox.
The Team
My criteria for co-driver selection included extensive Tesla FSD seat time in myriad conditions, extensive high-speed driving experience, high endurance/physical fitness, and quick reaction times. My selection for FSD run 001 met all these criteria: extreme athlete, Tesla owner, and former Porsche executive C. Fox.
The Rules
No Stopping - As per traditional Cannonball Run rules, there is no stopping except to refuel/recharge, or mechanical failure/repairs. In the context of FSD benchmarking, a non-stop drive guarantees a mix of day/night testing.
Maximize Engagement Rate - Since Tesla’s goal is unsupervised/driverless FSD, our goal is to maximize FSD’s engagement rate. That means engaging FSD at the earliest possible opportunity, and intervening only for critical safety issues or significant navigational errors. In the event of any type of disengagement, it is imperative to re-engage the system as soon as it is safe to do so.
Optimization - While FSD is engaged, users may optimize routing and/or maximum speed.
No Aftermarket “Cheat” Hardware or Software
Several Tesla influencer videos portray hands-free operation, in defiance of Tesla’s user manual. This suggests “cheat” hardware or software has been installed to defeat the steering wheel’s torque sensor, which is meant to confirm presence of the FSD supervisor’s hand(s).
We will not use any such cheats, as the purpose of periodic “nags” — to remind the FSD supervisor to maintain vigilance — is even more essential for maintaining safety over long drives.
Hands-On vs. Hands-Off
Even if Tesla FSD were a hands-off system, as long as supervision is necessary, maximization of safety requires keeping one or both hands on or hovering near the steering wheel. Why? Because if intervention is necessary, the shortest possible reaction time is essential.
Disengagement Criteria
Several autonomous vehicle developers have courted disaster by undertraining test drivers, and/or incentivizing them not to intervene, thereby juicing system performance metrics.
Having gone through rigorous AV safety driver training multiple times during my 4 years as an executive at AV developer Argo AI, I am confident in my ability to safely supervise Tesla FSD for long periods and voluntarily disengage only when absolutely necessary. These protocols were shared with C. Fox over numerous practice sessions.
This is how we classify Disengagements (DE’s):
(IVDE) Involuntary
(VDE) Voluntary, for Charging
(VDE) Voluntary, Safety-Related
(VDE) Voluntary, Navigation-Related
Teledyne/FLIR Thermal Camera display, showing bounding boxes and object classifications.
The Missing Sensor: A FLIR Thermal Camera
I like to see what my eyes (or car) can’t. While others debate Musk’s decision to eschew Lidar and go with camera-only for FSD, I’ve been maximizing safety on my endurance drives for 20 years by adding a Teledyne FLIR thermal camera. A FLIR can see beyond any car’s headlights, and was instrumental in my 2006 Cannonball Run record. It made perfect sense to ask FLIR if they had something I could borrow for the first-ever Production Autonomy Cannonball Record. FLIR generously lent me a prototype thermal pencil camera and interior display, with integrated object detection and identification.
A thermal camera/display is a real confidence booster in low-light conditions. I wish FLIRs came standard on all cars. FLIR hardware is small, inexpensive, and reliable. (And no, I’m not being paid by FLIR.)
Tesla still has the best EV charging network in the USA, by far.
The Car
Vehicle: 2024 Tesla Model S
Wheels: Factory 19” Tempest
Tires: Continental ProContact RX
Mileage (post-run): 8,367 miles
Note the Highway Radar app UI on the bottom right iPhone. An impressive app.
Aftermarket Hardware & Software
All aftermarket hardware was installed and tested by AI Design, to whom I’ve entrusted my life and cars for ~22 years. Here’s the list for this drive, REC 011:
1x Starlink Mini
3x GoPro Hero Black (2x Forward, 1x Interior, GPS enabled)
1x Teledyne FLIR Prototype Thermal Pencil Camera (Forward) w/Interior Display
iPhone 16 running Waze
iPhone 16 running Highway Radar
iPhone 16 running Glympse (for live witness tracking)
iPhone 16 running Speed Tracker (for GPS recording)
The following were superfluous, as FSD restricted top speed to 85mph, and often much less, depending on the speed limit and conditions:
Stinger Laser Jamming & Radar Detection w/3x pair heads front & rear
Valentine V1 Gen 2 Laser/Radar Detector
Tesla FSD Cannonball Run Record 001 Data
The key metrics for a Production Autonomy Cannonball Run record are Engagement Rate and Disengagements, although the second number must be deconstructed to understand system performance over software versions.
Software: FSD v12.5.6.4
Engagement Rate: 98.52%
Disengagements (Total): 32
Disengagements (Voluntary, Charging): 21
Disengagements (Total, non-Charging): 11
Disengagements (Involuntary): 2
Disengagements (Voluntary, Safety-Related): 7
Disengagements (Voluntary, Navigation-Related): 2
Navigation Errors: 3 (VDEs for 2 noted above, 3rd was allowed to play out)
Other Errors: 1
System Resets (Navigation-related): 2
The legacy metrics, for those curious about time and EV stats:
Overall Time: 45 hours, 36 minutes
Drive Time: 39 hours, 44 minutes
Charge Time: 5 hours, 52 minutes
Charge Stops: 21
Total Energy: 941 kWh
Driving Speed (Average): 71 mph
Overall Speed (Average): 62 mph
Distance: 2,833 miles
Witnesses:
Jay Roberts - Semi-Autonomous Cannonball Record Holder
Matt Farah - Automotive Journalist
Mason Dibley - Cannonball Community Ground Controller/Record Keeper
Tesla FSD vs Comma.ai
My friend, fellow Cannonballer (and witness to this run) Jay Roberts recently set a Semi-Autonomous Cannonball Run record of 43 hours and 18 minutes, using an aftermarket Comma.ai unit installed on a Toyota Prius, which enables features similar to Tesla Autopilot on highways, but cannot operate on city streets like Tesla FSD. Although Jay’s overall time was faster than ours, his 98.4% system engagement rate was lower than ours. Also, Jay did not log and categorize disengagements as we did. With Jay’s help, I will address similarities and differences between FSD and Comma.ai in a subsequent post.
Tesla FSD Cannonball Run Record 001 Summary
The key metrics are Tesla FSD’s 98.52% Engagement Rate, and 32 Disengagements. With that baseline set, we are eager to share our findings on FSD v12 vs. v13.
Tesla FSD is by far the most advanced driver assistance system (ADAS) currently available for sale in the United States. However, like all ADAS, safety depends on driver understanding of its limitations, constant situational awareness, and near-instant reaction time. Tesla FSD is superior in its domain size, comfort and automation engagement rate to the Tesla Autopilot used by my team to set the EV Cannonball Run record in 2016.
Next Steps: Tesla FSD Cannonball Run Drive 002/003
We will publish more detailed data/content/comprehensive analysis in January 2025 after reviewing data and video footage from all three drives. Will FSD v13 set a new record for Engagement Rate, and Disengagements?
We can’t wait to find out.
(12/30/24: Engagement Rate revised from 98.7% to 98.52%)