The Dangers of Self-Driving Cars: Is it the Technology or the Humans?

Source: Car and Driver

On Thursday, Tesla recalled more than 350,000 automobiles with its experimental self-driving software technology.  Apparently, the software may lead to accidents in intersections, and it can cause cars to move at excess speeds at times. 

Interestingly, Missy Cummings, a George Mason University engineering professor and former safety official at the National Highway Traffic Safety Administration, had warned about self-driving technology's potential problems and limitations in a New York Times article on Wednesday.  She argued that some drivers place too much trust in the self-driving technology, and thereby place themselves and others at risk.  

The Cummings interview emphasized a concept I've been writing about for several years:  compensatory behavior.  The idea is simple.  If we know about certain systems intended to assist us and potentially make us safer, we grow dependent on those systems and sometimes compensate by actually taking more risk.  I first explored the idea in the context of Everest expedition teams.  I once asked the great mountaineer, Ed Viesturs, why he often climbs without supplemental oxygen.  He replied, "I don't think oxygen necessarily makes you safer in the mountains.  It can give you a false sense of security."  As a result, climbers engage in reckless behavior, not realizing quite how much danger they are encountering because of the safety blanket that they think the oxygen has provided them.  Similar issues arose in my study of the BP oil spill in the Gulf of Mexico, as well as the Boeing 737 MAX design and subsequent crashes.  In the latter case, I also learned that pilots can grow dependent on automated systems, and their skills may erode over time as a result.  Think about driving a car again.  If you always use the backup camera and parking-assist features, you may not be as capable of parallel parking if suddenly those systems are not available to you.  

In sum, I'm not against technologies such as self-driving cars.  I do think, however, that we must become aware of the dangers and risks.  Those risks are not all simply about potential flaws in the technology.  Those flaws also have to do with how humans interact with that technology over time. 

Is Tesla Highly Vulnerable to a Hidden Risk or Ambiguous Threat?

source: flickr

This week, I read Charles Duhigg's incredible article about Elon Musk at Tesla (ok, I'm biased... Charles is my former student). The lengthy piece, written for Wired, is titled Dr. Elon & Mr. Musk: Life Inside Tesla's Production Hell.  Duhigg's article is based on an in-depth investigation of the company and its mercurial leader.  He explains:

Over the past six months I’ve communicated with dozens of current and former Tesla employees, from nearly every division. They describe a thrilling and tumultuous workplace, where talented engineers and designers have done some of their proudest work but where, as one former executive put it, “everyone in Tesla is in an abusive relationship with Elon.” Almost all these employees spoke on the condition of anonymity because of nondisclosure agreements or fears of being sued or fired by Musk. (Even those with positive things to say asked for anonymity.)

The story describes Musk firing employees on the spot without much warning.  It describes his brilliance, as well as the incredible insights that he has had about revolutionizing the car industry.  Duhigg also describes a turbulent workplace and somewhat shocking tirades and rather unpredictable behavior by the boss.  He writes,

At Tesla, Musk’s oddness was accepted. He was, after all, the leader, the biggest stockholder, the visionary. But sometimes his impatience would turn into tirades. “We called it ‘the idiot bit,’ ” a senior engineering executive told me. “If you said something wrong or made one mistake or rubbed him the wrong way, he would decide you’re an idiot and there was nothing that could change his mind.” Musk would openly deride employees in meetings, according to numerous sources, insulting their competence, bullying those who had failed to perform, demoting people on the spot.

Duhigg describes the dismismall or departure of many of his senior managers over the past few years.  Some left due to burnout, disagreements about the direction of the firm, or other opportunities.  Others were fired by Musk.  As I read the article, I wondered about more than whether whether talented managers will continue to flock to Tesla because of the exciting work and ambitious goals.  I wondererd whether Tesla could be highly vulnerable right now, despite the fact that it has ramped up product and begun to turn a profit.  Clearly, they build a beautiful car that has wowed consumers.  They have cultivated high willingness-to-pay for a premium product.  On the other hand, what if there's a problem with product quality or safety?  What if an ambiguous risk or threat emerges?  Will people be willing to share the bad news with Musk, or will such risks remain hidden?  Could such a hidden risk become the firm's Achilles heel?  Duhigg writes about pushing back if you disagreed with Musk:

And a troubling trend emerged, according to former executives: If someone raised concerns or objections, Musk would sometimes pull the person’s manager aside and order that the offender be reassigned, or potentially terminated, or no longer invited to meetings. Some executives began excluding skeptics out of self-preservation. “If you were the kind of person who was likely to push back, you got disinvited, because VPs didn’t want anyone pissing off Elon,” one former executive who reported to Musk told me. “People were scared that someone would question something.”

Those statements worry me a great deal.  Bad news does not rise to the top in most organizations. It often remains hidden.  Ambiguous risks certainly get downplayed or suppressed if leaders create the wrong kind of climate or environment within the firm.  As the organization grows, it will become harder for Musk to know everything.   Will people filter the information that he receives in ways that could become very problematic?  Stay tuned.  

Musk, Tesla, and the Leap of Faith

Elon Musk and Tesla have dominated the business news over the past several months.  Observers and analysts have watched closely to see if Tesla can successfully ramp up production of the Model 3 sedan.  The ability to scale production will prove essential in the organization's efforts to achieve profitability.  The company has little chance to turn a profit, regardless of the appeal of the brand and the quality of the vehicles, if it cannot produce higher volumes so as to lower manufacturing cost per unit.  Of course, more recently, Tesla has made news of a different sort, as many people have questioned Musk's ability to manage an increasingly complex organization.   Late-night tweets, strange comments with regard to the Thai cave rescue, surprise pronouncements about taking the company private, and even smoking marijuana on a podcast have all captured a great deal of attention.   People wonder if he can keep up the long hours, as well as whether he can retain executives on his team.  He has churned through a great deal of managers in the past year, just as he needs the ability to delegate key operational responsibilities to trusted subordinates.   

For me, the interesting part about Tesla's situation has to do with the way people value the company.  How can a firm lose tons of money quarter after quarter, yet until recently maintain a market value over $50 billion?  Faith.  Lots of it.  Investors believe that the company has a promising future.  In particular, they have a great deal of faith in Musk himself.   Investors have viewed him as a visionary, and they expect strong future cash flows because of the strong brand that he has built and the lead he has taken in building electric vehicles.  However, in these cases, where so much of the market value is predicated on the faith in one leader, much risk comes along too.  What if investors lose faith in the leader?  How damaging will that be both to the firm's market value and its ability to raise additional capital?  In the case of Amazon, the firm lost money for years, yet investors and analysts retained a great deal of faith in Jeff Bezos not simply as a visionary, but as an operational leader.  They believed that he could execute.  Musk faces an inflection point now.  Can he inspire renewed faith among the investor community?  Can he regulate his behavior so as to make people comfortable with his ability to lead the organization as it scales production substantially?  Can he build a management team that is willing and able to work for him, despite his micromanagement tendencies?   Amidst all the numbers and the debates about production costs, margins, and the like, the future of Tesla rests in large part on the "soft stuff" - the behavior of a leader and the faith of others in that leader.  Small stuff matters.  Musk has to recognize that his every action and word will be scrutinized closely, fairly or unfairly.  Words and small deeds do matter, regardless of how beautiful the car looks and how well it drives.  Meanwhile, the Board has a crucial responsibility.  The directors must exercise their duties and obligations vigorously, to insure that Musk addresses some of his shortcomings quickly and effectively.  The coming weeks and months are sure to be fascinating to watch.  

Layoffs at Tesla: What Does It Mean?

Yesterday, Elon Musk announced substantial layoffs at Tesla (9% of its workforce).  Successful, high-growth companies typically do not let go of that many employees.   What's wrong at Tesla?  Clearly, the company has struggled with production of the Model 3.   Many analysts have said that Tesla will need to raise significant amounts of capital in the near future.  Musk has denied that such a move is necesssary.  His recent announcement regarding the scaling back of capital expenditure spending plans, and now the layoffs, suggest an ambitious effort to conserve cash.   Tesla's challenge is not the lack of profitability (it has not been profitable for 15 years).  Instead, it's problem is cash.  Many investors clearly have not been worried too much about the lack of profitability.  However, they would become very concerned if the company started running out of cash and then faced challenges raising new capital at attractive terms. 

Peter Eaves of the New York Times has an interesting take on the dilemma that Tesla now faces. He writes, 

"Still, sizable layoffs and moves to conserve cash typically are not the acts of growth companies that are having just a little trouble achieving their goals.And there is a danger for Tesla if it starts to behave like a normal company: Investors may start to value it as one, and its highflying stock may tumble.

Eaves' point is that investors have ignored the lack of profitability for years.   They have expressed profound belief in the growth story, and they have been willing to fund huge capital expenditures in pursuit of that growth.  If investors suddenly started valuing the company differently, then the stock price would face significant pressure. 

Tesla's competitors must watching this situation with much curiosity.  After all, the incumbent car companies have faced their own dilemma for years.  Consider a company such as BMW.  Investors have valued the strong profitability it has generated over the years.  It cannot simply persuade investors to absorb hugeTesla-like losses for years in pursuit of an electric vehicle future.  Somehow, BMW has to invest in the future while still maintaining enough profitability to placate investors.  In some ways, Tesla has had a huge advantage over firms such as BMW, because investors have given Musk essentially a license to lose money for years in pursuit of an EV future.   Is that changing now, and will it change the competitive dynamic in the industry?   We will learn a lot more in the coming months, as we see whether Tesla can ramp up Model 3 production successfully without raising more capital.