What is the Value of an AI-Generated Cover Letter?

Source: https://chatmaxima.com/

Cover letters used to provide insight to hiring managers and helped them identify which candidates to select for an interview. A well-written cover letter signaled something about the quality of a candidate. Moreover, a well-tailored letter also could signal that a candidate was serious about the particular job opening. Do cover letters still have signaling value in the age of AI?

Several months ago, Jingyi Cui, Gabriel Dias, and Justin Ye published a working paper titled "Signaling in the Age of AI: Evidence from Cover Letters." They studied over 5 million cover letters submitted to 100,000 jobs on freelancer.com platform.  The examined the impact of a new feature on the platform that uses AI to generate cover letters for job candidates.  Perhaps not surprisingly, "Access to the tool increased textual alignment between cover letters and job posts and raised callback rates."  

However, that is not the end of the story.  The key finding pertained to a substantial drop in the correlation between cover-letter tailoring and invitations to interview, as well as a significant drop in the correlation with job offers.  On the other hand, workers' review scores (a metric developed by the platform to evaluate past work experiences) became more meaningful.  The authors conclude "These patterns suggest that as AI adoption increases, employers substitute away from easily manipulated signals like cover letters toward harder-to-fake indicators of quality."  

Finally, the scholars examined whether people spent time revising or editing the AI-generated cover letter.  Many people did not.  Yet, those people who did edit the letters increased their probability of landing the job!   

Interestingly, another study by Galdin and Silbert also studied job candidates on the freelancer.com platform.  They found that the length of applications increased after the introduction of AI tools to help candidates.  At the same time, "employers had a high willingness to pay for workers with more customized applications in the period before LLMs were introduced, but not after."  In short, they discovered a drop in the value of the well-crafted application as a signal of quality.  That drop had important implications.  They write, "Without costly signaling, employers are less able to identify high-ability workers, causing the market to become significantly less meritocratic: compared to the pre-LLM equilibrium, workers in the top quintile of the ability distribution are hired 19% less often, workers in the bottom quintile are hired 14% more often." 

Have We Failed to Prepare Gen Z Properly for the Workforce?

Have We Failed to Prepare Gen Z Properly for the Workforce?  NYU Professor Tessa West tackles this question in today's Wall Street Journal.  She argues that many recent college graduates do not have the social skills required to communicate clearly, manage conflict, and respond well to constructive feedback. She argues that they lack these skills for three main reasons:

1. West points out that a substantially lower percentage of younger workers have experienced a romantic relationship. Therefore, they have not developed the social competencies that are cultivated through these types of complex relationships. Those competencies (such as how to express emotions and deal with conflict) often are critical for effective workplace interactions.

2. Online education has harmed their ability to collaborate effectively on in-person teams.

3. The heavy reliance on digital communication (texts and instant messages) have made them unprepared to handle high-stakes interactions such as key presentations and in-person meetings, as well as unplanned moments of engagement with their managers.

West has some helpful suggestions for improving the ability of younger workers to navigate the workforce. I particularly liked her thoughts about creating a "culture of asking." Here's an excerpt:

Create a culture of asking. Anxiety leads us to retreat, rather than asking how to approach situations. There will be many times when new employees are unsure of whether the criticism they faced was normal or toxic, if they should approach the team first or their boss over an interpersonal conflict, and what “casual Friday” really means. Leaders should showcase asking. Start with established employees doing it—asking for clarity over jargon in a meeting is a good place to start. People should feel comfortable asking, “Was this feedback negative from the boss? I can’t tell.” They will build social connections while learning the landscape.

Transparency + Clarity: Information Dumps Don't Work

Lately, we hear a great deal about the value of radical transparency.  Consultants argue that leaders need to share information broadly with members of their organization.  They argue that radical transparency builds buy-in, trust, and commitment.  Sounds sensible, right?  Is there a downside though?  What risks does this approach create?

Martin Gutmann and Antoni Lacinai examine this issue in a new Fast Company article.  They contend that transparency only works when coupled with clarity.  They write:

Transparency in a company setting typically means more dashboards, more all-hands, and more context. It feels responsible—especially in uncertain moments—because it signals you aren’t hiding anything.  But facts don’t organize themselves. People still have to decide what matters, what they need to ignore, and what to do next. When leaders don’t provide that structure, they leave teams confused, and teams will fill in the blanks with rumor and gossip. In the end, this leads to more insecurity and more internal politics.

Gutmann and Lacinai argue that leaders need to explain why information is important, and then they must provide clear direction to their team members.  They have be wary of creating cognitive overload and overwhelming their employees with information.  People need to know what the priorities are and how to interpret key data.  

For me, any attempt to add clarity begins with leaders stepping into their employees' shoes.  They need to try to see things from their team members' perspectives.  How will they interpret information?  What might confuse them?  How might they be unsettled by certain data?  Stepping into their shoes will help leaders frame their messages, anticipate challenges, and develop strategies for reducing stress and anxiety among workers.  

NASA Issues Investigative Report About Starliner Failure

Twenty-three years ago, the Columbia Accident Investigation Board issued its comprehensive report on the 2003 Space Shuttle Accident.  My colleagues Amy Edmondson, Richard Bohmer, and I began an intensive research project about the tragedy, and we published a case study, book chapter, and Harvard Business Review article with our analysis.  We argued that the failed mission demonstrated how and why organizations and their leaders actively downplay ambiguous threats at times.  We examined how the culture and team dynamics made it difficult for technical experts to express dissenting views and share bad news.  

Sadly, it seems that many of these same challenges led to the problems on the 2024 Starliner mission which left astronauts stranded on the space station for approximately nine months.  Yesterday, NASA issued its investigative report on the Starliner failure.   Here is the excerpt from the report that caught my attention: 

A posture of risk acceptance was communicated by CCP (NASA's Commercial Crew Program) and Boeing leadership, creating division within the large working/joint team and eroded trust. During the mission, CCP and Boeing operational leadership consistently conveyed a position of risk acceptance and readiness to undock, which many perceived as premature and dismissive of unresolved technical concerns. This was particularly apparent regarding the Service Module RCS 138thruster anomalies. This posture gave the impression that completing the sortie mission was prioritized over a thorough assessment of crew safety risks.  

One interviewee noted, “People said, ‘Why bother? He’s driving in one direction and that’s what he wants.’”   

Some interviewees also mentioned the shuttle operational background of the SMMT Chair, NOM, and CCP PM, and the possible preconceived notion that accepting risk to return the vehicle and crew was the only real path forward. This mirrors decisions made for the shuttle when no safe haven in LEO or alternative return capability was available. 

This forward leaning approach led to a breakdown in open dialogue. NASA institutional stakeholders, including ISSP, FOD, and Technical Authorities, felt their input was undervalued or ignored, requiring governance intervention to ensure additional data analysis occurred before a final crew return decision. The perception that CCP leadership had formed a position before hearing all viewpoints created organizational silence, resistance to collaboration, and stagnation in decision making. 

Strong personalities within CCP and Boeing were seen as overly optimistic in presenting data, which some interviewees interpreted as lobbying rather than objective analysis. This dynamic discouraged dissenting views and contributed to a growing sense of distrust. As one interviewee described, opposing positions felt like “pushing a rock uphill.” 

The situation improved later in the mission when key personnel changes were made within the Boeing team and there was collective recognition that senior leadership should have played a more active role in facilitating respectful engagement across differing perspectives. These changes allowed for more productive conversations regarding the technical qualification campaign of the hardware and testing at the WSTF. The lack of early intervention to address team dysfunction allowed conflict to overshadow mission objectives and delayed consensus on critical decisions. 

Organizational silence, discouraging of dissenting views, dismissed technical concerns, overly optimistic analysis... the pattern is clear.  Once again, we see ample evidence that leadership did not create a culture in which open and candid dialogue could occur about ambiguous risks.  I'm glad to see a careful after-action review taking place here, with transparency about the organizational problems that have been identified (rather than only focusing on the technical problems).  Having said that, now the challenge is clear: can NASA turn these lessons into action and fundamentally change the way future programs are led?  

Overestimating What We Know: The Trap of Effortless Search

You would like to learn about a particular topic.  You have two options.  You can search online for information, which AI tools and search engines summarize quickly for you.  Alternatively, you could dig deeper, read an article or two, listen to a podcast, and... maybe even read a book on the subject!  How much do we rely on the easy route to knowledge, and how confident are we that we have become highly knowledgeable through our rapid search for information?

In her amazing book, Uncertain: The Wisdom and Wonder of Being Unsure, author and journalist Maggie Jackson explores this topic.  She writes:

After even a brief online search, information seekers tend to think they know more than they actually do, according to a decade of studies. In one set of five experiments, people were asked to study weighty topics, such as autism or inflation, before taking a quiz on the subject. Half the participants were told to find an online article on the topic, while others were simply given the same information without having to search for it.  People who searched online were far more overconfident going into the quiz.  In one round, they predicted that, on average, they would get two-thirds of the questions right, although they scored less than 50%. 

In contrast, people who had been given the information studied longer, absorbed more, and got about 60% of the questions right - about what they had expected.  Rarely if ever in life are just handed information. Searching and seeking are the human condition. But how we do so matters.  In the virtual realm, we seem to lose the ability to sense that we don't know, the starting point of discernment.  This false confidence blossoms even when people learn nothing from an online search, further studies show. By assuming we can know effortlessly, we close our eyes to our failings and so to chances to explore.  We run from the work of fully attuning to the here and now, finding in hubris a retreat from the challenges of facing up to reality as potent as that of outcome-oriented fear. 

This research suggests that we need to proceed with caution when we jump to conclusions based on a breezy online search or quick prompt on an AI tool such as ChatGPT, Gemini, or Claude.   Ask yourself: What do I actually know?  How deep and accurate is my knowledge?  Should I be making critical decisions based on this superficial knowledge?

Careful How You Handle Familiar Faces on Your Team

When you take on a new role, do you like adding some familiar faces to your team?  It seems logical.  We want to surround ourselves with a few people we trust and can depend on when we move into a new position.   We hope that having some trusted lieutenants on the team will lead to faster, more impactful action. We may encounter some key risks, though, when we build our team in this fashion.  Morag Barrett, a leadership development expert, has written a terrific article for Fast Company about how leaders should (and should not) build their teams.  

Barrett argues that inner and outer circles may form on the team.  In academic terms, we describe this type of situation as a "fault line" between subgroups, and that type of fracture can harm team performance a great deal.  Barrett writes, "Leaders who 'go way back' share shorthand, context, and trust earned elsewhere. Others, often equally capable with deep institutional knowledge, find themselves outside that orbit."  

She goes on to give an example of one CEO with whom she worked.  She writes, "I coached a CEO who’d brought three former colleagues into a 10-person executive team. Within months, critical decisions were being pre-discussed among “The Four” before formal meetings. The other six leaders became increasingly passive, not because they lacked capability, but because challenging pre-baked decisions felt politically risky."  

I've seen this type of situation as well.  If team members outside the inner circle feel as though decisions are being presented at staff meetings as a fait accompli, then buy-in and commitment will decline precipitously.  Motivation and decision implementation will suffer.  We can bring trusted colleagues onto our team, but we need to work hard to integrate these members with the others in the group.  We can't let people feel as though there is a huge status gap between two subgroups.   

What Happens When We Can Quantify Some Aspects of a Decision But Not Others?

Do we make biased decisions because we are obsessed with quantifying our decision analysis? Linda Chang, Erika Kirgios, Sendil Mullainathan, and Katherine Milkman have published an interesting new study titled, "Does counting change what counts? Quantification fixation biases decision-making."  They asked the question: "Do people decide differently when some dimensions of a choice are quantified and others are not?"

The scholars conducted a series of experiments.  Each decision that the research subjects encountered involved some tradeoffs.  Some dimensions of the tradeoffs were described quantitatively and others qualitatively.  The results of the experiments demonstrated that people tended to prefer the alternatives about which numerical data was offered, rather than qualitative information.  Thea authors explain that, "This 'quantification fixation' is driven by the perception that numbers are easier to use for comparative decision-making."  

The scholars argue that we face many decisions in business and in life in which some dimensions of the tradeoffs can be quantified, but others simply cannot.  The qualitative information may be rich and useful though.  The numbers may tell only a portion of the story.  Think about a manager facing a decision about a brand extension.  Numbers may be readily available demonstrating the potential for sales growth, market share increases, and profitability enhancement.  On the other hand, the risks around brand dilution may be more readily described in qualitative fashion.  Do managers pay less attention those very real brand dilution risks simply because they can't easily produce numbers about how dilution may arise and impact the business?

The scholars conclude, "Those who structure decision contexts ignore quantification fixation at their peril. As quantification becomes increasingly prevalent, people may be pulled away from valuable qualitative information toward potentially less diagnostic numeric information."