Who Judges Creative Ideas Well... and Who Doesn't

Stanford Professor Justin Berg has conducted some interesting research on people's ability to evaluate creative ideas.   He asked over 300 circus professionals to forecast how popular a series of circus acts would be with audiences.   He then tracked their actual popularity with more than 13,000 circus audience members.  What did Berg find?   The people who actually created the new circus acts are not very good predictors of their success with audiences.  Interestingly, managers are not very good judges of their success either.   Who is a better judge than either the creators or the managers?  It turns out that peers of the creators were more accurate judges.  In particular, peers of the creators were much better at identifying the potential success of novel/unconventional circus acts.   The research found that, "managers tended to undervalue novel ideas in favor of conventional performances."  Interestingly, this finding about the lack of accuracy of managers' predictions holds despite that the fact that many managers were indeed creators themselves at one point in their careers. 

This finding reminded me of the value of the "Brain Trust" at Pixar.   At the animation studio, the brain trust is a group of people who gather to provide unvarnished feedback to the director of a new film.  The brain trust often includes a few higher-level managers at the firm.  However, the focus of the brain trust is peer-to-peer feedback.  Other directors offer their constructive criticism to the person leading a particular project.  The Brain Trust's success at Pixar seems to reinforce the findings of Berg's experimental research.  

The (Not-So) Hidden Value of Pixar's Short Films

The Wall Street Journal reports this week on Pixar's purpose in making short films.  For instance, Lava is the short film that runs before the newest Pixar feature film, Inside Out.    The short films are a wonderful opportunity for experimentation.   As a result, they help fuel the creative process at the highly successful film studio.  Here's one excerpt from the article:  

[Making short films] is a wonderful opportunity to step out and do something you may not be able to do on a feature, really take some chances,” said Mr. Murphy, a senior animator at the studio. “I think each short really represents the tastes of the individual that directed them.” 

The article highlights one additional value from making these short films.   It enables Pixar to develop and evaluate its talent.  Providing young directors and animators an opportunity to create a short film is a relatively inexpensive way to see who might be ready to work on a feature film.  Moreover, it enables Pixar to grow and develop those high potentials.   They learn by doing in an environment in which they can take some risks.  Every firm should be looking for opportunities to experiment as a means of fostering innovation, but they should also be examining how these experiments can be a useful means of nurturing and testing younger talent. 

Ed Catmull: Let Your Ideas Suck

Fast Company has a great feature describing Pixar President Ed Catmull's views on innovation and creativity.  I had the chance to interview Catmull several months ago, and he has a terrific perspective on how to develop a highly creative work environment.   Here's an excerpt from the Fast Company article:

Let Your Ideas Suck

Pixar movies have multi-layered, compelling stories and are beautifully put together, but they don’t start that way. Catmull shared the process that the beloved movies go though, starting with a story that bears no resemblance to the final product. He said, "All that anyone sees is the final product and there’s almost a romantic illusion about how you got there. When we first put up something—these stories suck." For example, he shared that the first version of the movie Up included a king in a castle in the clouds. They threw everything out from that first idea except a bird and the word "up," from there it went through several other iterations with a little more of the final story emerging each time. They had to make a lot of mistakes and have a lot of failures along the way to get the final product, he said.

Speaking of Pixar, they have released the trailer for their upcoming movie, Inside Out.  It sounds like it could be another hit (14 Pixar movies in a row have achieved #1 status at the box office). 

Disney*Pixar's Inside Out - Official Trailer by FanReviews

The Braintrust at Pixar

In this Fast Company excerpt from his book, Creativity, Inc., Pixar President Ed Catmull describes the company's "Braintrust."   Catmull explains the concept of the Braintrust: 

"The Braintrust meets every few months or so to assess each movie we're making. Its premise is simple: Put smart, passionate people in a room together, charge them with identifying and solving problems, and encourage them to be candid. The Braintrust is not foolproof, but when we get it right, the results are phenomenal."

The Braintrust includes some of the best and brightest folks at Pixar.   The core value at the heart of this feedback process is candor.  As Catmull explains, "Believe me, you don't want to be at a company where there is more candor in the hallways than in the rooms where fundamental ideas or policy are being hashed out."

What's interesting about the Braintrust, however, is that these people do not seek to dictate to a movie director what to do.    They share ideas with the director so as to "bring true causes of problems to the surface."   They do not advocate particular solutions.   Their feedback is designed to stimulate the director's thinking, and perhaps to encourage him or her to think differently about an aspect of the film.   The director must then take that feedback and find a way to address to the issues that have been raised. 

Too often, I think that people who are offering feedback and criticism do not follow this path.  They advocate their preferred solution.   Recipients of the feedback become defensive, and the entire conversation takes on a very negative tone.  The Pixar process tries to avoid that fate.

Pixar Storytelling Rules: One Rule Could Help You Make Better Decisions?

I found this terrific list of Pixar storytelling rules the other day.  I encourage all my readers to take a look.  The lessons apply to all storytelling, not just the development of animated movies.  Since great leaders tell stories to communicate key messages, the rules are a must-read for business executives. 

I'm particularly interested in Rule #9: When you’re stuck, make a list of what WOULDN’T happen next. Lots of times the material to get you unstuck will show up.

Think of a tough decision you have had to make recently, one characterized by high stakes and a great deal of ambiguity.   Were you stuck at some point, unsure of how to proceed?  If so, you might try applying this rule.  Make a list of the options you would clearly NOT pursue.  Ask yourself why.   Developing the options that are clearly not plausible or attractive may help you generate alternatives that do make sense.  

Hire for Excellence... in any field

Fast Company has a terrific interview with Oren Jacob, former Chief Technical Officer at Pixar.  Jacob describes some of the key attributes of the Pixar culture and processes.   Among the many great tidbits, Jacob explains a crucial facet of the Pixar hiring process.  He calls it "hire for excellence." Here's the explanation:

When Pixar is evaluating potential hires they look for three traits: humor, the ability to tell a story, and an example of excellence. These aren’t unique qualities to assess in applicants, but how excellence is defined is not that common. It doesn’t matter what you are excellent at, just that you have reached a level of excellence. It’s important that you know what excellence feels like and what it takes to achieve it. It could be gardening, jujitsu, or cooking. The main thing is you’ve had a taste of excellence and will know how to get there again.

I really like the concept of hiring for excellence.   Developing mastery in a particular field or endeavor demonstrates self-discipline, work ethic, and perseverance.   It also means that individuals have probably engaged in the kind of deliberate practice that is required to excel.  Familiarity with that process of learning and development will suit them well when they have to develop new skills and capabilities.   Finding people who have achieved excellence in a range of endeavors also creates a workforce with incredibly diverse perspectives and experiences that they can bring to bear on any problem.