Which Music Do We Prefer? Interesting New Research

https://freesvg.org/

We know that Spotify, Apple, and others have developed sophisticated algorithms to recommend music to listeners. While many customers know precisely what they would like to listen to, others are open to discovery. Spotify distinguishes between those listeners with a "closed" vs. "open" mindset. The algorithms are especially important for those with an open mindset. 

Now we have a new academic study that suggests an interesting way to predict the music we will enjoy. David Greenberg, Sandra Matz, Andrew Schwartz, and Kai Fricke have published a paper titled, "The Self-Congruity Effect of Music" in the Journal of Personality and Social Psychology.  They write: 

Across three studies we show that people prefer the music of artists who have publicly observable personalities (“personas”) similar to their own personality traits (the “self-congruity effect of music”)... Our findings are largely consistent across two methodological approaches to operationalizing an artist’s public personality: (a) the public personality as reported by the artist’s fans, and (b) the public personality as predicted by machine learning on the basis of the artist’s lyrics.

The scholars also show powerful evidence of a gender effect in music preferences.  They write:

The present article also provides the first evidence that listeners tend to prefer music from artists of the same sex, which had been previously theorized but not studied empirically (Hagen & Bryant, 2003). That is, the gender-fit between the listener and artist was found to be a significant predictor of musical preferences.

So, the next time you say yourself, "I love that artist's music."  Ask yourself, what do you know about that performer?  Why do you like that artist?   For companies, of course, the implications are profound.  Can they use this finding to refine their algorithms, without intruding on our personal privacy?  What if listeners were willing to disclose more about their personality in order to discover more music they would enjoy?   

Algorithm Aversion: How Can We Overcome It?

Algorithms can help us make better decisions in a variety of situations.  However, human beings tend to have an aversion to using algorithms.  They trust their gut more than the computer, even though the algorithms may lead to better decisions. Knowledge@Wharton reports on a stream of fascinating research by Cade Massey, Joseph Simmons, and Berkeley J. Dietvorst.  They found that you can persuade people to use algorithms if you give them a choice as to whether to use the algorithm or not.  In other words, don't force them to use it; make them feel a sense of control.  That will help convince them to choose the algorithm.  However, many people stop using the algorithm after some period of time, because they become frustrated with the mistakes that the computer makes.  Of course, the computer might make fewer mistakes than a human using intuition, but people don't recognize that possibility. Instead, they fixate on the mistakes and lose faith in the algorithm. Simmons points out, "People want algorithms to be perfect and expect them to be perfect, even though what we really want is for them to simply be a little better than the humans."

The scholars also found that you could persuade people to use the algorithms if you gave them an ability to adjust the computer's recommendation slightly.   Of course, the algorithm's predictions and recommendations become less accurate when humans intervene in this manner.  However, the researchers found that you only have to give people an ability to adjust the algorithm slightly to enhance adoption.   Providing them an ability to adjust more substantially does not increase adoption more than offering a slight adjustment possibility.  Thus, you might be willing to tolerate a bit of degradation in the algorithm's accuracy simply because giving people some sense of control increases adoption of the computer-assisted decision-making system.  For more on this research, see the video below in which Massey and Simmons are interviewed about the research.  

Netflix: Geography, Age, Gender Not Good Predictors

David Morris wrote an article this week for Fortune titled, "Netflix: Geography, Age, and Gender are 'Garbage' for Predicting Taste." Morris writes, 

"'Geography, age, and gender? We put that in the garbage heap,' VP of product Todd Yellin said. Instead, viewers are grouped into “clusters” almost exclusively by common taste, and their Netflix homepages highlight the relatively small slice of content that matches their taste profile. Those profiles could be the same for someone in New Orleans as someone in New Delhi (though they would likely have access to very different libraries)."

Why is this statement so fascinating?  To me, it speaks directly to Netflix's competitive advantage.  If geography, age, and gender were, in fact, accurate predictors of viewers' preferences, then Netflix would have a far less formidable advantage over rivals.  Why?  Well, those variables are easy to identify and measure.  Others can get access to that data quite easily and build predictive algorithms using that information.  However, if more accurate predictive algorithms involve data that are not as publicly available, then Netflix has a key advantage.  In other words, if the predictive power rests with variables that come from proprietary data that Netflix has collected about us, then the sustainability of Netflix's advantage over competitors rises substantially.  The same holds true for any company trying to take advantage of "big data" to develop predictive algorithms.   The key is to unearth variables that matter, but hopefully, variables that are not easily identified and measured by others.  

Qwikster gone rather quickly

Netflix has abandoned its plans to separate its DVD by mail service from its streaming business.  Qwikster is dead.  The stunning twists and turns in Netflix's strategy have left most of us dizzy.  The collapse of the stock price in recent months proves that investors don't like uncertainty.  While it's ok to change strategy, investors do not want to see constant twists, turns, and reversals. 

Beyond the uncertainty, I was never quite clear regarding the notion that Netflix and Qwikster would not share information regarding a customer's queue, movie preferences, recommendations, and rental history.  That lack of sharing made me wonder whether Netflix was failing to capitalize on one of its greatest strengths, namely its powerful predictive algorithms that it uses to recommend movies.  Would lack of sharing across sites mean that it would not capitalize on the wealth of data that it had accumulated?   It wasn't clear based on what I had read.  To me, the predictive algorithms lie at the heart of Netflix's success, and no change in strategy should undermine that strength.   After all, the algorithms enabled Netflix to take advantage of the "Long Tail Effect" - the idea that a large percentage of Netflix rentals always came from movies that were not new releases.   That strategy proved very profitable over the years.
 

Progessive: Usage-Based Car Insurance Pricing

Progressive has been a leading innovator in the auto insurance market for many years.  Twenty years ago, it made its mark by developing sophisticated algorithms to price policies more effectively for high risk drivers (think of it as the subprime segment of the car insurance market).   Progressive aimed to cull the best risks out of the high risk pool, leaving the worst risks to their rivals.  They became a very profitable company with a strategy focused on the higher risk drivers.   Since then, they have expanded their reach to a broader set of consumers, but they still aim to stay ahead of the competition by having finer-grained methods for evaluating and pricing risk.  

Progressive now has developed an onboard diagnostic device called Snapshot that can provide Progressive a real-time driving report, including the number and time of miles driven, incidents of hard braking or quick acceleration, and speed.   Progressive aims to use the data to provide discounts to those drivers with the best evidence on good driving practices from its Snapshot database.  Naturally, some people have expressed privacy concerns about such devices.  However, for Progressive, Snapshot provides them another way to price risk more accurately than the competition.   To this point, auto insurers have tried to estimate an individual's riskiness based on demographic characteristics, driving record, etc.   Now, Snapshot offers a way to personalize the pricing of risk in a way that has not been done previously.  Eventually, with enough data, Progressive will build even more sophisticated algorithms that connect the frequency rate of incidents such as "hard braking" or "quick acceleration" to the probability of an accident.  

For many companies, predictive algorithms have become a source of competitive advantage (think NetFlix and Amazon, to name a few).   For all these firms, the more personalized the predictive power, the more that they can derive a competitive advantage.   For NetFlix, it's been easier to personalize than for a firm like Progressive, but technology may be changing that now. 

Using Twitter to Make Money on Wall Street

The USA Today ran a feature story today on investment firms that are trying to mine Twitter for insight as to how the market will move, and thereby improve investment returns.   What's the logic here?  Apparently, some experts have found that rigorous analysis of tweets can yield insight as to people's emotional state.  Experts then believe those emotions drive investment behavior.

These investment firms cite the research conducted at Indiana University last year by Johan Bollen, a professor of informatics.   He found a correlation between the collective mood, as determined by an analysis of millions of tweets, and the movement of the Dow Jones average in subsequent days.  Bollen reports an 87% accuracy rate for his algorithms which use Twitter mood measurements to predict the DJIA over the next 3-4 days. Other research focuses on specific companies.  Arthur O'Connor, a doctoral candidate at Pace University, has performed a research study which found a positive correlation between social media popularity of major brand names and the performance of those firms' stock prices.

While one might doubt the findings of these particular studies, the overall trend bears watching.  More and more investors will try to glean insights from this abundance of data that is available online.  Some algorithms will be better than others, but ignoring the data altogether surely cannot make sense.    Information is power in the investment community, and social media does provide a great deal of data that may be relevant.  The key question is how to mine that dataset most effectively, and then how to build the best predictive algorithms.