The Napoleon Dynamite problem
This is an old (If You Liked This, You’re Sure to Love That, Nov ‘08) but good article from the New York Times that I read parts of online, but just read in its entirety in the NYT print version. Very interesting discussion of the ability of computers to predict our movie preferences, especially in the cases where computers seem to be able to discern some sort of hidden pattern, opaque to human interpretation, behind our preferences.
Author Clive Thompson writes:
But many categorizations are now so obscure that they cannot see the reasoning behind them. Possibly the algorithms are finding connections so deep and subconscious that customers themselves wouldn’t even recognize them. At one point, Chabbert showed me a list of movies that his algorithm had discovered share some ineffable similarity; it includes a historical movie, “Joan of Arc,” a wrestling video, “W.W.E.: SummerSlam 2004,” the comedy “It Had to Be You” and a version of Charles Dickens’s “Bleak House.” For the life of me, I can’t figure out what possible connection they have, but Chabbert assures me that this singular value decomposition scored 4 percent higher than Cinematch — so it must be doing something right.
I would love to read Hubert Dreyfus’ response to this.