Course blog

http://www.theatlantic.com/technology/archive/2014/01/how-netflix-reverse-engineered-hollywood/282679/

http://www.wired.com/2014/08/i-liked-everything-i-saw-on-facebook-for-two-days-heres-what-it-did-to-me/

A few years ago, I remember scrolling through Netflix and finding a few movies that looked slightly interesting. Now, I have about 5-6 movies on my “to-watch” list at all times. Netflix, the online movie and TV subscription service, currently has over 50 million subscribers globally. In 2006 they had 10 million subscribers. It’s clear that Netflix has grown to be one of the top web apps to date. The question is how they did it – did they improve their movie selection, or did they improve their movie selection suggestions?

We can assume that Netflix did both. However, the more important development was the suggestions. Alexis Madrigal spent several weeks with a group of coworkers pulling apart Netflix’s magic. According to their results presented in the article “How Netflix Reverse Engineered Hollywood”, Netflix has 76,897 unique ways to describe movies. It all boils down to tags; Netflix has analyzed and tagged every movie and TV show in every possible way imaginable. “When these tags are combined with millions of users viewing habits, they become Netflix’s competitive advantage. The company’s main goal as a business is to gain and retain subscribers. And the genres that it displays to people are a key part of that strategy”.

There is only one other similar intelligence approach in existence. “Netflix has built a system that really only has one analog in the tech world: Facebook’s NewsFeed”. Both Netflix and Facebook’s NewsFeed operate under the users viewing and liking habits – depending on how you interact with Netflix or Facebook, their algorithms spit out other certain movies or links you might be interested in. But what happens if you mess with this algorithm and like everything?

Mat Honan did. In “I liked Everything I saw on Facebook for Two Days. Here’s What It Did to Me” featured on Wired, Honan explains that after his Like binge, his Facebook NewsFeed became extremely liberal and extremely conservative. None of his friends popped up anymore; it was all advertisements and articles. “As I went to bed that first night and scrolled through my NewsFeed, the updates I a saw were (in order): Huffington Post, Upworthy, Huffington Post, Upworthy, a Levi’s ad, Space.com, Huffington Post, Upworthy, The Verge, Huffington Post, Space.com, Upworthy, Space.com.”

So I did the same thing, but with Netflix and only for 20 minutes. I went to “Personalize” and 5-starred and clicked “interested” for every movie that popped up. On Netflix’s splash page I did the same thing. The difference between Honan and my result’s was that Netflix recycled the movies I already 5-starred. They had no other movies to suggest, no other reserves to pull from. Facebook, on the other hand, has the entire internet to suggest to you. Which ends with a clogged NewsFeed full of stuff that in the end, Honan didn’t like anyways.

Algorithms do an amazing job of feeding consumers stuff that they’d be interested in. But they can be screwed with, and that’s because in the end, they’re robots trying to cater to humans.

After reading the article “How Netflix Reverse Engineered Hollywood” I couldn’t be more intrigued. It was always nice that Netflix would recommend movies to me but I never really put too much thought into it. I would say nine times out of ten, there is a recommendation for me waiting when I finish my series or movie that I’ll actually watch. For example, (and please don’t judge me for my taste in television here), I had just finished watching One Tree Hill when Netflix recommended that I watch Desperate Housewives. These series are so completely different in terms of what they’re about, the age demographic, the setting, etc. but Netflix knew based on who else watched it and everything else that I had watched that I would love it too! And now, thanks to Alexis Madrigal, we know that this is because of the absurd amounts of categorization, metadata and refined vocabulary that Netflix uses.

For me personally, I found it interesting that the categorizations were so specific. It’s hard to fathom that there were people that went through all of the movies available on Netflix to tag them with enormous amounts of metadata – and some of the categories are so oddly specific. This got me thinking about Pandora. Similar to Netflix, Pandora uses what you like/ have listened to in order to recommend new music. Although there are differences – Pandora is music, Netflix is film, Pandora uses “thumbs up” and Netflix uses stars for ratings, I have to imagine that Pandora must have a similarly specific metadata process it uses in order to produce music that people will enjoy. Like Madrigal said, “The better Netflix shows that it knows you, the likelier you are to stick around.” And this is true for Pandora as well. If I put on a station that gives me five songs in a row that I don’t really care for, I’m going to use a different site to listen to music. However, the main difference between these two sites is probably that Pandora is free while Netflix costs about eight dollars per month. Because of this, it would be more detrimental for Netflix to lose its customers. Of course Pandora wants to be a successful site (which it already is) but they don’t have a tangible loss when people stop listening to them.

 

Madrigal, Alexis C. “How Netflix Reverse Engineered Hollywood.” The Atlantic. Atlantic Media Company, 02 Jan. 2014. Web. 20 Oct. 2014.

This was a very comprehensive article that triggered a couple examples of other generators I encountered after reading it. Near the beginning of his piece, Madrigal had an interesting point about tracking the URL and its incremental values at the end of the web address. I’ve also been able to navigate between pages using that same line of logic. It strikes me that for some databases, a lot is revealed to the public, while others are much more privatized depending on how the system was created in the first place. Once you discover the thought process behind the ways some things are categorized, you can easily find what you’re looking for in a general sense.

After playing with the generator for a few minutes, I started to wonder if any director will rise up to the challenge and actually create a movie based on the results of this generator. Maybe that way, we’ll be able to see more original movies. I’ve always believed in the notion that people can do great things once they’re given some limitations. It’s an intriguing thought that something original can be made based on unoriginal words, descriptions, and genres produced by algorithms. This truly showcases the wide range of possibilities that any given combination can produce.

I also love this quote from the article, “It’s where the human intelligence of the taggers gets combined with the machine intelligence of the algorithms. There’s something in the Netflix personalized genres that I think we can tell is not fully human, but is revealing in a way that humans alone might not be.” In a very digital humanitarian sense, this project was able to produce many eye-opening graphs that gives the public an inside look to what types of things human beings prefer just by analyzing and presenting the data a different way. Instead of being recommended different genres and searching through them, we are now able to generate our own and the results that pop up says a lot about our diverse preferences and creativity of past directors that have shaped the movie industry.

Because of how many times Spotify was mentioned in the class, the first thing I did after reading the article was to google for a ‘Spotify based music genre generator’. What I found was this playlist generator site linked to Spotify that lets you search for playlists that were made from other users and were categorized by either mood or genre. It’s unfortunate that you can’t search for both simultaneously, but when creating a playlist, you can tag descriptions under both types of categories. Another site that I encountered was a more minimalist music genre generator that operated under a similar idea as the Netflix one in that it combined a couple of descriptive words from a database to create new music genres. Lastly, I found an actual site that lets you generate your own generators. Even though there was a user-created movie genre generator, it only allowed you to mash together two random genres and I’m willing to bet that the database it was pulling the genres from is a lot less detailed than the one Yellin created under Netflix.

Sites Used in Order of Appearance:

http://www.theatlantic.com/technology/archive/2014/01/how-netflix-reverse-engineered-hollywood/282679/6/

http://playlists.net/

http://jbdowse.com/genres

http://www.generatorland.com/

Bowker and Star continue to expand upon a topic we began discussing last week, classification. They discuss means of classification as a “spatial, temporal, or spatio-temporal segmentation of the world”. In lecture we went over many classification and organizational systems that are currently in use, from library classifications to those used in social medias. These exist to make data retrieval easier, they create ways to expedite retrieval by categorizing information into relevant subgroups. We have been discussing so many useful and innovative systems of classifying observations and information that I want to bring in a very specific, and now obsolete, classification system subset related to how crania appear: phrenology.

Phrenology hails back to late 18^th century Germany, springing from the observations of physician Franz Joseph Gall. Now considered a pseudo-science, phrenology was invented to classify specific physiological features of the skull as belonging to specific characteristics or faculties. There were 27 faculties that Gall identified, ranging from reproductive instincts and the love of one’s offspring to murderous instincts and metaphysics. Franz Joseph Gall’s classification of specific traits within this range of physical contours in the cranium is an example of how even if something can be classified or has already been classified that does not necessarily lend truthfulness its subject. With that said, the phrenology classification can be seen as a precursor to psychology and its physiological correlations.

In the more modern era, we think of phrenology as a funny side note in medical and scientific history. It seems laughable now that something as arbitrary and even changeable as the lumps and indentations on a head could denote specific personalities or behavioral tendencies. Had there been a better reception of phrenology, or perhaps more scientific evidence, this system could have become a standardized method of evaluating personalities. These shortcomings on a scientific scope are tied to its shortcomings as a classification system as Bowker and Star describe it. It is interesting to me that even though Bowker and Star lay down three qualities for a classification system to have (consistent unique classificatory principles, mutually exclusive categories, a complete system) they only exist in a theoretical ideal setting. Phrenological studies identify different faculties, categories of behavior and personality, but they are difficult to quantify on their own; it becomes difficult to say with certainty that a specific node on the head is of a size to denote that a person has an aptitude for education. This lack of ability to definitively categorize what size bump or indentation in the skull leads to personality traits, combined with pseudo-scientific reasoning (this being the biggest reason) that ultimately makes phrenology an idea of the past.

 

You can learn more about phrenology from books that UCLA currently has in its collection (and you get to see the Library of Congress classification system in action!). The Biomed History and Special Collections Cage has a few really old books on the subject! See here and here

There is seemingly an infinite amount of ways to categorize information. In most cases, classification is biased towards an individual or a cultural preference, giving priority to the social norms of each respective community. However, in our digitized world, cultures lacking modern technology are deemed incapable of making these classification decisions, and this notion incorrectly contributes to the ever-growing digital divide. This idea unethically promotes a division between cultures who have access to new technology, and those who don’t. We must understand that in some cases, modern, digital technologies aren’t relevant or useful in certain culture’s practices.

Modern technology is thought to provide a more efficient and powerful way to accomplish tasks, but as we investigate cultures and their practices, we find that new digital and technological advances aren’t always the most practical. Last Winter, I took a class taught by Professor Ramesh Srinivasan in the Information Studies Department. We thoroughly discussed the concept of the digital divide and how cultural appropriation in regards to promoting new technologies must be a slow and adaptive process, as different  communities stress different needs. One article in particular struck me, as it fought the urge to close the gap the digital divide is apparently creating. A sociotechnical experiment took place in India in the early 2000s that documented the results of a technology influx: several computers were setup in a rural farming village in Southern India with the expectation for residents to become educated on the various modern technologies in hopes of bringing the community up to speed on the digital age. Instead of appropriating the new technology into their lives for the benefit of their community and economic infrastructure, the children were seen playing video games and creating an unnecessarily competitive environment that took away from their studies and daily chores. This agricultural village had no apparent need for the technology, nor did they understand how it could provide a benefit to their community, as they were very comfortable in their way of doing things.

What is important to note, is that the individuals who provided the technology were only present for the setup and removal of the devices, and didn’t make themselves available to facilitate the usage of the computers. This is ethical in terms of cultural appropriation standards, as it allowed the community to learn for themselves how to use the devices, as exemplified in the success of the Plateau Peoples’ Web Portal. However, introducing such a foreign object into the community requires more mentoring and technical assistance than there was provided. With the assistance from computer experts, this community could’ve benefitted from learning how to track weather patterns and the prices of goods to help their agricultural economy improve. There is a balance between introducing and enforcing knowledge on other cultures, and with the abundance of new technologies, we must be careful in the ways in which these devices are presented in order to maintain the unique practices and the integrity of cultures around the world.

After reading Alexis Madrigal’s article, “How Netflix Reverse Engineered Hollywood,” it made me even more interested in the human work and technological software used to classify, or in this case, microtag information on the internet. I did not know what tagging or micotagging data on the Internet consisted of. The thing that interested me the most was that Netflix altered the system of tagging and microtagging by going deeper into the content of the movies featured on the website. In order to gain more content-based information and make the Netflix experience personal, Netflix’s Vice President of Product, Todd Yellin, and a Netflix crew used a mix of human and machine intelligence to create the Netflix Quantum Theory system of tagging movies and shows. The idea of combining both human and machine intelligence for a more powerful system reminds me of the book, The Singularity Is Near, written by computer scientist, futurist, and inventor, Ray Kurzweil.

Ray Kurzweil is a futurist, or in his terms, a Singulatarian. In his book he explains this theory of the Singularity. The Singularity is Near defines the term Singularity as “a future period during which the pace of technological change will be so rapid, its impact so deep, that human life will be irreversibly transformed.” Kurzweil’s idea of the Singularity emphasizes the emergence of transhumanism through technological advancements in society, which will create such immense progress in technology that it will lead to the transcendence of humanity to a post-human race.

Obviously the Netflix Quantum Theory is not as extreme as Ray Kurzweil’s theory of the Singularity; however, Yellin’s system of combining human and machine intelligence is comparable to Kurzweil’s theory of transformation from a human intelligent society into a machine intelligent society. Kurzweil thinks in order to become the most intelligent and successful society that the human race must combine with machine technology and then later transform to a post-human machine intelligent society. Technology like Netflix’s Quantum Theory system is constantly improving and becoming more personable to the people using these technologic devices. Yes, Netflix is not capable of taking over the human race, but the Quantum Theory is learning to ‘outsmart’ consumers by categorizing information of consumer interests in a way that manipulates the consumers to continue to watch Netflix movies and shows. After gaining an understanding for Netflix’s Quantum Theory, Madrigal even said, “But if Netflix’s system didn’t already exist, most people would probably say that it couldn’t exist either.” Who knows what boundaries of technology, in this case, categorizing and tagging data will be broken? A tagging system like Netflix has just added to this whirlwind of out-of-the-box yet logical systems and can only improve from the systems that exist today.

Works cited:
Kurzweil, Ray. The Singularity Is Near: When Humans Transcend Biology. New York: Penguin Group, 2005. Print.

After reading Alexis C. Madrigal’s article “How Netflix Reverse Engineered Hollywood”, I began to think about how well the Internet knows us. Madrigal, in her article, describes how she uncovered Netflix’s 76,897 unique movie genres, which, initially, does not appear to have much significance besides a good laugh over the ridiculous genre titles. However, this discovery provides insight into how Netflix uses their “hybrid human and machine intelligence approach” to provide movie suggestions to users based on the tags attached to each movie in his or her viewing history. Madrigal explains, “the underlying tagging data isn’t just used to create genres, but also to increase the level of personalization in all the movies a user is shown. So, if Netflix knows you love Action Adventure movies with high romantic ratings (on their 1-5 scale), it might show you that kind of movie, without ever saying, “Romantic Action Adventure Movies.” I know that I, along with every other Netflix binger, have fallen victim to their recommendation feature—It is simply too hard to pass up.

 

Soon after reading Madrigal’s article, I began to realize that a majority of the websites and apps that I use on a daily basis attempt to manipulate their users in similar ways. Facebook, Instagram, and iTunes all employ specific methods to recommend certain products to their users. On Instagram, there are two different methods. First, after following a high profile page, immediately a tab drops down recommending three other possible pages that you might be interested in. The other method is on the explorer page, where they display pictures based on who your friends follow, or pictures you’ve previously viewed or liked. iTunes recommends music in similar ways; at the bottom of every page there are “Listeners Also Bought” and “Genius Recommendation” sections, which suggest music based on your purchase history.

Facebook, comparably, recommends pages to follow but it also utilizes cookies as well. Cookies are “a small piece of data sent from a website and stored in a user’s web browser while the user is browsing that website. Every time the user loads the website, the browser sends the cookie back to the server to notify the website of the user’s previous activity (Wikipedia). The usage of cookies is easily noticeable by the advertisements displayed on your Facebook Newsfeed. I’ve always noticed that after I browse certain websites, my feed suddenly fills up with advertisements from that particular website and those similar to it. It is crazy to think just how well the Internet knows you.

Works Cited:

• Alexis C. Madrigal, “How Netflix Reverse Engineered Hollywood,” The Atlantic, January 2, 2014
• http://en.wikipedia.org/wiki/HTTP_cookie

 

 

 

There is a consistent theme in many articles relating to digital humanities: metadata is important, and good categorization of information is essential for a digital database, website or exhibit to be functional. An ontology is a “formal framework for representing knowledge…and that framework names and defines the types, properties and interrelationships of the entities in a domain of discourse.” (Wikipedia).

In Local-Global, J. Wallack and R. Srinivasan highlight the importance of intersecting thought-out ontologies with information systems. Ontologies that are mismatched “impede communities’ ability to impart and communicate information and states’ ability to fully understand the territories they govern.” (link to the article)

Although ontologies are meant to represent some sort of reality, they can also be used to shape new realities. If properly designed and executed, these information systems can serve as incredible tools. They can, and should be used for better city planning (as discussed in J. Wallack and R. Srinivasan’s article), better health care systems, etc. The opportunities are endless, but it takes intersecting human thought and decision making with the power of digital tools. The authors of Local-Global write about improvements that need to be made to information systems, but some success stories already exist. Pandora created the still unsurpassed music library, in my opinion, by building the music genome project and Netflix created an online movie library. Both systems learn your preferences and tailor a unique experience. They have an extensive vocabulary and grammar system to categorize and describe their content, and follow impressive algorithms written to learn about the user. For both of these systems to be successful though, it took a perfect marriage of human intuition and decision making with the searching/learning/sorting functions of digital tools. I haven’t really heard anyone describe this better than A. Madrigal, in How Netflix Reverse Engineered Hollywood: “to me, that’s the key step: It’s where the human intelligence of the taggers gets combined with the machine intelligence of the algorithms.”

I extrapolate this “perfect marriage” to apply to the interaction of humans and technology in general, not just in information systems. Right now, wearable technology is growing in popularity and market size. There are tons of wearables already on the market, and new ones continue to emerge: Forbes covered Microsoft’s announcement of a wearable expected to be released this holiday season, which got a lot of attention. The “coming-soon” wearable that caught my attention, though, was will.i.am’s new PULS. Wearables are remarkable new pieces of technology—not only do they incorporate many of the same functions as your smart phone, but they also serve as metadata collectors. (PULS reportedly can even read your emotions!)

Mostly I have been most interested in the fitness and health trackers included in these devices. Wearables collect all kinds of information about you—your sleep habits, your daily activity and levels of exertion, among many others—then presents that information back to you in a way that can shape your decisions and future behavior. These devices connect humans to technology in a newly involved way. And, although I have been impressed and interested in all of the functions of these types of wearables, I have resisted entering the market. For some reason the watches (or ‘cuffs’ as will.i.am calls his new PULS devices) did not seem “human” enough. To me, they were all ugly and clunky, and certainly did not serve as a fashion statement. This is why will.i.am’s new PULS campaign caught my attention.

Human elements need to be incorporated in our development of new technology and information systems—and Pandora and Netflix serve as great testimonials. Will.i.am and his new brand FASHIONOLOGY believe something similar: that it is “inevitable that fashion and technology will come together”. People like me have been hesitant to enter the wearable market because it lacked a certain human element, fashionable design. In a recent press conference, former Vogue editor Andre Leon Talley stated: “it doesn’t matter if a gadget can organize my life and make my dinner, if it’s ugly to look at”, and I couldn’t agree more. He insists that closer collaboration between fashion and technology is urgently required—a collaboration between humans and technology that I think should be extended to all aspects of the digital humanities.

Check out will.i.am’s promotional video for his wearable cuff. His new brand i.am/FASHIONOLGY seeks to take the wearable market mainstream, and once you see the video it’s hard to resist the movement.

After completing the reading, I initially found Madrigal’s Netflix quest to be daunting and failed to see the significance behind her efforts.  Simply put, Alexis Madrigal, with the aid of others, discovered that Netflix possessed 76,897 unique genres on their website.  As a result, this indicated how precise and descriptive the teams of taggers at Netflix truly were.  This high level of specificity has allowed Netflix to accurately provide suggestions to subscribers of what to watch based on their history.

 

I soon realized just how remarkable this complex and interrelated system was by comparing it to the densely-packed world wide web.  Michael Stevens from Vsauce goes into detail about the origins of the web and how it connects various sources through a nonlinear fashion.  This relates to the Netflix article in some sense because all of the tags that are created are linked to one another in some level.  As Madrigal describes, “every movie gets a romance rating, not just the ones labeled ‘romantic’ in the personalized genres.”  Furthermore, every movie’s plot is tagged, as well as the job of the actors and the locations.  Thus, all of the movies have a degree of similarity that they share and continue to make the system evermore complex.

 

During the beginning of the internet, information was organized illogically through a hierarchical method.  As explained by Michael, it was not until Tim Berners-Lee sought to change how information was connected to one another by writing a proposal.  In his Information Management: A Proposal, Tim desired a structure that would allow information to develop and evolve and reduce information loss.  He continued to argue that by having “web” of notes with links between information would be far more efficient than the fixed hierarchical system that was present at that time.  In other words, documents would be connected to one another through nonlinear ways, known as hypertexts, which would ultimately allow unification between the web and the internet.

 

The way this all ties into the article regarding Netflix is by acknowledging how intricate the system or organization has become.    What started as simple relationships between information, or in Netflix’s case, tags, has developed into vast webs that have evolved through continual ingestion of new data and algorithms.  In the Netflix article, Todd Yellin, VP of product management, discusses the way in which these instances of unexplainable occurrences makes life interesting by serendipity.  He even states, “The more complexity you add to a machine world, you’re adding serendipity that you couldn’t imagine.”  To think that in a digital world of 1’s and 0’s there can still be surprising elements that cannot be entirely foreseen is in a sense quite beautiful.  Whether a bug or a feature, as both Todd and Madrigal described, these imperfections contribute to an intricately dense system, thus producing an element of surprise and excitement to an often-perceived realm of rigid analytics.

 

Work Citied:

1. Alexis C. Madrigal, “How Netflix Reverse Engineered Hollywood,” The Atlantic, January 2, 2014

2. http://www.w3.org/History/1989/proposal.html

3. Video Provided in Post

When reading this week’s article about Netflix’s use of metadata and use of categorizing genres, I was struck by the author’s question: “How do you systematically dismember thousands of movies using a bunch of different people who all need to have the same understanding of what a given microtag means?” This inquiry took me back to our discussion in class where we talked about how assigning a category to something implies a belief about that item or an ideology about the world that may not be universally held. If it were up to the viewers to assign the categories, their differences in perspective would therefore yield different interpretations of what the genres should be. The way Netflix was able to address this problem was through establishing a systemized rating system for different parts of movies; in other words, turning to the actual content to speak for itself when choosing a label for it. In this way, many different tangible parts of the movie came together to create a single, specific, and coherent genre for itself. By allowing the content itself to create the categories, the possibility of introspection (when genres tell the viewer not just what they would like, but what kind of things they would like) becomes possible and adds more to the viewer’s discovery of not just movies, but himself in general.

This introspection reminded me of Pinterest’s use of its “Guided Search” feature, described in the article “Pinterest puts metadata to good use with Guided Search” (http://www.techtimes.com/articles/6081/20140425/pinterest-puts-metadata-to-good-use-with-guided-search.htm ). Basically, the system uses user-generated metadata from the titles, comments, and descriptions made on individual pins to classify it with sub-categories that pop up when a user makes a broad search, allowing him or her to choose a more specific search within the broad category if he or she chooses. This use of metadata derived from the actual content of the pin allows users to stumble across subcategories that are actually pertinent to them, instead of being confined to only the website’s broad thirty-two categories. The more tailored your search, the more the system can detect the user’s specific likes, and thus make more suggestions to material it knows they will like. Similar to Netflix, this process also displays introspection in that it shows the user what kind of things they like, not just what they like. This reliance on content to complete the digital categorization of a topic mirrors that of the field of Digital Humanities in general. Our job is to unite human created content with technologically created classification systems to enhance the way we discover, view, and analyze information.