Great talk! 💯 I'm sure has an amazing data science team, but in a lot of companies, I think A/B testing causes teams to focus on what's easy to measure, and that doesn't necessarily correlates with what the person using the product is trying to achieve
Cool talk… Here is the thing, I know great designs model real world situations best way they can. Netflix also does that… In my country, movie shops ALWAYS display LATEST MOVIE POSTERS internally on the WALLS to let customers know the type of content sold in the shop <Movies> Example link : https://www.google.com/search?q=Kenyan+movie+shops&client=opera&hs=ZIo&source=lnms&tbm=isch&sa=X&ved=0ahUKEwjLgp6blobkAhXm4IUKHfunChYQ_AUIESgB&biw=1457&bih=720#imgrc=qZz9J-RarlofuM:
People watching more content on TV is just coz of the display size of it…. (movies are better on a larger screen with external speakers…) >>> especially since netflix helps most people pick what to watch so well…. I would want to watch good content with someone… watching on a small screen doesnt make sense, and we all don`t feel the need for the strain / dockers.
Humans are social, and we love sharing good things/ the best of us with others… I want to watch a good movie at night with a friend…. <chillin>
Users had indicated that they'd like to know what content they can get before they decide whether or not to get the subscription. So the measure of success here should be that it leads to better decisions, not more subscriptions.
So the users were not wrong to mention that in the survey.
It is important to differentiate that "Improving user experience in decision making" is different from "maximizing subscription rate" are 2 different hypothesis.
Navin's null hypothesis was "A/B testing is the best testing for experimentation". He should have designed a between subject comparison using T test to prove his null hypothesis. After he skipped the evidence by just making this statement, now I don't feel like watching the rest of the video.