One way to solve this could be to break the network up into clusters of users who follow each other, and perform the random assignment at the cluster level. From a user perspective, if a user could search and invite their friend within Spotify, but their friend was in the control group and couldn’t see the invitation, that is going to disrupt their experience and impact the way they use the feature. As such, if we were to run this as a traditional AB test, with each user as the randomisation unit, then we’d be violating SUTVA. Since interaction between multiple users is critical to the feature, the way one user engages with the feature is not independent of the way another user engages with it. Being a data driven PM, we’d like to run this as an experiment to validate our assumptions and hypothesis. We want to change this part of the feature to allow users to search for users by username (or pick from a drop down of followers/following) in order to add them to the blend. We hypothesise that by keeping this experience within the Spotify app, users are more likely to successfully invite their friends and create a blend. The native sharing modal when you click on the Invite button on the Spotify Blend feature. Or it could be that the friends aren’t responding to the invite message. We don’t have much data on exactly why this is, since it could be due to users not following through with sending the invite through the sharing modal. Let’s assume that, looking at the data, we see a considerable drop off between users clicking on the Invite button and their friends successfully accepting their invite. When the friends click on the link shared with them they are then added to the blend. When inviting friends to the blend, users are shown a native share modal to share an invite link with friends (as shown below). As the user listens to new music this taste match score changes, and the recommendations provided by the playlist get better.Īs a PM on this feature, let’s assume we discover that users are having some friction with the experience of setting up the blend. It also shows the user taste match scores between their music taste and the friend’s. The playlist is then autogenerated with songs that both the user and their friend like. For anyone unfamiliar with this feature, it’s a feature which allows a user to create a playlist with a friend of their choosing. Let us assume we’re a PM in the Blend Playlist Experience team at Spotify. Photo by Deepak Choudhary on Unsplash Product Context: Spotify Blend
0 kommentar(er)
