Loading…

Sam Kenkel

Data Science, Machine Learning, DevOps, CCNA, ACSR
Learn More

Anime_Rec4: Predicting User Scores with Neural Nets

Part 1 of this series explained why I was making an Anime recommendation system, gave a brief overview of the approach I was taking. Part 2 explained how I got my Data. Part 3 explained  how I tuned my 3 Item-Item similarity models to generate ‘possible’ recs. In this part, I’ll talk about predicting user scores with neural nets.

Why 3 Neural Nets: Ensembling by targeting different scores:

There are 3 pre-trained neural nets. Each net has been trained to predict one type of score: Score, User Scaled Score, Anime Scaled score.
The Nets are loaded with their trained weights, only the user embedding layer is unlocked .
Each neural net predicts the user’s score for every possible rec (from the item-item methods). These predicted scores are sorted to give recommendations.

Every neural net has the same architecture:

Using Keras I could make this neural net in very few lines of code:

I prefer Keras for problems like this: I can spend more time iterating (because it’s a higher level language).

After the net is trained, all of it’s weights are loaded into an almost identical neural net, but with “trainable” set on many of the layers. This net therefore has “locked” weights for every layer except the user embeddings.

Neural Net 1: Predicting user score
Data Prep:All scores of 0 (status but no score) are dropped. Scores from outlier (<25 scores,=””>1500 scores) users are dropped.
No Modifications are made for Watching, On-hold, Dropped, etc. Both users and Animes are embedded with N hidden factors. I use 36 factors in the final model. (These factors are not shared between models)
The point of this neural net is simply to predict the highest score. The status information is totally ignored in this net.  Because it is trained against the entire dataset, this will tilt towards ‘popular’ shows. The purpose of this neural net was to learn that “Most people like Steinsgate” so it’s a safer recommendation.

Neural Net 2: Only ‘completed’ or ‘Dropped’ shows matters: User_scaled Score
Data Prep:All scores not listed with a status of ‘Completed’ or ‘Dropped’ are ignored. Scores from outlier (<25 scores,=””>1500 scores) users are dropped. Both users and Animes are embedded with 36 latent factors. (These factors are not shared between models) On the 1 million score test set, the best MAE was: 0.916866 The final MAE was 0.949197 This neural net is trying to predict shows that users will like more than their personal average. This user_scaled score is the most interpretable, and in my experience with this dataset, best way to start working with user scores.
Neural Net 3: Only ‘completed’ or ‘Dropped’ shows matters: Anime_scaled Score
Data Prep:All scores not listed with a status of ‘Completed’ or ‘Dropped’ are ignored. Scores from outlier (<25 scores,=””>1500 scores) users are dropped. Both users and Animes are embedded with 36 latent factors. (These factors are not shared between models) On the 1 million score test set, the best MAE was:0.932877 The final MAE was 0.98392

This score is standardized by subtracting the mean of the Anime’s score, and it therefore becomes a reflection of how much more or less than the rest of the community a user likes it’s data.

Evaluation: