Machine Learning Recommender Engine with AWS SageMaker

Introduction

Objective

Context

Image for post
Image for post

Environment and tools

Our solution

Libraries imports

import pandas as pd
import numpy as np
import io
import sagemaker.amazon.common as smac
from pandas import DataFrame
import boto3
from sagemaker import get_execution_role
import sagemaker
from sagemaker import KMeans
from datetime import datetime
from sklearn.preprocessing import MinMaxScaler
import matplotlib.pyplot as plt
import seaborn as sns

Loading data

role = get_execution_role()
bucket='guruchallenge-bucket'
sub_folder = 'movielens_dataset'
data_key = 'movies.csv'
data_location = 's3://{}/{}/{}'.format(bucket, sub_folder, data_key)
movies = pd.read_csv(data_location, low_memory=False, delimiter=',', encoding='utf-8')
data_key = 'ratings.csv'
data_location = 's3://{}/{}/{}'.format(bucket, sub_folder, data_key)
ratings = pd.read_csv(data_location, low_memory=False, delimiter=',', encoding='utf-8')

Feature engineering

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Training and deployment

data_train = df_scaled
data_train = data_train.astype('float32')
num_clusters = 10
output_location = 's3://' + bucket + '/model-artifacts'
kmeans = KMeans(role=role,
train_instance_count=1,
train_instance_type='ml.c4.xlarge',
output_path=output_location,
k=num_clusters)
%%time
kmeans.fit(kmeans.record_set(data_train.values), job_name=job_name)
kmeans_predictor = kmeans.deploy(initial_instance_count=1,
instance_type='ml.m4.xlarge')
%%time 
result = kmeans_predictor.predict(data_train.values[0:len(data_train)])
cluster_labels = [r.label['closest_cluster'].float32_tensor.values[0] for r in result]
CPU times: user 51.3 ms, sys: 82 µs, total: 51.4 ms
Wall time: 297 ms
sagemaker.Session().delete_endpoint(kmeans_predictor.endpoint)

Let’s recommend items!

clust_movieRatings = ratings.loc[ratings['userId'].isin(users_cluster)]
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

Conclusion

Entrepreneur, Tech Enthusiast & Musician.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store