Machine Learning Costs

Use case assumption:

• Medium sized eCommerce website
• 40M visits / month
• Latency: P99 = 100 ms
• 30 ML models in production
• Each model has 4 variants during testing
• 5 models use public data sets (pre-labeled data)
• 5 models use need customized labeling (ex: using Labelbox)
• 20 models use unsupervised learning
• Some are deep learning models

Summary of cost scenario: 

We have 30 models running production. 24 of those can be run on c5.xlarge instance configured with an Amazon Elastic Inference eia2.medium accelerator machines We also have 6 deep learning models that need to run on ml.p2.xlarge instances.   

Monthly cost of hosting 24 models running on ml.eia2.large:  

• Hourly price of c5.xlarge instance: $0.17
• Hourly price of a eia2.medium accelerator: $0.12
• Total hourly price of the instance configured with the accelerator: $0.29
• Total monthly cost = $0.29 * 24 * 31 = $215.76 * 24 = $5178/month

6 Deep learning models running on ml.p2.xlarge

• $1.26/hr = $907/month * 6 = $5442

Testing/labeling costs 

• Labeling (MTurk):  $0.08 per label for 20k labels  = $1,600 (per month) 
• 4 model variants during testing (4 hours each):  120 ml.m5.2xlarge running 4 hours each:($0.538/hr = $258/month)

Total Cost Per Month = $5178+$4116+$5442+$1600+$258 = $16,594 / month