What if you want your algo adapt as new data comes in?

Stochastic gradient descent
Good choice if:
You have continuous amount of data (way too much data to retrain the algorithm with all data)
Your data behavior slowly changes
(Say, tv ad use to have a dominant effect on sales, but as number of online users grow, tv ad effect isn't as dominant anymore )