This is a very broad question, and the answer would basically fill an entire book. In a nutshell, I would come up with the

• continuous target variable? -> regression
• categorical (nominal) target variable? -> classification
• ordinal target variable? -> ranked classification
• no target variable and want to find structure in data? -> cluster analysis, projection

• use "cheaper" models/algorithms
• dimensionality reduction
• feature selection
• lazy learner (e.g,. k-nearest neighbors)

• out of core learning
• distributed systems

• hard to know the answer upfront
• always a good idea to compare different models

• increase regularization strength if supported by the model
• dimensionality reduction or feature selection otherwise
• collect more training data if possible (check via learning curves first)

• one option are lazy learners (e.g., K-nearest neighbors); needs to keep training data around; no learning necessary but more expensive predictions
• it's generally relatively cheap to update generative models
• another option is stochastic gradient descent for online learning

...

The list goes on and on :). I think Andreas Mueller's scikit-learn algorithm "cheat-sheet" is an excellent resource. (Click on the image to view the original, interactive version on scikit-learn)

[Source: http://scikit-learn.org/dev/tutorial/machine_learning_map/index.html]