Heat all but 1 Tbsp. of olive oil in a large skillet over low heat. Add anchovies and cook, stirring until the anchovies dissolve.
Add garlic and cook for about 15 seconds, taking care not to brown it (garlic gets bitter if browned).
Raise the heat to medium-high and add tomatoes, bay leaf, and salt. When the sauce comes to a boil, turn the heat down and simmer until the tomatoes have reduced and separated from the oil. This may take 20-40 minutes.
Remove sauce from heat, remove Bay leaf and set sauce aside. (If you like, you can safely refrigerate the sauce and save for the next day.)
Bring 4 quarts of water to a boil in a large pot; add 1 tsp. of salt, and drop in the penne all at once, making sure that it is completely submerged. Stir for at least one minute and let it return to a boil.
When the pasta is halfway done (4-5 minutes), return the sauce skillet to a medium heat, adding the basil, capers, and olives. Cook pasta until al dente; drain, and return to pot. Mix with sauce and serve.