There is a famous story told in Chassidic literature that addresses this very question. The Master teaches the student that God created everything in the world to be appreciated, since everything is here to teach us a lesson.
One clever student asks “What lesson can we learn from atheists? Why did God create them?”
The Master responds “God created atheists to teach us the most important lesson of them all – the lesson of true compassion. You see, when an atheist performs and act of charity, visits someone who is sick, helps someone in need, and cares for the world, he is not doing so because of some religious teaching. He does not believe that god commanded him to perform this act. In fact, he does not believe in God at all, so his acts are based on an inner sense of morality. And look at the kindness he can bestow upon others simply because he feels it to be right.”
“This means,” the Master continued “that when someone reaches out to you for help, you should never say ‘I pray that God will help you.’ Instead for the moment, you should become an atheist, imagine that there is no God who can help, and say ‘I will help you.’”
– Tales of Hasidim Vol. 2 by Martin Buber