Being around in a busy and noisy town like New York City sometimes get me to seek more quiet places. Actually you can find peace and quiet quite easily by just going to some of the parks: High Line, Central and Fort Tryon parks spring immediately to mind.
You might also be lucky to discover cafes or wine bars that do not have loud music nor PCs around, and where you can just relax with a cup of tea or a glass of good wine and do some meaningful thinking.
But even more hidden places exist. I discovered one the other day and it is not easy to find, but when you arrive inside you're immediately in another world. This place is the SAORI weaving studio. Spend a few hours there just concentrating on which colour next to put on your loom and you'll go away in a calm spirit.