"There's a big difference between consciously doing nothing versus actually wasting time," Vanderkam says. "Wasting time is spending it on things that aren't particularly meaningful or even enjoyable to you."

Surfing channels for an hour is wasting time, she says. Watching an episode or two of your favorite show, on the other hand, is healthy downtime.

"You want to make sure downtime is doing what it's supposed to do, which is rejuvenate you so you can return to your busy life more refreshed," Vanderkam says. "If it's not adding to your energy levels you may want to stop doing it."

That doesn't mean you have to spend your downtime reading Tolstoy.