We are physical, sensory beings living in physical sensory bodies and
that is how we navigate the world. If you want your readers to live
inside your story, not just be told about events from a distance, you
have to give them a physical sensory world to live inside. And that
means describing it.
Description often gets a bad rep in writing, accused of "bogging the story down" or "swathes of unnecessary description" - as if description had no part in story or were always, by definition, unnecessary. We might feel like that now about some nineteenth-century novels. When I was sixteen, I taught my brother and his two best friends their final-year setwork, Thomas Hardy's Far from the Madding Crowd (oh, the things we do for love, or at least for teenage crushes on brothers' friends) and the endlessly eloquent descriptions of fields drove all three boys beserk with frustration. Even for the literature-loving younger sister, three pages on one vista was hard to stomach. So yes, all that rhapsodising about a single "featureless convexity of chalk and soil" may well feel unnecessary and bog the story down, to a contemporary reader, or at least in a contemporary book. A total lack of description, however, is just as bad, if not worse.
I had a similar experience with Orson Scott Card's Pathfinder. (I think he's sufficiently established for naming him to do no harm.) I love his other books; I've reread what we have of the Alvin Maker series several times, lived inside the Ender's Game series, and parts of Songbird remain glowing in my mind like a Vermeer.
And then I read Pathfinder. It started in a forest, of some sort. Not sure what sort. With a village nearby, over a waterfall. They travelled on a river. Narrow as a canal, wide as the Amazon, grey or green or blue, rapid or slow? No idea. Likewise, the boat. I could see nothing. I had high hopes of the Tower of O, of which so much had been made, and then finally, "they saw it: the Tower of O, rising above the trees that lined the river." (I still don't know if these are mangrove swamps or willow trees or hunched low as hawthorn or tall as redwoods, they're just Some Trees. And the tower rises above them.) In the next sentence, we learn it is "a steel cylinder rising up and rounding off in a dome at the top". That's it. All the description we're to get of this all-important tower. If this weren't an author who'd fed me much joy of worlds in other books, I'd have thrown it aside then, but I ploughed on through another 500 pages, baffled and blindfolded. Beserk with frustration indeed.
If you want your readers, or your listeners, to live in your story's world, you have to tell them what the place looks like. What it smells like. Feels like. Sounds like. Show them where they are; let them smell, feel, and hear where they are. Place is an essential part of what makes a story immersive. It also has another, equally crucial role. Place is memory.
A memory palace, that ancient and seemingly magical memory technique, is no myth - nor does it need to be a palace, specifically. For those new to it, you imagine a detailed place you know well (a palace, if that's your lifestyle, or a favourite walk, or a house, a ship, anything really) and then attach everything you learn to different places within it: if it's a ship, you could put the maps in the captain's cabin, and fill the drawers with poetry. Thomas Cromwell was reputed to use a memory palace, and likewise, fictionally, was Sherlock Holmes. Contrary to popular thinking, though, it's not restricted to very special savantes: a memory palace isn't a sign that you really are quite spectacularly intelligent, only that you know the best way to remember things, and use it. It's also known as the "method of loci" (loci = place in Latin), using our remarkable knack for spatial learning.
We attach memories and knowledge to places so easily, whether the place is real or simply richly conceived in our imagination. As I've written before, whenever I ask my students about their favourite books, the sense of place comes up: the story lives in the place and the place lives on in their minds.
By witholding a sense of place from your readers, by not giving them a richly conceived physical environment in which things happen, by cutting description down to "essentials", you're robbing them of one of their most powerful memory tools - and robbing your story of much of its memorability.
Spatial memory is vital for far more than simply remembering "what the world looks like". How often have you met someone out of context - the woman from the Co-op, walking down the street - and struggled to place them? Or tried to regain the thread of events after an interruption and said, "Where were we?" (These types of metaphorical truths are called embodied cognition.) We'll remember your characters more easily if we meet them more or less where we expect them, the first few times. For readers of Robin Hobb's Farseer trilogy - how much of your sense of Chade is wrapped up in his tower room, your sense of Burrich in the stables, where he sits in his room above, with his bottle of brandy and the tack he's mending?
A sense of place is also one of the fastest, deftest ways to resituate your readers after an interruption - after a flashback, perhaps, or a diversion into a subplot thread. If you come back to the same place as you left the characters (they were in a forest - is the mulch on the forest floor starting to freeze?), the familiarity of place lets us swiftly regather the thread.
Within the story, your descriptions, your sense of place, allow us to remember so much, and when the story's finished, to remember the book itself: one of those that lives on in our heads, as a place we can visit.