Given that you do not want to railroad your players by insisting that they work through the maze in the manner you insist, it still remains to be said that your players are not in control of your world, nor should they be.
Over the weekend I stumbled across this post by the Retired Adventurer, John Bell. It's a good article, though somewhat unclear in its use of academic buzzwords which are generally used to obfuscate when simpler, more direct language would make the point better. The other failing is that it hinges on an article that requires a password to read, which adds to the pretentiousness of the whole.
Bell defines his "anti-narrative" stance thusly:
"I mean that my primary focus as a referee is not on "telling a story," moving PCs from rising action to rising action until an emotionally cathartic climax is attained, but on presenting interesting and meaningful choices with escalating consequences, and pushing the PCs to decide which option to take."
This more or less relates to what I've always tried to say on this blog. But of what Bell has written, it is the word 'pushing' that I think is the most interesting.
My last post on Rail Rodders, the players who must ride the rails the DM creates, emphasized that there shouldn't be a railroad at all, not even one that is hidden. However, this is not to say that the players are defacto placed in charge of everything that happens in the world. Far, far from it. The players may make choices, as Bell says, but the players must also put up with obstacles.
Where it gets easy to create a narrative as a DM is in that many, many things which happen to players are entirely out of the player's sphere of control. This is true with all of us in our daily lives. Count how many things - particularly dangerous, life threatening things - have nothing whatsoever to do with our choices, but which are pushed on us from outside by the chance of events.
You are sitting in your car on a road, dutifully stopped at a stop light. A driver moving perpendicular to the position of your front bumper is hurrying to work and has no stop sign to contend with. However, the night before a service truck passed by here leaking oil because the worker who was supposed to fix the leak was at the hospital attending to his wife who was injured falling on the kitchen floor. The temperature this particular day is just right to give the oil the right amount of slickness for the tires on the other driver's car, so that the driver loses control. A moment later, you're seeing his car skid towards yours ... you have seven tenths of a second to react before a large white bubble explodes in your face. Next, you're staring at a hospital ceiling, aware only that someone is holding your hand ...
Not every action we experience is a 'choice.' A vast number of experiences we have everyday are anything but choices, and it must be realized that in the fundamental construction of a campaign, there will be many times when events must conspire to force players to take the only rational action and hope for the best. When faced with a lava flow, there is only one direction to go.
Moments of the game are going to be like that. A host of goblins are simply going to be intractable. The motivations of the local overlord are going to be intractable. Fact is, many people ARE intractable, and imposing them into your D&D world can be a satisfying experience for a party. You don't always get what you want. You don't always get your way. Sometimes, you get screwed and you learn to get over it. This undermines that part of your players that make them haughty and pompous and on the whole intolerable. Humiliation - at the hands of a speeding car, or an implacable enemy or a force beyond their ken - causes people to suck it up, sort themselves out and reidentify the world as something that simply cannot be 'got around' in the same old way. Humility in turn breeds character.
This cannot be done by endlessly offering players a way out of every crisis simply by being smart enough. Like the car accident above proves, we're all mortal - no matter how puffed up we are, there's always a moment when something unexpected can cut us down a notch. Accepting this about the world is what makes us human ... and anything that humanizes your players will deepen your world.
So now and then - not all the time, not even as often as rarely - you have to give a three-card monty game to your players that they cannot win. You have to choose these moments very, very carefully. You have to make sure they apply to every person in the party, fairly and equally. You have to design them so they seem to have happened from a place outside your decision-making process.
I shall try to give you an example, created by a writer, which John Bell above might describe as an unfair narrative, but which I would describe as the only possible result of these two characters interacting with one another. This is what makes good writing - when the narrative has nothing to do with what the author wants ... but which merely describes a circumstance occurring in the only way it CAN occur.
The film is Hair. In the clip below (starting at 5:15), the character George Berger tries to talk his way onto an army post. Berger, being what Berger is, as defined by the movie, simply cannot understand why he can't simply do as he wants. The guard, in character a million miles from anything that could begin to understand what Berger is or what he wants, cannot act or be any other way that how this guard acts. There are no choices here. There could never be a choice here. Not in terms of this interaction.
Of course, Berger does find a way around it. With very, very bad consequences. Berger doesn't find a way to win, he finds a way to lose more profoundly.
Once again, you cannot argue with a lava flow. You can only avoid them.