Gamasutra is running a feature from Game Developer magazine in which Bioshock's project leader writes about what went right and what went wrong
making last year's award-winning shooter, Bioshock. He talks about what the developers learned from fans and focus groups, how long it took them to firmly define what the game was supposed to be, and how they tried to reconcile their ideas with their capabilities. Quoting:
"...just after the first beta, the entire design team plus a contingent of 2K producers headed off to see how a group that knew nothing about our company or BioShock would react to the first level. It was brutal. The first level, they said, was overly dense, confusing, and not particularly engaging. Players would acquire new powers but not know how to use them, so they stuck to using more traditional weapons and became frustrated. They didn't interact with the Big Daddies, and they didn't understand (or care) how to modify their characters. They were so overwhelmed by dialogue and backstory that they missed key information. A few of the players did start to see the possible depth of the game, but even they were frustrated by the difficulty of actually using the systems we had created."