Lurker,
Having played Ultimate Frisbee before, I will agree with you that having no referee works. But, we are talking about a frisbee game. In Ultimate Frisbee, you don't typically see many slide-tackles, intentionally kicking other players (i.e. Keepers), etc.
I think the quality of play would go down horribly without officials. It would be one, huge cascade effect because one player would become slightly more aggressive than the others. Then, another one wants to be more aggressive than THAT player. Pretty soon, it would be a game of fouls, not a game of soccer.
Officials do more than just call player-to-player fouls. What about offsides, illegal throw-in, calling who kicked the ball out? Players would be too biased towards themselves to be able to make a proper call unless the call was blatantly obvious. Even so, I don't think players would be able to fairly call a game.
So would I think some sports could benefit from no officials? Some sports, maybe but soccer is not one of them.