well, speaking from a coach, a ref and a player, i can agree with everything you are saying. To be honest, the parents on the sidelines are a real problem for the AR that has to deal with them. the center ref can typically ignore the parents, if he/she chooses to do so. Coaches can be just as abrasive to the AR on that side and can also be ignored if the center offical wants to. Players are a different story. Dealing with players who complain about every call and wine about not getting a call are the true issue that i dealt with as a coach.
Now.....i was one of those players who gave the refs an earful. I was also one of those coaches who gave the ref and earful. All of that led my players to trend in the same direction. I did take the stance to my players telling them that it was my job to talk to the ref and their job to play. Unfortunately, that just doesnt always work and the players will show their displeasure in different levels.
I do not ref or coach any longer. I am getting back into playing and do not yell at refs any more. Getting older and seeing all the bad stuff that i can now see from another view, makes me see what the real issues were and continue to be. All levels of all sports are going to have fans who dont like the calls that a ref makes. Parents and coaches have to keep this in mind and teach the players that it is a game that is meant to be fun and competitive at the same time. If the ref is truly awful, and i admit that there are some of those, report him to the assignor at the end. But dont just send it reports on the bad ones without commending the good ones. The assignor needs to know who is doing good and who is doing bad, but if all he hears is the bad, he may not listen to the complaints as much.
Get a handle on the yelling and screaming and i think you will keep refs around. The pay isnt bad. Travelling 20-30 minutes for work is something refs choose to do to make some cash. it fits the high school and college kids perfectly, but they wont listen to the bickering. Once you lose a young ref, they probably wont come back.