I'm currently in the middle of writing a small article with some (of my) best practices for managing a VoIP-heavy network.
I would be curious to hear the experiences of others.
Generally, I setup my installations or my clients' installations with a pfSense router. That is, I install pfSense on a PC with dual-LAN for managing the network. Yes, there are more possible points of failure than just grabbing a hardware router but pfSense is far more customizable than even my second-favourite management suite: tomato. If properly managed, a pfSense box should have very few issues.
I don't have any redundancy set up as far as routing goes, although perhaps I should.
When running ViciDial, I always have the server within the same LAN, and use g711 uLaw codec between each station and the Dialer. I use SIP protocol to connect to the dialer, but externally I use an IAX2 trunk to terminate with my carrier. I setup QoS internally for SIP RTP/signaling ports at highest priority, and prioritize all external traffic for IAX2 (4569).
I setup port forwarding usually for HTTP (80) to my ViciDial server, so I can access its data from anywhere just by navigating to the IP of the router. It would probably be more secure to change this port.
I also set port forwarding to the (main) telephony server in my cluster for SSH, as it's usually the one I need to access if something is going wrong, and that way I can quickly SSH into it from my phone or workstation wherever I am.
I run reports from pfSense on usage so I can monitor traffic throughout the day, and if there was a specific point where something happened, like everybody dropping their calls at once, it is easier to pinpoint what it is that failed.
Any input welcome.