We have a layered firewall architecture. Would it be safe to allow UDP traffic through the second layer of firewalls?...
Meaning the components in the first layer (Internet facing) require UDP communications with devices in our second tier and possibly to our internal network. With defensive measures in place, is it safe to allow this type traffic?
If you have blocked all UDP traffic from the outside, then you are safe from that UDP traffic. Allowing UDP traffic from the first layer nodes to the second layer, and perhaps to the inside network, does allow for malicious insiders to exploit the same UDP vulnerabilities you are protecting against from the outside. If this is an acceptable risk, then yes, it is "safe" to allow that type of traffic. If you are concerned about the insider risk, then no, it is not safe.
In general, you should deny all traffic and only allow that traffic that is needed. If you need specific UDP ports to be opened in the second tier for your setup to work, then only open those specific ports. There is no need to open them all, if only certain ones are needed.
What you need to do is balance the risk of allowing those communications with the operational need to have them. If the risk is considered to be low enough by someone with the authority to make that decision, then go ahead. What you really have is what is known as an Accreditation decision.
For more information on this topic, visit these other SearchSecurity resources:
Ask the Expert: Determining which TCP/IP services are needed
Ask the Expert: Explanation of ports
Learn how a new QUIC protocol is improving Web apps