The predecessor probably doesn't support the AM4 socket, or only with an additional adapter that takes up more space. That's why there is a special AM4 version and I wouldn't take anything else myself. Do you perhaps mean NH-L9a-AM4 (3.7cm instead of 6.5cm)? The L9A version is a lot more compact.

There are standard dimensions that should actually guarantee that a cooler fits on a motherboard, but Mini-ITX often goes to the absolute minimum. However, you will hardly find smaller, more space-saving professional coolers than these two models, I think. With RAM, I would do without headspreaders if they make the modules higher, as the low cooler might overlap at least one module. Alternatively, you can remove the spreader very carefully with a hair dryer (warranty gone).

Whether the BD drive fits depends on the case (size/type). In mini-towers, the 5.25" drive bay is often above/below the mainboard height (i.e. no conflict). With a flat case and 3.5" drive on top, the drive should not be too close above the cooler (1cm for airflow is certainly too little).

In general, I think the triangle is: price - performance - size. Choose two of them, i.e. I would only choose a miniaturised form for valid reasons.