Over the last decade, silicon photonic neural networks have demonstrated the possibility of photonic-enabled machine learning at the edge. These systems enable low-latency ultra-wideband classifications, channel estimations, and many other signal characterization tasks within wireless environments. While these proof-of-concept experiments have yielded promising results, poor device and architectural designs have resulted in sub-optimal bandwidth and noise performance. As a result, the application space of this technology has been limited to GHz bandwidths and high signal-to-ratio input signals. By applying a microwave photonic perspective to these systems, the authors demonstrate high-bandwidth operation while optimizing for RF performance metrics: instantaneous bandwidth, link loss, noise figure, and dynamic range. The authors explore the extended capabilities due to these improved metrics and potential architectures to continue further optimization. The authors introduce novel architectures and RF analysis for RF-optimized neuromorphic photonic hardware.