Model neuronal networks provide a phenomenological description of brain activity and serve as a primary tool for interpreting experimental observations in Neuroscience. Specifically, random networks exhibiting chaotic dynamics represent a standard framework for modeling the spatiotemporal irregularity observed in cortical activity. However, fundamental questions remain unanswered regarding what controls the geometry and dimensionality of the chaotic attractor. Here, we attempt to predict qualitative and quantitative features of the dynamics by investigating its equilibria and their stability. In the chaotic regime, a large number of equilibria are present. They are all saddles with an extensive, but fractionally small, number of unstable directions. Despite the network's connectivity being completely random, the equilibria are strongly correlated and, as a result, they occupy a relatively small region in the phase space. The attractor is located within this region. Because of this geometric organization, quantitative properties of the equilibria provide natural bounds on the network's dynamics. In particular, the fraction of positive Lyapunov exponents, and therefore the attractor dimension, is fundamentally constrained by the fractional dimension of the unstable manifold of the typical equilibria. This explains why the chaotic dynamics in these models can be described by a fractionally small number of collective modes. However, because the attractor dimension is extensive, data-driven geometric methods and system identification techniques for reconstructing the dynamics from purely observational data are fundamentally limited. We argue that the study of the dynamics must instead rely on ergodic theory by focusing on invariant measures that robustly capture the system's macroscopic properties.