Lower-limb exoskeletons have the potential to transform the way we move1–14, but current state-of-the-art controllers cannot accommodate the rich set of possible human behaviours that range from cyclic and predictable to transitory and unstructured. We introduce a task-agnostic controller that assists the user on the basis of instantaneous estimates of lower-limb biological joint moments from a deep neural network. By estimating both hip and knee moments in-the-loop, our approach provided multi-joint, coordinated assistance through our autonomous, clothing-integrated exoskeleton. When deployed during 28 activities, spanning cyclic locomotion to unstructured tasks (for example, passive meandering and high-speed lateral cutting), the network accurately estimated hip and knee moments with an average R2 of 0.83 relative to ground truth. Further, our approach significantly outperformed a best-case task classifier-based method constructed from splines and impedance parameters. When tested on ten activities (including level walking, running, lifting a 25 lb (roughly 11 kg) weight and lunging), our controller significantly reduced user energetics (metabolic cost or lower-limb biological joint work depending on the task) relative to the zero torque condition, ranging from 5.3 to 19.7%, without any manual controller modifications among activities. Thus, this task-agnostic controller can enable exoskeletons to aid users across a broad spectrum of human activities, a necessity for real-world viability. A task-agnostic controller assists the user on the basis of instantaneous estimates of lower-limb biological joint moments from a deep neural network so exoskeletons can aid users across a broad spectrum of human activities.