Neural Rule Lists: Learning Discretizations, Rules, and Order in One Go
Sascha Xu, Nils Philipp Walter, Jilles Vreeken
Interpretable machine learning is essential in high-stakes domains like healthcare. Rule lists are apopular choice due to their transparency and accuracy, but learning them effectively remains a challenge.Existing methods require feature pre-discretization, constrain rule complexity or ordering, or struggleto scale. We present NeuRules, a novel end-to-end framework that overcomes these limitations. At itscore, NeuRules transforms the inherently combinatorial task of rule list learning into a differentiableoptimization problem, enabling gradient-based learning. It simultaneously discovers feature conditions,assembles them into conjunctive rules, and determines their order—without pre-processing or manualconstraints. A key contribution here is a gradient shaping technique that steers learning toward sparserules with strong predictive performance.  To produce ordered lists, we introduce a differentiable relaxationthat, through simulated annealing, converges to a strict rule list. Extensive experiments showthat NeuRules consistently outperforms combinatorial and neural baselines on binary as well asmulti-class classification tasks across a wide range of datasets.