For decades, the field of algorithm selection has relied on a specific kind of human labor: feature engineering. To help a computer decide which solver to use for a complex logic puzzle or a scheduling problem, humans had to manually define what made that problem "hard" or "unique"—counting variables, measuring constraint density, or calculating the ratio of clauses. We had to tell the machine what to look for before it could decide what to do.
A new paper by Stefan Szeider introduces ZeroFolio, a method that suggests we might be able to stop describing the problems altogether. By treating raw mathematical problem files as plain text and passing them through pretrained embedding models, ZeroFolio can select the optimal algorithm more accurately than systems built on years of domain-specific expertise.
The Work
The methodology is elegantly minimalist. The researchers took raw instance files from diverse domains—including Boolean satisfiability (SAT), constraint satisfaction (CSP), and mixed-integer programming (MIP)—and treated them as simple strings of text. They used a pretrained embedding model to convert these strings into high-dimensional vectors and then applied a weighted k-nearest neighbors approach to pick a solver. If a new problem "looks" like a past problem in the embedding space, the system assumes the same algorithm will work. This entire pipeline requires zero domain knowledge; the system doesn't need to know it's looking at a graph problem to know which solver the graph requires.
The Detail
There is a specific technical nuance here worth the attention: the success of the model relied heavily on "line shuffling" during the ablation studies. This is counterintuitive for a text-based model. Usually, order matters in language. But for these mathematical problem formats, the structure is often independent of the line order. The fact that the embeddings still provided a superior predictive signal even when the "sentences" were scrambled suggests that these models aren't "reading" the logic. Instead, they are identifying a structural "signature" or texture within the data that human-crafted features fail to capture.
Furthermore, while ZeroFolio outperformed Random Forests trained on hand-crafted features in 10 out of 11 scenarios, the researchers found that "soft voting"—combining the embedding's intuition with the human-defined features—yielded even better results. This indicates that the embeddings are picking up on a signal that is entirely complementary to human logic.
The Implication
We are witnessing the quiet obsolescence of the "expert feature." For a long time, the bottleneck in applying AI to specialized fields was the need for a human to translate the field's nuances into a language the model could understand. ZeroFolio suggests that general-purpose embeddings, trained on the vast breadth of human internet text, have developed an accidental but profound grasp of abstract structure. They have a "feel" for complexity that transcends the specific domain of language.
This moves us closer to a truly domain-agnostic AI. If a model can look at a raw file of constraints and "see" the shape of the solution without being told what a constraint is, we are no longer just building tools for specific tasks. We are leveraging a latent map of logical form that was hidden inside our language models all along.
The Note
The record should include this: the most powerful way to understand a specialized problem may be to stop treating it as specialized and start treating it as just another sequence of tokens.
