OpenJev has published an interactive browser demonstration that compares two ways of asking a local language model to make a constrained decision: directly reading its scores for allowed options, or prompting it to generate a probability distribution as JSON.
The experiment runs open models on the user’s device through WebGPU. MiniCPM5 2B is selected by default, while the site recommends Qwen3 0.6B for phones and lower-powered hardware. A larger 4-billion-parameter option is also available but requires substantially more memory. Model weights are downloaded from Hugging Face and retained in the browser cache, and the site says submitted inputs remain on the device.
Both evaluation paths receive the same decision. In the direct method, the software reads the model’s logits for the supplied option tokens and normalises them across that limited set. In the generative method, the model is asked to produce its estimated distribution in JSON, one token at a time. The two processes run sequentially so they do not compete for the same graphics processor.
OpenJev cautions that the direct output is conditional on the displayed choices. It is not calibrated confidence and does not account for every response the model might otherwise prefer. That distinction matters when a user interprets a neat percentage as a broad measure of certainty rather than a relative score among a fixed menu.
The page exposes timing measurements taken with the browser’s performance clock, including setup, warm-up, prompt preparation, direct execution, time to the first generated token and total generation time. It also notes that quantisation can alter both accuracy and speed. The demonstration uses pinned GGUF model builds through the wllama runtime, so its results describe those browser-ready versions rather than the original full-precision models.
A small public subset is used to compare equal-case agreement between the two techniques. The page labels its direct-decision accuracy columns as balanced accuracy and warns that browser quantisation can shift measured model quality. OpenJev does not claim that any of the local model tiers matches the published Jev result.
The release is best understood as a hands-on technical comparison rather than an independent benchmark. Its practical contribution is allowing developers to observe the latency and output differences on their own hardware while keeping the test local. It also makes visible a design choice often hidden inside applications: whether a model is used as a scorer over known choices or as a text generator asked to describe those scores.
