AesCode Infographics Rule: The Deterministic Metric and Its Role in Evaluating Aesthetic Code

Definition of the Rule Metric and Its Function in the AesCode Infographics Suite

The Rule metric in the AesCode Infographics benchmark is defined as the average of three deterministic checks: Text, Bound, and Chart. These verifications evaluate structural and semantic aspects of the generated code that can be programmatically extracted from the source code, the DOM, or the browser rendering.

Unlike visual metrics, which rely on a Vision-Language Model (VLM) judge to assess non-parsable properties such as hierarchy or style, Rule focuses on verifiable correctness. Its function is to ensure that the code meets exact content requirements, canvas containment, and data structure, providing an objective baseline before evaluating aesthetic quality.

Text, Bound, and Chart represent separate kinds of checks rather than interchangeable tests. Text concerns whether required strings appear; Bound concerns whether the content stays within the canvas and matches the expected aspect ratio; Chart concerns the structure and data of tables or charts. Their average gives a compact summary of those checks, but it does not describe every property of the generated infographic. Interpreting the total therefore requires keeping the components’ distinct purposes in view.

The distinction between Rule and Visual follows from the kinds of properties each can assess. A verifier can look for a required string or compare measurable geometry and structured data against a condition. Judging whether a visual hierarchy feels clear or a style feels cohesive is not reduced to the same sort of exact programmatic test. Rule provides a bounded account of verifiable correctness; it does not turn satisfaction of those requirements into a complete measure of design value.

A single average is useful as a summary, but it can conceal differences among the component checks. Text, Bound, and Chart need not respond in the same way to a particular output, and the aggregate does not identify which component accounts for a score. The figure should therefore be read alongside the checks it combines: it describes compliance with the specified conditions, not every characteristic that may matter when evaluating an infographic.

Execution Methodology: How Verification Rules Are Applied

The AesCode evaluation harness requires a browser rendering (Chromium via Playwright) and the use of PaddleOCR to extract text. Each score is calculated from this rendering, not just the source code, ensuring the metric reflects the final visual output.

The process involves comparing the target design graph (derived from the prompt and reference image) with the candidate graph generated by the model. Properties with structured evidence in the code or measurable in the rendering are assigned to deterministic verifiers. If a channel like PaddleOCR fails or is missing, the channel returns zero instead of abstaining, which can corrupt the reward if not properly managed.

Rendering matters because the evaluation concerns not only the HTML and CSS that a model produces but also what that code displays in the browser. Text extraction and geometry measurement provide signals for comparing the rendered output with required content and canvas conditions. Inspecting source code alone does not reproduce the stated procedure, since the score is calculated from the browser rendering and the evidence extracted from it.

The target design graph and candidate graph play different roles in the comparison. The target represents what should be checked from the prompt and reference image; the candidate represents the model’s generated result. Deterministic verifiers are assigned to properties for which structured evidence in the code or measurable evidence in the rendering is available. Not every difference between the two graphs consequently becomes an automatic Rule check, particularly when it concerns visual qualities outside the observable requirements of Text, Bound, and Chart.

A PaddleOCR failure being returned as zero makes the availability of that channel relevant to interpreting the score. Unless an extraction failure is distinguished from text that is genuinely absent, the returned value can reflect a tool failure as well as the model’s compliance. The same care applies to browser execution and the other harness steps: interpreting the result depends on the required evaluation process completing, rather than on the assumption that a missing signal is equivalent to a verified failure.

The sequence of operations clarifies what Rule measures: the generated HTML and CSS are rendered in Chromium, text and pixel geometry are extracted, the content, boundary, and chart or table verifiers are run, and the resulting component scores are averaged. A change in the evidence available at one of these stages can affect the calculated value. This workflow describes checks applied to selected properties of a particular rendering, not a direct evaluation of every feature visible in the image.

Comparison with Visual Metrics: Stability and Reproducibility

In the 300-instance infographic benchmark, rule-based scores (Rule) are consistently higher and more stable than visual scores for the evaluated closed-source code models. This stability stems from the deterministic nature of the checks: they do not depend on the subjectivity of a vision-language model.

However, this stability does not imply absolute superiority in quality evaluation. While Rule measures structural correctness, visual metrics (Content, Layout, Style) capture human perception of design. Weaker models show a sharper drop in visual metrics than in rule compliance, suggesting that Rule may be less sensitive to subtle aesthetic failures.

The reported stability applies to the 300-instance infographic benchmark and the evaluated closed-source code models; it does not establish that Rule is more stable in every dataset or configuration. In that stated setting, rule-based scores are described as higher and more stable than visual scores. The channels assess different things: Rule checks bounded requirements, while visual metrics assess design dimensions that those checks do not capture.

A difference between the scores does not make either result redundant. A model can satisfy required strings and canvas limits while still producing weak visual hierarchy or inappropriate colors. In such a case, Rule provides favorable information only about the properties it checks, while a lower Visual score indicates problems in dimensions assessed by that channel. Reading the scores together helps distinguish these kinds of outcomes without treating structural compliance as an aesthetic judgment.

The sharper decline in visual scores than in rule compliance among weaker models suggests that the channels have different sensitivities; it does not make their scores equivalent. Rule may miss subtle aesthetic failures because its checks concern content, containment, and data structure. A gap between Rule and Visual therefore describes the distinct reach of the metrics, but by itself it does not establish general model competence or define a universal threshold for competence.

Limits of the Rule Metric for Evaluating Actual Aesthetic Quality

The Rule metric has inherent limits for evaluating actual aesthetic quality. It does not capture properties such as typography, color usage, visual hierarchy, or design cohesion, which are essential for the perception of 'good design'. These aspects are evaluated through the Visual channel, which uses a sample-specific rubric (VGR) and a VLM judge.

Furthermore, model comparability can be affected if the Rule metric does not capture human design perception. A model may have a high Rule but a low Visual score, indicating that the code is structurally correct but aesthetically deficient. Therefore, Rule should be interpreted as a correctness metric, not an aesthetic quality metric.

Rule’s limits follow from what its checks are designed to verify. Typography, color, visual hierarchy, and design cohesion are not captured by the metric, even though they matter to the perception of good design. An output can meet the checked requirements and still look unattractive or lack cohesion. The score expresses structural correctness within the stated scope, not general approval of the design.

Visual complements Rule by assessing properties that are not resolved by deterministic checks, using a sample-specific rubric and a VLM judge. This complementarity does not make the metrics interchangeable, nor does a Visual score replace or correct a Rule score. Each answers a different question: whether specified structural and content conditions are met, and how the evaluated visual dimensions are judged.

Keeping this distinction explicit matters when comparing results. A high Rule score does not establish that typography, colors, or hierarchy are suitable; a low Visual score does not, by itself, show that content, containment, or chart-structure checks failed. When the scores diverge, the warranted interpretation is that the channels are describing different aspects of the output. The stated results do not define a universal quality category or a Rule threshold that guarantees an attractive design.

Impact on Model Comparability and Conclusions

Using Rule as a separate metric allows for a more granular evaluation of code generation models. By separating correctness from visual quality, teams can identify whether a model fails due to structural errors or aesthetic limitations. This is crucial for integration into production pipelines, where correctness is a prerequisite for quality.

AesCode results show that closed-source code models tend to have higher scores in Rule than in Visual, with Style being the weakest dimension. This suggests that structural correctness is easier to achieve than aesthetic quality, and that deterministic metrics like Rule are useful but insufficient for a complete evaluation of design quality.

Keeping Rule separate from Visual makes evaluation more granular by distinguishing verifiable structural requirements from visual judgment. This distinction can help identify whether a result has problems in checked code, content, or boundary conditions, or whether its difficulties appear in aesthetic dimensions. The diagnosis remains limited: the Rule average alone does not show which of its three components explains the result, and it does not replace assessment through the visual channel.

For the 300-instance benchmark with a reference image, the table reports Rule scores above Visual scores for the selected models. This specific comparison accompanies the observation that Style is the weakest dimension, but it should not be extended beyond the models, instances, and evaluation setting described. Rule and Visual are different evaluations, not a shared scale of aesthetic quality; their difference describes the reported pattern rather than establishing universal superiority.

The usefulness of Rule in model comparisons depends on keeping its scope visible. When the checks cover required strings, canvas limits, and the structure of tables or charts, score differences describe performance on those checks, not every possible reason an infographic might be better or worse. A fuller reading of the results also considers the visual metrics, since aesthetic quality can vary without corresponding changes in structural checks.

The bounded conclusion from the reported results is that, in this benchmark, closed-source code models tend to score higher on Rule than on Visual, while Style is the weakest dimension. This pattern suggests that meeting structural conditions may be more attainable than achieving the aesthetic quality assessed by the visual channel in this setting; it does not show that structural correctness is always easier in every situation. Rule makes one kind of compliance explicit, but by design it cannot summarize design quality on its own.

Keep exploring

Corrections and transparency

If you spot incorrect or outdated information, send us a correction with the page and source we should review.