Author
A review method for a proposed product-identifier text transformation: keep the source string, identify which equivalence form is applied, record collisions as findings, and stop when business identity is unresolved. Text equivalence is not catalog-item equivalence, and no AI visibility outcome is promised.
A catalog import proposes normalized product-ID strings, and the team has to decide whether to accept the transformation. The decision belongs to the catalog owner, not to the normalizer. A text transform can tell you that two strings are equivalent under a formal rule; it cannot tell you that two catalog records describe the same sellable item.
That separation is the whole point of this review. The character standard treats canonically equivalent sequences as representations of the same abstract character, with matching intended appearance and behavior. It defines compatibility as a weaker equivalence that can retain differences in appearance or behavior. Both are statements about characters. Neither is a statement about SKUs, part numbers, packaging variants or commercial identity.
So the first check is not technical at all. Before any transformation runs, the team should be able to say which value is the authoritative source identifier and who owns it. If that answer is missing, the normalization proposal has nothing safe to operate on, and the review should stop there rather than proceed on the assumption that similar-looking strings are the same product.
This is a narrower task than several neighboring ones. It is not about long model-code title links, product-family variant markup, URL canonicalization or RFQ text length. Those decisions have their own objects and their own evidence. Here the object is a single proposed string transformation and whether it would merge distinct business identifiers.
The second check is to identify which equivalence the proposed transformation actually uses. The standard distinguishes NFD and NFC, which use canonical mappings, from NFKD and NFKC, which also use compatibility mappings. The D forms decompose; the C forms decompose and then apply canonical composition. The difference between the C and KC forms depends on whether the resulting text is meant to be canonically equivalent or compatibility equivalent to the original.
That difference is not cosmetic for a product identifier. A canonical form preserves distinctions that compatibility forms may fold away. The standard cautions against applying the KC and KD forms indiscriminately: they can discard meaningful formatting or semantic distinctions, and conversion back to some legacy encodings may no longer reproduce the original. A product code is exactly the kind of text where a removed distinction can carry meaning.
The practical question for the reviewer is therefore: does the proposed transformation claim only canonical equivalence, or does it also fold compatibility variants? If the proposal does not state which form it applies, that is an unresolved dependency, not a detail to settle later. The cited standard is available at the Unicode normalization report for anyone who wants to read the definitions directly.
One caution belongs here rather than in a separate chapter. Normalization addresses formal character equivalence; it does not resolve every visual confusable, and it does not automatically identify equal SKUs. A transformation that passes a string comparison has cleared a text test, not a catalog test.
Related reading: Our Checks for Product Variants in AI Search
The comparison record is where the review becomes checkable. The table below is a proposed field set, not an observed catalog and not a filled example. Its purpose is to keep the authoritative identifier visible next to whatever derived value the transformation produces, so a reviewer can see both at once.
| Proposed field | What it should capture |
|---|---|
| Authoritative source identifier | The original string as held by the owning system, unmodified |
| Proposed transformation and form | The operation applied and the normalization form it claims |
| Derived lookup value | The string the transformation produces |
| Potential collision | Any other source record that maps to the same derived value |
| Confirmed business relation | Whether the catalog owner has confirmed the records are the same item |
| Review state | Open, confirmed or rejected, with the reviewer recorded |
The reason the source string stays in the record is the warning already cited: compatibility forms can remove distinctions that matter to the semantics of the text. If the derived value is stored alone, the removed distinction is no longer recoverable from that record, and the reviewer has lost the evidence needed to judge the case.
The record should not be filled with invented codes to demonstrate the method. An empty template that the team populates from its own catalog is more useful than a plausible-looking sample, because the sample would carry no authority and could be mistaken for a real identifier.
A collision is the case where two distinct source records map to the same derived lookup value. The correct response is to record it and route it to the catalog owner. It is not an instruction to merge the records, and it is not evidence that the records are duplicates.
The reasoning follows from the two equivalence types. Canonical equivalence covers characters that represent the same abstract character and should display identically; compatibility equivalence covers forms that may have distinct visual appearances or behaviors. A collision produced by a compatibility-folding transformation may therefore be an artifact of the transformation rather than a real business relationship.
The recommended handling is to preserve both source records, keep the derived value as a lookup aid rather than a replacement, and require business evidence before any item merge. If the catalog owner cannot confirm the relationship, the collision stays open. This article does not prescribe a universal storage form for identifiers; that choice depends on the catalog’s own constraints and should be made with the owning system in view.
Once the transformation is accepted for internal lookup, a separate question remains: which value does the buyer see on the page? A derived lookup string is not automatically the public product identifier. The public identifier should trace back to the same approved source that the catalog owner signed off.
The cited search guidance recommends making sure that structured data matches the visible text on the page. Applied here, that means the identifier shown to a buyer and any identifier declared in markup should come from the same approved source, rather than one being the source string and the other a transformed lookup value.
The same guidance recommends making sure that important content is available in textual form. For an identifier, that supports publishing the approved value as readable text rather than only inside an image or an unlabeled attribute. It does not establish how any AI engine tokenizes or matches identifiers, and nothing in this review should be read as a claim about that behavior.
If the site’s publishing structure is the open question rather than the identifier itself, the CMS-ready publishing structure described in SHMLANG’s website systems scope is the relevant starting point, because it addresses how pages are produced and maintained rather than how strings are compared.
The release decision should be expressed as stop states, so that an incomplete review has a defined outcome rather than drifting into a default. The transformation should not be released when the authoritative source identifier is unavailable, when the equivalence form is unknown, when the transformation removes distinctions the catalog treats as meaningful, or when business identity remains unresolved after review.
The third stop state has direct support. The standard warns that compatibility forms may remove distinctions that are important to the semantics of the text, and that they must not be blindly applied to arbitrary text. A catalog that cannot say whether a removed distinction mattered has not finished the review.
Keep this work separate from adjacent decisions. Text normalization is not canonical URL selection, not translation, and not product-family or variant ownership. Mixing them produces a release note that claims more than was checked.
A release also carries no visibility promise. The cited search guidance states that meeting all requirements, best practices and policies does not mean the page will be crawled, indexed or served. A documented, consistent identifier is a precondition the team controls; it is not a guarantee of any search or AI outcome.
Where the team wants the identifier work connected to a broader review, SHMLANG’s GEO service scope connects question baselines, approved business facts, useful pages and source research. That scope covers review and implementation of published facts; it does not extend to controlling how any AI system reads or matches a product code.
No comments yet. Be the first!