U bent hier

Story Needle

Abonneren op feed Story Needle
Content strategy for a post-device era
Bijgewerkt: 1 uur 7 min geleden

Supporting content compliance using Generative AI

21 maart 2024 - 11:53pm

Content compliance is challenging and time-consuming. Surprisingly, one of the most interesting use cases for Generative AI in content operations is to support compliance.

Compliance shouldn’t be scary

Compliance can seem scary. Authors must use the right wording lest things go haywire later, be it bad press or social media exposure, regulatory scrutiny, or even lawsuits. Even when the odds of mistakes are low because the compliance process is rigorous, satisfying compliance requirements can seem arduous. It can involve rounds of rejections and frustration.

Competing demands. Enterprises recognize that compliance is essential and touches more content areas, but scaling compliance is hard. Lawyers or other experts know what’s compliant but often lack knowledge of what writers will be creating. Compliance is also challenging for compliance teams. 

Both writers and reviewers need better tools to make compliance easier and more predictable.

Compliance is risk management for content

Because words are important, words carry risks. The wrong phrasing or missing wording can expose firms to legal liability. The growing volume of content places big demands on legal and compliance teams that must review that content. 

A major issue in compliance is consistency. Inconsistent content is risky. Compliance teams want consistent phrasing so that the message complies with regulatory requirements while aligning with business objectives.

Compliant content is especially critical in fields such as finance, insurance, pharmaceuticals, medical devices, and the safety of consumer and industrial goods. Content about software faces more regulatory scrutiny as well, such as privacy disclosures and data rights. All kinds of products can be required to disclose information relating to health, safety, and environmental impacts.  

Compliance involves both what’s said and what’s left unsaid. Broadly, compliance looks at four thematic areas:

  1. Truthfulness
    1. Factual precision and accuracy 
    2. Statements would not reasonably be misinterpreted
    3. Not misleading about benefits, risks, or who is making a claim
    4. Product claims backed by substantial evidence
  2. Completeness
    1. Everything material is mentioned
    2. Nothing is undisclosed or hidden
    3. Restrictions or limitations are explained
  3. Whether impacts are noted
    1. Anticipated outcomes (future obligations and benefits, timing of future events)
    2. Potential risks (for example, potential financial or health harms)
    3. Known side effects or collateral consequences
  4. Whether the rights and obligations of parties are explained
    1. Contractual terms of parties
    2. Supplier’s responsibilities
    3. Legal liabilities 
    4. Voiding of terms
    5. Opting out
Example of a proposed rule from the Federal Trade Commission source: Federal Register

Content compliance affects more than legal boilerplate. Many kinds of content can require compliance review, from promotional messages to labels on UI checkboxes. Compliance can be a concern for any content type that expresses promises, guarantees, disclaimers, or terms and conditions.  It can also affect content that influences the safe use of a product or service, such as instructions or decision guidance. 

Compliance requirements will depend on the topic and intent of the content, as well as the jurisdiction of the publisher and audience.  Some content may be subject to rules from multiple bodies, both governmental regulatory agencies and “voluntary” industry standards or codes of conduct.

“Create once, reuse everywhere” is not always feasible. Historically, complaince teams have relied on prevetted legal statements that appear at the footer of web pages or in terms and conditions linked from a web page. Such content is comparatively easy to lock down and reuse where needed.

Governance, risk, and compliance (GRC) teams want consistent language, which helps them keep tabs on what’s been said and where it’s been presented. Reusing the same exact language everywhere provides control.

But as the scope of content subject to compliance concerns has widened and touches more types of content, the ability to quarantine compliance-related statements in separate content items is reduced. Compliance-touching content must match the context in which it appears and be integrated into the content experience. Not all such content fits a standardized template, even though the issues discussed are repeated. 

Compliance decisions rely on nuanced judgment. Authors may not think a statement appears deceptive, but regulators might have other views about what constitutes “false claims.” Compliance teams have expertise in how regulators might interpret statements.  They draw on guidance in statutes, regulations, policies, and elaborations given in supplementary comments that clarify what is compliant or not. This is too much information for authors to know.

Content and compliance teams need ways to address recurring issues that need to be addressed in contextually relevant ways.

Generative AI points to possibilities to automate some tasks to accelerate the review process. 

Strengths of Generative AI for compliance

Generative AI may seem like an unlikely technology to support compliance. It’s best known for its stochastic behavior, which can produce hallucinations – the stuff of compliance nightmares.  

Compliance tasks reframe how GenAI is used.  GenAI’s potential role in compliance is not to generate content but to review human-developed content. 

Because content generation produces so many hallucinations, researchers have been exploring ways to use LLMs to check GenAI outputs to reduce errors. These same techniques can be applied to the checking of human-developed content to empower writers and reduce workloads on compliance teams.

Generative AI can find discrepancies and deviations from expected practices. It trains its attention on patterns in text and other forms of content. 

While GenAI doesn’t understand the meaning of the text, it can locate places in the text that match other examples–a useful capability for authors and compliance teams needing to make sure noncompliant language doesn’t slip through.  Moreover, LLMs can process large volumes of text. 

GenAI focuses on wording and phrasing.  Generative AI processes sequences of text strings called tokens. Tokens aren’t necessarily full words or phrases but subparts of words or phrases. They are more granular than larger content units such as sentences or paragraphs. That granularity allows LLMs to process text at a deep level.

LLMs can compare sequences of strings and determine whether two pairs are similar or not. Tokenization allows GenAI to identify patterns in wording. It can spot similar phrasing even when different verb tenses or pronouns are used. 

LLMs can support compliance by comparing text and determining whether a string of text is similar to other texts. They can compare the drafted text to either a good example to follow or a bad example to avoid. Since the wording is highly contextual, similarities may not be exact matches, though they consist of highly similar text patterns.

GenAI can provide an X-ray view of content. Not all words are equally important. Some words carry more significance due to their implied meaning. But it can be easy to overlook special words embedded in the larger text or not realize their significance.

Generative AI can identify words or phrases within the text that carry very specific meanings from a compliance perspective. These terms can then be flagged and linked to canonical authoritative definitions so that writers understand how these words are understood from a compliance perspective. 

Generative AI can also flag vague or ambiguous words that have no reference defining what the words mean in the context. For example, if the text mentions the word “party,” there needs to be a definition of what is meant by that term that’s available in the immediate context where the term is used.

GenAI’s “multimodal” capabilities help evaluate the context in which the content appears. Generative AI is not limited to processing text strings. It is becoming more multimodal, allowing it to “read” images. This is helpful when reviewing visual content for compliance, given that regulators insist that disclosures must be “conspicuous” and located near the claim to which they relate.

GenAI is incorporating large vision models (LVMs) that can process images that contain text and layout. LVMs accept images as input prompts and identify elements. Multimodal evaluations can evaluate three critical compliance factors relating to how content is displayed:

  1. Placement
  2. Proximity
  3. Prominence

Two writing tools suggest how GenAI can improve compliance.  The first, the Draft Analyzer from Bloomberg Law, can compare clauses in text. The second, from Writer, shows how GenAI might help teams assess compliance with regulatory standards.

Use Case: Clause comparison

Clauses are the atomic units of content compliance–the most basic units that convey meaning. When read by themselves, clauses don’t always represent a complete sentence or a complete standalone idea. However, they convey a concept that makes a claim about the organization, its products, or what customers can expect. 

While structured content management tends to focus on whole chunks of content, such as sentences and paragraphs, compliance staff focus on clauses–phrases within sentences and paragraphs.  Clauses are tokens.

Clauses carry legal implications. Compliance teams want to verify the incorporation of required clauses and to reuse approved wording.

While the use of certain words or phrases may be forbidden, in other cases, words can be used only in particular circumstances.  Rules exist around when it’s permitted to refer to something as “new” or “free,” for example.  GenAI tools can help writers compare their proposed language with examples of approved usage.

Giving writers a pre-compliance vetting of their draft. Bloomberg Law has created a generative AI plugin called Draft Analyzer that works inside Microsoft Word. While the product is geared toward lawyers drafting long-form contracts, its technology principles are relevant to anyone who drafts content that requires compliance review.

Draft Analyzer provides “semantic analysis tools” to “identify and flag potential risks and obligations.”   It looks for:

  • Obligations (what’s promised)
  • Dates (when obligations are effective)
  • Trigger language (under what circumstances the obligation is effective)

For clauses of interest, the tool compares the text to other examples, known as “precedents.”  Precedents are examples of similar language extracted from prior language used within an organization or extracted examples of “market standard” language used by other organizations.  It can even generate a composite standard example based on language your organization has used previously. Precedents serve as a “benchmark” to compare draft text with conforming examples.

Importantly, writers can compare draft clauses with multiple precedents since the words needed may not match exactly with any single example. Bloomberg Law notes: “When you run Draft Analyzer over your text, it presents the Most Common and Closest Match clusters of linguistically similar paragraphs.”  By showing examples based on both similarity and salience, writers can see if what they want to write deviates from norms or is simply less commonly written.

Bloomberg Law cites four benefits of their tool.  It can:

  • Reveal how “standard” some language is.
  • Reveal if language is uncommon with few or no source documents and thus a unique expression of a message.
  • Promote learning by allowing writers to review similar wording used in precedents, enabling them to draft new text that avoids weaknesses and includes strengths.
  • Spot “missing” language, especially when precedents include language not included in the draft. 

While clauses often deal with future promises, other statements that must be reviewed by compliance teams relate to factual claims. Teams need to check whether the statements made are true. 

Use Case: Claims checking

Organizations want to put a positive spin on what they’ve done and what they offer. But sometimes, they make claims that are subject to debate or even false. 

Writers need to be aware of when they make a contestable claim and whether they offer proof to support such claims.

For example, how can a drug maker use the phrase “drug of choice”? The FDA notes: “The phrase ‘drug of choice,’ or any similar phrase or presentation, used in an advertisement or promotional labeling would make a superiority claim and, therefore, the advertisement or promotional labeling would require evidence to support that claim.” 

The phrase “drug of choice” may seem like a rhetorical device to a writer, but to a compliance officer, it represents a factual claim. Rhetorical phrases can often not stand out as facts because they are used widely and casually. Fortunately, GenAI can help check the presence of claims in text.

Using GenAI to spot factual claims. The development of AI fact-checking techniques has been motivated by the need to see where generative AI may have introduced misinformation or hallucinations. These techniques can be also applied to human written content.

The discipline of prompt engineering has developed a prompt that can check if statements make claims that should be factually verified.  The prompt is known as the “Fact Check List Pattern.”  A team at Vanderbilt University describes the pattern as a way to “generate a set of facts that are contained in the output.” They note: “The user may have expertise in some topics related to the question but not others. The fact check list can be tailored to topics that the user is not as experienced in or where there is the most risk.” They add: “The Fact Check List pattern should be employed whenever users are not experts in the domain for which they are generating output.”  

The fact check list pattern helps writers identify risky claims, especially ones about issues for which they aren’t experts.

The fact check list pattern is implemented in a commercial tool from the firm Writer. The firm states that its product “eliminates [the] risk of ‘plausible BS’ in highly regulated industries” and “ensures accuracy with fact checks on every claim.”

Screenshot of Writer screenWriter functionality evaluating claims in an ad image. Source: VentureBeat

Writer illustrates claim checking with a multimodal example, where a “vision LLM” assesses visual images such as pharmaceutical ads. The LLM can assess the text in the ad and determine if it is making a claim. 

GenAI’s role as a support tool

Generative AI doesn’t replace writers or compliance reviewers.  But it can help make the process smoother and faster for all by spotting issues early in the process and accelerating the development of compliant copy.

While GenAI won’t write compliant copy, it can be used to rewrite copy to make it more compliant. Writer advertises that their tool can allow users to transform copy and “rewrite in a way that’s consistent with an act” such as the Military Lending Act

While Regulatory Technology tools (RegTech) have been around for a few years now, we are in the early days of using GenAI to support compliance. Because of compliance’s importance, we may see options emerge targeting specific industries. 

Screenshot Federal Register formats menuFormats for Federal Register notices

It’s encouraging that regulators and their publishers, such as the Federal Register in the US, provide regulations in developer-friendly formats such as JSON or XML. The same is happening in the EU. This open access will encourage the development of more applications.

– Michael Andrews

The post Supporting content compliance using Generative AI appeared first on Story Needle.

What’s the value of content previews?

18 maart 2024 - 2:52pm

Content previews let you see how your content will look before it’s published.  CMSs have long offered previews, but preview capabilities are becoming more varied as content management is increasingly decoupled from UI design and channel delivery. Preview functionality can introduce unmanaged complexity to content and design development processes.  

Discussions about previews can spark stong opinions.

Are content previews:

  1. Helpful?
  2. Unnecessary?
  3. A crutch used to avoid fixing existing problems?
  4. A source of follow-on problems?
  5. All of the above?

Many people would answer previews are helpful because they personally like seeing previews. Yet whether previews are helpful depends on more than individual preferences. In practice, all of the above can be true.  

It may seem paradoxical that a feature like previews can be good and bad. The contradiction exists only if one assumes all users and preview functionality are the same. Users have distinct needs and diverging expectations depending on their role and experience. How previews are used and who is impacted by them can vary widely. 

Many people assume previews can solve major problems authors face. Previews are popular because they promise to bring closure to one’s efforts. Authors can see how their content will look just before publishing it. Previews offer tangible evidence of one’s work. They bring a psychic reward. 

Yet many factors beyond psychic rewards shape the value of content previews. 

What you see while developing content and how you see it can be complicated. Writers are accustomed to word processing applications where they control both the words and their styling. But in enterprise content publishing, many people and systems become involved with wording and presentation. How content appears involves various perspectives. 

Content teams should understand the many sides of previews, from the helpful to the problematic.  These issues are becoming more important as content becomes uncoupled from templated UI design. 

Previews can be helpful 

Previews help when they highlight an unanticipated problem with how the content will be rendered when it is published. Consider situations that introduce unanticipated elements. Often, these will be people who are either new to the content team or who interact with the team infrequently. Employees less familiar with the CMS can be encouraged to view the preview to confirm everything is as expected.  Such encouragement allows the summer intern, who may not realize the need to add an image to an article, to check the preview to spot a gap.  

Remember that previews should never be your first line of defense against quality problems. Unfortunately, that’s often how previews are used: to catch problems that were invisible authors and designers when developing the content or the design.

Previews can be unnecessary 

Previews aren’t really necessary when writers create routine content that’s presented the same way each time.  Writers shouldn’t need to do a visual check of their writing and won’t feel the need to do so provided their systems are set up properly to support them. They should be able to see and correct issues in their immediate work environment rather than seesaw to a preview. Content should align with the design automatically. It should just work.

In most cases, it’s a red flag if writers must check the visual appearance of their work to determine if they have written things correctly. The visual design should accommodate the information and messages rather than expect them to adapt to the design. Any constraints on available space should be predefined rather than having writers discover in a preview that the design doesn’t permit enough space. Writers shouldn’t be responsible to ensuring the design can display their content properly. 

The one notable exception is UX writing, where the context in which discrete text strings appear can sometimes shape how the wording needs to be written. UX writing is unique because the content is highly structured but infrequently written and revised, meaning that writers are less familiar with how the content will display. For less common editorial design patterns, previews help ensure the alignment of text and widgets. However, authors shouldn’t need previews routinely for highly repetitive designs, such as those used in e-commerce.

None of the above is to say a preview shouldn’t be available; only that standard processes shouldn’t rely on checking the preview. If standard content tasks require writers to check the preview, the CMS setup is not adequate. 

Previews can be a crutch 

Previews are a crutch when writers rely on them to catch routine problems with how the content is rendered. They become a risk management tool and force writers to play the role of risk manager. 

Many CMSs have clunky, admin-like interfaces that authors have trouble using. Vendors, after all, win tenders by adding features to address the RFP checklist, and enterprise software is notorious for its bad usability (conferences are devoted to this problem).  The authoring UI becomes cluttered with distracting widgets and alerts.  Because of all the functionality, vendors use “ghost menus” to keep the interface looking clean, which is important for customer demos. Many features are hidden and thus easy for users to miss, or they’ll pop up and cover over text that users need to read.  

The answer to the cluttered UI or the phantom menus is to offer previews. No matter how confusing the experience of defining the content may be within the authoring environment, a preview will provide a pristine view of how the content will look when published.  If any problems exist, writers can catch them before publication. If problems keep happening, it becomes the writer’s fault for not checking the preview thoroughly and spotting the issue.

At its worst, vendors promote previews as the solution to problems in the authoring environment. They conclude writers, unlike their uncomplaining admin colleagues, aren’t quite capable enough to use UIs and need to see the visual appearance. They avoid addressing the limitations of the authoring environment, such as:

  • Why simple tasks take so many clicks 
  • Why the UI is so distracting that it is hard to notice basic writing problems
  • Why it’s hard to know how long text or what dimensions images should be

Writers deserve a “focus” mode in which secondary functionality is placed in the background while writers do essential writing and editing tasks. But previews don’t offer a focus mode – they take writers away from their core tasks. 

Previews can cause follow-on problems

Previews can become a can of worms when authors use them to change things that impact other teams. The preview becomes the editor and sometimes a design tool. Unfortunately, vendors are embracing this trend.

Potential problems compound when the preview is used not simply to check for mistakes but as the basis for deciding writing, which can happen when:

  1. Major revisions happen in previews
  2. Writers rely on previews to change text in UI components 
  3. Writers expect previews to guide how to write content appearing in different devices and channels 
  4. Writers use previews to change content that appears in multiple renderings
  5. Writers use previews to change the core design substantially and undermine the governance of the user experience 

Pushing users to revise content in previews. Many vendors rely on previews to hide usability problems with the findability and navigation of their content inventory. Users complain they have difficulty finding the source content that’s been published and want to navigate to the published page to make edits. Instead of fixing the content inventory, vendors encourage writers to directly edit in the preview. 

Editing in a preview can support small corrections and updates. But editing in previews creates a host of problems when used for extensive revisions, or multi-party edits because the authoring interface functionality is bypassed. The practices change the context of the task.  Revisions are no longer part of a managed workflow. Previews don’t display field validation or contextual cues about versioning and traceability.  It’s hard to see what changes have been made, who has made them, or where assets or text items have come from. Editing in context undermines content governance. 

Relying on previews to change text in UI components. Previews become a problem when they don’t map to the underlying content. More vendors are promoting what they call “hybrid” CMSs (a multi-headed hydra) that mix visual UI components with content-only components – confusingly, both are often called “blocks.” Users don’t understand the rendering differences in these different kinds of components. They check the preview because they can’t understand the behavior of blocks within the authoring tool. 

When some blocks have special stylings and layouts while others don’t, it’s unsurprising that writers wonder if their writing needs to appear in a specific rendering. Their words become secondary to the layout, and the message becomes less important than how it looks. 

Expecting previews to guide how to write content appearing in different devices and channels. A major limitation of previews occurs when they are relied upon to control content appearing in different channels or sites. 

In the simplest case, the preview shows how content appears on different devices. It may offer a suggestive approximation of the appearance but won’t necessarily be a faithful rendering of the delivered experience to customers. No one, writers especially, can rely on these previews to check the quality of the designs or how content might need to change to work with the design.

Make no mistake: how content appears in context in various channels matters. But the place to define and check this fit is early in the design process, not on the fly, just before publishing the content. Multi-channel real-time previews can promote a range of bad practices for design operations.

Using previews to change content that appears in multiple renderings. One of the benefits of a decoupled design is that content can appear in multiple renderings. Structured writing interfaces allow authors to plan how content will be used in various channels. 

We’ve touched on the limitations of previews of multiple channels already.  But consider how multi-channel previews work with in-context editing scenarios.  Editing within a preview will  focus on a single device or channel and won’t highlight that the content supports multiple scenarios. But any editing of content in one preview will influence the content that appears in different sites or devices. This situation can unleash pandemonium.

When an author edits content in a preview but that content is delivered to multiple channels, the author has no way of knowing how their changes to content will impact the overall design. Authors are separated from the contextual information in the authoring environment about the content’s role in various channels. They can’t see how their changes will impact other channels.

Colleagues may find content that appears in a product or website they support has been changed without warning by another author who was editing the content in a preview of a different rendering, unaware of the knock-on impact. They may be tempted to use the same preview editing functionality to revert to the prior wording. Because editing in previews undermines content governance, staff face an endless cycle of “who moved my cheese” problems. 

Using previews to substantially change the core design. Some vendors have extended previews to allow not just the editing of content but also the changing of UI layout and design. The preview becomes a “page builder” where writers can decide the layout and styling themselves. 

Unfortunately, this “enhancement“ is another example of “kicking the can” so that purported benefits become someone else’s problem. It represents the triumph of adding features over improving usability.

Writers wrestle control over layout and styling decisions that they dislike. And developers celebrate not having to deal with writers requesting changes.  But page building tries to fix problems after the fact.  If the design isn’t adequate, why isn’t it getting fixed in the core layout? Why are writers trying to fix design problems?

Previews as page builders can generate many idiosyncratic designs that undermine UX teams. UI designs should be defined in a tool like Figma, incorporated in a design system, and implemented in reusable code libraries available to all. Instead of enabling maturing design systems and promoting design consistency, page builders hurt brand consistency and generate long term technical debt.

Writers may have legitimate concerns about how the layout has been set up and want to change it. Page builders aren’t the solution. Instead, vendors must improve how content structure and UI components interoperate in a genuinely decoupled fashion. Every vendor needs to work on this problem.

Some rules of thumb
  • Previews won’t fix significant quality problems.
  • Previews can be useful when the content involves complex visual layouts in certain situations where content is infrequently edited. They are less necessary for loosely structured webpages or frequently repeated structured content.
  • The desire for previews can indicate that the front-end design needs to be more mature. Many design systems don’t address detailed scenarios; they only cover superficial, generic ones. If content routinely breaks the design, then the design needs refinement.
  • Previews won’t solve problems that arise when mixing a complex visual design with highly variable content. They will merely highlight them. Both the content model and design system need to become more precisely defined.
  • Previews are least risky when limited to viewing content and most risky when used to change content.
  • Preview issues aren’t new, but their role and behavior are changing. WYSIWYG desktop publishing metaphors that web CMS products adopted don’t scale. Don’t assume what seems most familiar is necessarily the most appropriate solution.

– Michael Andrews

The post What’s the value of content previews? appeared first on Story Needle.

Digital transformation for content workflows

7 maart 2024 - 5:15pm

Content workflows remain a manually intensive process. Content staff face the burden of deciding what to do and who should do it. How can workflow tools evolve to reduce burdens and improve outcomes? 

Content operations are arguably one of the most backward areas of enterprise business operations. They have been largely untouched by enterprise digital transformation. They haven’t “change[d] the conditions under which business is done, in ways that change the expectations of customers, partners, and employees” – even though business operations increasingly rely on online content to function. Compared with other enterprise functions, such as HR or supply chain management, content operations rely little on process automation or big data. Content operations depend on content workflow tools that haven’t modernized significantly.  Content workflow has become a barrier to digital transformation.

The missing flow 

Water flows seamlessly around any obstacle, downward toward a destination below.  Content, in contrast, doesn’t flow on its own. Content items get stuck or bounce around in no apparent direction. Content development can resemble a game of tag, where individuals run in various directions without a clear sense of the final destination.  Workflow exists to provide direction to content development.

Developing content is becoming more complex, but content workflow capabilities remain rudimentary. Workflow functionality has limited awareness of what’s happened previously or what should (or could) happen later. They require users to perform actions and decisions manually. They don’t add value.

Workflow functionality has largely stayed the same over the years, whether in a CMS or a separate content workflow tool. Vendors are far removed from the daily issues the content creators face managing content that’s in development. All offer similar generic workflow functionality. They don’t understand the problem space.  

Vendors consider workflow problems to be people problems, not software problems. Because people are prone to be “messy” (as one vendor puts it), the problem the software aims to solve is to track people more closely. 

To the extent workflow functionality has changed in the past decade, it has mainly focused on “collaboration.” The vendor’s solution is to make the workflow resemble the time-sucking chats of social media, which persistently demand one’s attention. By promoting open discussion of any task, tools encourage the relitigation of routine decisions rather than facilitating their seamless implementation. Tagging people for input is often a sign that the workflow isn’t clear. Waiting on responses from tagged individuals delays tasks. 

End users find workflow tools kludgy. Workflows trigger loads of notifications, which result in notification fatigue and notification blindness. Individuals can be overwhelmed by the lists and messages that workflow tools generate. 

Authors seek ways to compensate for tool limitations. Teams often supplement CMS workflow tools with project management tools or spreadsheets. Many end users skirt the built-in CMS workflow by avoiding optional features. 

Workflow optimization—making content workflows faster and easier—is immature in most organizations. Ironically, writers are often more likely to write about improving other people’s workflows (such as those of their customers or their firm’s products and services) than to dedicate time to improving their own content workflows.  

Content workflows must step up to address growing demands.  The workflow of yesterday needs reimagining.

Deane Barker wrote in his 2016 book on content management: “Workflow is the single most overpurchased aspect of any CMS…I fully believe that 95% of content approvals are simple, serial workflows, and 95% of those have a single step.”

Today, workflow is not limited to churning out simple static web pages. Content operations must coordinate supply chains of assets and copy, provide services on demand, create variants to test and optimize, plan delivery across multiple channels, and produce complex, rich media. 

Content also requires greater coordination across organizational divisions. Workflows could stay simple when limited to a small team. But as enterprises work to reduce silos and improve internal integration, workflows have needed to become more sophisticated. Workflows must sometimes connect people in different business functions, business units, or geographic regions. 

Current content workflows are hindered by:

  • Limited capabilities, missing features, and closed architectures that preclude extensions
  • Unutilized functionality that suffers from poor usability or misalignment with work practices

Broken workflows breed cynicism. Because workflow tools are cumbersome and avoided by content staff, some observers conclude workflow doesn’t matter. The opposite is true: workflows are more consequential than ever and must work better. 

While content workflow tools have stagnated, other kinds of software have introduced innovations to workflow management. They address the new normal: teams that are not co-located but need to coordinate distinct responsibilities. Modern workflow tools include IT service management workflows and sophisticated media production toolchains that coordinate the preproduction, production, and postproduction of rich media.

What is the purpose of a content workflow?

Workflow isn’t email. Existing workflow tools don’t solve the right problems. They are tactical solutions focused on managing indicators rather than substance. They reflect a belief that if everyone achieves a “zero inbox” with no outstanding tasks, then the workflow is successful.  But a workflow queue shouldn’t resemble an email box stuffed with junk mail, unsolicited requests, and extraneous notices, with a few high-priority action items buried within the pile. Workflows should play a role in deciding what’s important for people to work on.

Don’t believe the myth that having a workflow is all that’s needed. Workflow problems stem from the failure to understand why a workflow is necessary. Vendors position the issue as a choice of whether or not to have a workflow instead of what kind of workflow enterprises should have.  

Most workflow tools focus on tracking content items by offering a fancy checklist. The UI covers up an unsightly sausage-making process without improving it. 

Many tools prioritize date tracking. They equate content success with being on time. While content should be timely, its success depends on far more than the publication date and time. 

A workflow in itself doesn’t ensure content quality. A poorly implemented workflow can even detract from quality, for example, by specifying the wrong parties or steps. A robust workflow, in contrast, will promote consistency in applying best practices.  It will help all involved with doing things correctly and making sound decisions.  

As we shall see, workflow can support the development of high-quality content if it:

  • Validates the content for correctness
  • Supports sound governance

A workflow won’t necessarily make content development more productive. Workflows can be needlessly complex, time-consuming, or confusing. They are often not empowering and don’t allow individuals to make the best choices because they constrain people in counterproductive ways.  

Contrary to common belief, the primary goal of workflow should not be to track the status of content items. If all a workflow does is shout in red that many tasks are overdue, it doesn’t help. They behave like airport arrival and departure boards that tell you flights are delayed without revealing why.  

Status-centric workflow tools simply present an endless queue of tasks with no opportunity to make the workload more manageable. 

Workflows should improve content quality and productivity.  Workflow tools contribute value to the extent they make the content more valuable. Quality and productivity drive content’s value. 

Yet few CMS workflow tools can seriously claim they significantly impact either the quality or productivity of the content development process. Administratively focused tools don’t add value.

Workflow tools should support people and goals –  the dimensions that ultimately shape the quality of outcomes. Yet workflow tools typically delegate all responsibility to people to ensure the workflow succeeds. Administratively focused workflows don’t offer genuine support. 

A workflow will enhance productivity – making content more valuable relative to the effort applied – only if it: 

  • Makes planning more precise
  • Accelerates the completion of tasks
  • Focuses on goals, not just activities
Elements of content workflow Generic workflows presume generic tasks

Workflow tools fail to be “fit for purpose” when they don’t distinguish activities according to their purpose. They treat all activities as similar and equally important. Everything is a generic task: the company lawyer’s compliance review is no different than an intern’s review of broken links.  

Workflows track and forward tasks in a pass-the-batton relay. Each task involves a chain of dependencies. Tasks are assigned to one or more persons. Each task has a status, which determines the follow-on task.

CMS workflow tools focus on configuring a few variables:

  • Stage in the process
  • Task(s) associated with a stage
  • Steps involved with a task
  • Assigned employees required to do a step or task
  • Status after completing a task
  • The subsequent task or stage

From a coding perspective, workflow tools implement a series of simple procedural loops. The workflow engine resembles a hampster wheel. 

Hamster wheelLike a hamster wheel, content workflow “engines” require manual pushing. Image: Wikimedia

A simple procedural loop would be adequate if all workflow tasks were similar. However, generic tasks don’t reflect the diversity of content work.

Content workflow tasks vary in multiple dimensions, involving differing priorities and hierarchies. Simple workflow tools flatten out these differences by designing for generic tasks rather than concrete ones. 

Variability within content workflows

Workflows vary because they involve different kinds of tasks.  Content tasks can be:

  • Cognitive (applying judgment)
  • Procedural (applying rules)
  • Clerical (manipulating resources) 

Tasks differ in the thought required to complete them.  Workflow tools commonly treat tasks as forms for users to complete.  They highlight discrete fields or content sections that require attention. They don’t distinguish between:

  1. Reflexive tasks (click, tap, or type)
  2. Reflective tasks (pause and think)

The user’s goal for reflexive tasks is to “Just do it” or “Don’t make me think.” They want these tasks streamlined as much as possible.  

In contrast, their goal for reflective tasks is to provide the most value when performing the task. They want more options to make the best decision. 

Workflows vary in their predictability. Some factors (people, budget, resources, priorities) are known ahead of time, while others will be unknown. Workflows should plan for the knowns and anticipate the unknowns.

Generic workflows are a poor way to compensate for uncertainty or a lack of clarity about how content should proceed. Workflows should be specific the content and associated business and technical requirements.  

Many specific workflows are repeatable. Workflows can be classified into three categories according to their frequency of use:

  1. Routine workflows 
  2. Ad hoc, reusable workflows
  3. Ad hoc, one-off workflows 

Routine workflows recur frequently. Once set, they don’t need adjustment. Because tasks are repeated often, routine workflows offer many opportunities to optimize, meaning they can be streamlined, automated, or integrated with related tasks. 

Ad hoc workflows are not predefined. Teams need to decide how to shape the workflow based on the specific requirements of a content type, subject matter, and ownership. 

Ad hoc workflows can be reusable. In some cases, teams might modify an existing workflow to address additional needs, either adding or eliminating tasks or changing who is responsible. Once defined, the new workflow is ready for immediate use. But while not routinely used, it may be useful again in the future, especially if it addresses occasional or rare but important requirements.  

Even when a content item is an outlier and doesn’t fit any existing workflow, it still requires oversight.  Workflow tools should make it easy to create one-off workflows. Ideally, generative AI could help employees state in general terms what tasks need to be done and who should be involved, and a bot could define the workflow tasks and assignments.

Workflows vary in the timing and discretion of decisions.  Some are preset, and some are decided at the spur of the moment.  

Consider deadlines, which can apply to intermediate tasks in addition to the final act of publishing.  Workflow software could suggest the timing of tasks – when a task should be completed – according to the operational requirements. It might assign task due dates:

  • Ahead of time, based on when actions must be completed to meet a mandatory publication deadline. 
  • Dynamically, based on the availability of people or resources.

Similarly, decisions associated with tasks have different requirements. Content task decisions could be 

  • Rules-driven, where rules predetermine the decision   
  • Discretionary and dependent on the decisionmaker’s judgment.

Workflows for individual items don’t happen in isolation. Most workflows assume a discrete content item. But workflows can also apply to groups of related items.  

Two common situations exist where multiple content items will have similar workflows:

  • Campaigns of related items, where items are processed together
  • A series of related items, where items are processed serially

In many cases, the workflow for related items should follow the same process and involve the same people.  Tools should enable employees to reuse the same workflow for related items so that the same team is involved.

Does the workflow validate the content for correctness?

Content quality starts with preventing errors. Workflows can and should prevent errors from happening.  

Workflows should check for multiple dimensions of content correctness, such as whether the content is:

  • Accurate – the workflow draws on checks that dates, numbers, prices, addresses, and other details are valid.
  • Complete – the workflow checks that all required fields, assets, or statements are included.
  • Specific – the workflow accesses the most relevant specific details to include.
  • Up-to-date – the workflow validates that the data is the most recent available.
  • Conforming – the workflow checks that terminology and phrasing conform to approved usage.
  • Compliant – the workflow checks that disclaimers, warranties, commitments, and other statements meet legal and regulatory obligations.

Because performing these checks is not trivial, they are often not explicitly included in the workflow.  It’s more expeditious to place the responsibility for these dimensions entirely on an individual.  

Leverage machines to unburden users. Workflows should prevent obvious errors without requiring people to check themselves if an error is present. They should scrutinize text entry tasks to prevent input errors by including default or conditional values and auto-checking the formatting of inputs. In more ambiguous situations, they can flag potential errors that require an individual to look at. But they should never act too aggressively, where they generate errors through over-correction.

Error preemption is becoming easier as API integrations and AI tools become more prevalent. Many checks can be partially or fully automated by:

  • Applying logic rules and parameter-testing decision trees
  • Pulling information from other systems
  • Using AI pattern-matching capabilities 

Workflows must be self-aware. Workflows require hindsight and foresight. Error checking should be both reactive and proactive.  They must be capable of recognizing and remediating problems.

One of the biggest drivers of workflow problems is delays. Many delays are caused by people or contributions being unavailable because:

  • Contributors are overbooked or are away
  • Inputs are missing because they were never requested

Workflows should be able to anticipate problems stemming from resource non-availability.  Workflow tools can connect to enterprise calendars to know when essential people are unavailable to meet a deadline.  In such situations, it could invoke a fallback. The task could be reassigned, or the content’s publication could be a provisional release, pending final input from the unavailable stakeholder.

Workflows should be able to perform quality checks that transcend the responsibilities of a single individual to ensure these issues are not so dependent on one person. Before publication, it can monitor and check what’s missing, late, or incompatible. 

Automation promises to compress workflows but also carries risks. Workflows should check automation tasks in a staging environment to ensure they will perform as expected. Before making automation functionality generally available, the workflow staging will monitor discrete automation tasks and run batch tests on the automation of multiple items. Teams don’t want to discover that the automation they depend on doesn’t work when they have a deadline to meet. 

Does the workflow support sound governance?

Governance, risk, and compliance (GRC) are growing concerns for online publishers, particularly as regulators introduce more privacy, transparency, and online safety requirements. 

Governance provides reusable guidelines for performing tasks. It promotes consistency in quality and execution. It enables workflows to run faster and more smoothly by avoiding repeated questions about how to do things.  It ensures compliance with regulatory requirements and reduces reputation, legal, and commercial risks arising from a failure to vet content adequately.  

Workflow tools should promote three objectives:

  • Accountability (who is supposed to do what)
  • Transparency (what is happening compared to what’s supposed to happen)
  • Explainability (why tasks should be done in a certain way)

These qualities are absent from most content workflow functionality.

Defining responsibilities is not enough. At the most elemental level, a generic workflow specifies roles, responsibilities, and permissions. It controls access to content and actions, determining who is involved with a task and what they are permitted to do.  This kind of governance can prevent the wrong actors from messing up work, but they don’t help people responsible for the work from making unintended mistakes.

Assigned team members need support. The workflow should make it easier for them to make the correct decisions.  

Workflows should operationalize governance policies. However, if guidance is too intrusive, autocorrecting too aggressively, or making wrong assumptions, team members will try to short-circuit intrusive it.  

Discretionary decisions need guardrails, not enforcement. When a decision is discretionary, the goal should be to guide employees to make the most appropriate decision, not enforce a simple rule.  

Unfortunately, most governance guidance exists in documentation that is separated from workflow tools. Workflows fail to reveal pertinent guidance when it is needed. 

Incorporate governance into workflows at the point of decision. Bring guidance to the task so employees don’t need to seesaw between governance documents and workflow applications.  

Workflows can incorporate governance guidance in multiple ways by providing:

  • Guided decisions incorporating decision trees
  • Screen overlays highlighting areas to assess or check
  • Hints in the use interface
  • Coaching prompts from chatbots

When governance guidance isn’t specific enough for employees to make a clear decision, the workflow should provide a pathway to resolve the issue for the future. Workflows can include Issue management that triggers tasks to review and develop additional guidelines.

Does the workflow make planning more precise?

Bad plans are a common source of workflow problems.  Workflow planning tools can make tasks difficult to execute.

Planning acts like a steering wheel for a workflow, indicating the direction to go. 

Planning functionality is loosely integrated with workflow functionality, if at all. Some workflow tools don’t include planning, while those that do commonly detach the workflow from the planning.  

Planning and doing are symbiotic activities.  Planning functionality is commonly a calendar to set end dates, which the workflow should align with. 

But calendars don’t care about the resources necessary to develop the content. They expect that by choosing dates, the needed resources will be available.

Calendars are prevalent because content planning doesn’t follow a standardized process. How you plan will depend on what you know. Teams know some issues in advance, but other issues are unknown.  

Individuals will have differing expectations about what content planning comprises.  Content planning has two essential dimensions:

  • Task planning that emphasizes what tasks are required
  • Date planning that emphasizes deadlines

While tasks and dates are interrelated, workflow tools rarely give them equal billing.  Planning tools favor one perspective over the other.  

Task plans focus on lists of activities that need doing. The plan may have no dates associated with discrete tasks or have fungible dates that change.  One can track tasks, but there’s limited ability to manage the plan. Many workflows provide no scheduling or visibility into when tasks will happen.  At most, they show a Kanban board showing progress tracking.  They focus on if a task is done rather than when it should be done.

Design systems won’t solve workflow problems. Source: Utah design system

Date plans emphasize calendars. Individuals must schedule when various tasks are due. In many cases, those assigned to perform a task are notified in real time when they should do something. The due date drives a RAG (red-amber-green) traffic light indicator, where tasks are color-coded as on-track, delayed, or overdue based on dates entered in the calendar.

Manually selecting tasks and dates doesn’t provide insights into how the process will happen in practice.  Manual planning lacks a preplanning capability, where the software can help to decide in advance what tasks will be completed at specific times based on a forecast of when these can be done. 

Workflow planning capabilities typically focus on setting deadlines. Individuals are responsible for setting the publication deadline and may optionally set intermediate deadlines for tasks leading to the final deadline. This approach is both labor-intensive and prone to inaccuracies. The deadlines reflect wishes rather than realistic estimates of how long the process will take to complete. 

Teams need to be able to estimate the resources required for each task. Preplanning requires the workflow to: 

  1. Know all activities and resources that will be required  
  2. Schedule them when they are expected to happen.  

The software should set task dates based on end dates or SLAs. Content planning should resemble a project planning tool, estimating effort based on task times and sequencing—it will provide a baseline against which to judge performance.

For preplanning to be realistic, dates must be changeable. This requires the workflow to adjust dates dynamically based on changing circumstances. Replanning workflows will assess deadlines and reallocate priorities or assignments.

Does the workflow accelerate the completion of tasks?

Workflows are supposed to ensure work gets done on schedule. But apart from notifying individuals about pending dates, how much does the workflow tool help people complete work more quickly?  In practice, very little because the workflow is primarily a reminder system.  It may prevent delays caused by people forgetting to do a task without helping people complete tasks faster. 

Help employees start tasks faster with task recommendations. As content grows in volume, locating what needs attention becomes more difficult. Notifications can indicate what items need action but don’t necessarily highlight what specific sections need attention. For self-initiated tasks, such as evaluating groups of items or identifying problem spots, the onus is on the employee to search and locate the right items. Workflows should incorporate recommendations on tasks to prioritize.

Recommendations are a common feature in consumer content delivery. But they aren’t common in enterprise content workflows. Task recommendations can help employees address the expanding atomization of content and proliferation of content variants more effectively by highlighting which items are most likely relevant to an employee based on their responsibilities, recent activities, or organizational planning priorities.

Facilitate workflow streamlining. When workflows push manual activities from one person to another, they don’t reduce the total effort required by a team. A more data-driven workflow that utilizes semantic task tagging, by contrast, can reduce the number of steps necessary to perform tasks by:

  • Reducing the actions and actors needed 
  • Allowing multiple tasks to be done at the same time 

Compress the amount of time necessary to complete work. Most current content workflows are serial, where people must wait on others before being told to complete their assigned tasks. 

Workflows should shorten the path to completion by expanding the integration of: 

  1. Tasks related to an item and groups of related items
  2. IT systems and platforms that interface with the content management system

Compression is achieved through a multi-pronged approach:

  • Simplifying required steps by scrutinizing low-value, manually intensive steps
  • Eliminating repetition of activities through modularization and batch operations  
  • Involving fewer people by democratizing expertise and promoting self-service
  • Bringing together relevant background information needed to make a decision.

Synchronize tasks using semantically tagged workflows. Tasks, like other content types, need tags that indicate their purpose and how they fit within a larger model. Tags give workflows understanding, revealing what tasks are dependent on each other.  

Semantic tags provide information that can allow multiple tasks to be done at the same time. Tags can inform workflows:

  • Bulk tasks that can be done as batch operations
  • Tasks without cross-dependencies that can be done concurrently
  • Inter-related items be worked on concurrently

Automate assignments based on awareness of workloads. It’s a burden on staff to figure out to whom to assign a task. Often, task assignments are directed to the wrong individual, wasting time to reassign the task. Otherwise, the task is assigned to a generic queue, where the person who will do it may not immediately see it.  The disconnection between the assignment and the allocation of time to complete the task leads to delays.

The software should make assignments based on:

  • Job roles (responsibilities and experience) 
  • Employee availability (looking at assignments, vacation schedules, etc.) 

Tasks such as sourcing assets or translation should be assigned based on workload capacity. Content workflows need to integrate with other enterprise systems, such as employee calendars and reporting systems, to be aware of how busy people are and how is available.

Workload allocation can integrate rule-based prioritization that’s used in customer service queues. It’s common for tasks to back up due to temporary capacity constraints. Rule-based prioritization avoids finger-pointing. If the staff has too many requests to fulfill, there is an order of priority for requests in the backlog.  Items in backlog move up in priority according to their score, which reflects their predefined criticality and the amount of time they’ve been in the backlog. 

Automate routine actions and augment more complex ones. Most content workflow tools implement a description of processes rather than execute a workflow model, limiting the potential for automation. The system doesn’t know what actions to take without an underlying model.

A workflow model will specify automatic steps within content workflows, where the system takes action on tasks without human prompting. For example, the software can automate many approvals by checking that the submission matches the defined criteria. 

Linking task decisions to rules is a necessary capability. The tool can support event-driven workflows by including the parameters that drive the decision.

Help staff make the right decisions. Not all decisions can be boiled down to concrete rules. In such cases, the workflow should augment the decision-making process. It should accelerate judgment calls by making it easier for questions to be answered quickly.  Open questions can be tagged according to the issue so they can be cross-referenced with knowledge bases and routed to the appropriate subject matter expert.

Content workflow automation depends on deep integration with tools outside the CMS.  The content workflow must be aware of data and status information from other systems. Unfortunately, such deep integration, while increasingly feasible with APIs and microservices, remains rare. Most workflow tools opt for clunky plugins or rely on webhooks.  Not only is the integration superficial, but it is often counterproductive, where trigger-happy webhooks push tasks elsewhere without enabling true automation.

Does the workflow focus on goals, not just activities?

Workflow tools should improve the maturity of content operations. They should produce better work, not just get work done faster. 

Tracking is an administrative task. Workflow tracking capabilities focus on task completion rather than operational performance. With their administrative focus, workflows act like shadow mid-level managers who shuffle paper. Workflows concentrate on low-level task management, such as assignments and dates.

Workflows can automate low-level task activities; they shouldn’t force people to track them.   

Plug workflows’ memory hole. Workflows generally lack memory of past actions and don’t learn for the future. At most, they act like habit trackers (did I remember to take my vitamin pill today?) rather than performance trackers (how did my workout performance today compare with the rest of the week?)

Workflow should learn over time. It should prioritize tracking trends, not low-level tasks.

Highlight performance to improve maturity. While many teams measure the outcomes that content delivers, few have analytic tools that allow them to measure the performance of their work. 

Workflow analytics can answer: 

  • Is the organization getting more efficient at producing content at each stage? 
  • Is end-to-end execution improving?  

Workflow analytics can monitor and record past performance and compare it to current performance. They can reveal if content production is moving toward:

  • Fewer revisions
  • Less time needed by stakeholders
  • Fewer steps and redundant checks

Benchmark task performance. Workflows can measure and monitor tasks and flows, observing the relationships between processes and performance. Looking at historical data, workflow tools can benchmark the average task performance.

The most basic factor workflows should measure is the resources required. Each task requires people and time, which are critical KPIs relating to content production, 

Analytics can:

  1. Measure the total time to complete tasks
  2. Reveal which people are involved in tasks and the time they take.

Historic data can be used to forecast the time and people needed, which is useful for workflow planning. This data will also help determine if operations are improving.  

Spot invisible issues and provide actionable remediation.  It can be difficult for staff to notice systemic problems in complex content systems with multiple workflows. But a workflow system can utilize item data to spot recurring issues that need fixing.  

Bottlenecks are a prevalent problem. Workflows that are defined without the benefit of analytics are prone to develop bottlenecks that recur under certain circumstances. Solving these problems requires the ability to view the behavior of many similar items. 

Analytics can parse historical data to reveal if bottlenecks tend to involve certain stages or people. 

Historical workflow data can provide insights into the causes of bottlenecks, such as tasks that frequently involve:

  • Waiting on others
  • Abnormal levels of rework
  • Approval escalations

The data can also suggest ways to unblock dependencies through smart allocation of resources.  Changes could include:

  • Proactive notifications of forecast bottlenecks
  • Re-scheduling
  • Sifting tasks to an alternative platform that is more conducive

Utilize analytics for process optimization. Workflow tools supporting other kinds of business operations are beginning to take advantage of process mining and root cause analysis.  Content workflows should explore these opportunities.

Reinventing workflow to address the content tsunami

Workflow solutions can’t be postponed.  AI is making content easier to produce: a short prompt generates volumes of text, graphics, and video. The problem is that this content still needs management.  It needs quality control and organization. Otherwise, enterprises will be buried under petabytes of content debt.

Our twentieth-century-era content workflows are ill-equipped to respond to the building tsunami. They require human intervention in every micro decision, from setting due dates to approving wording changes. Manual workflows aren’t working now and won’t be sustainable as content volumes grow.

Workflow tools must help content professionals focus on what’s important. We find some hints of this evolution in the category of “marketing resource management” tools that integrate asset, work, and performance management. Such tools recognize the interrelationships between various content items and are expected to do.  

The emergence of no-code workflow tools, such as robotic process automation (RPA) tools, also points to a productive direction for content workflows. Existing content workflows are generic because that’s how they try to be flexible enough to handle different situations. They can’t be more specific because the barriers to customizing them are too high: developers must code each decision, and these decisions are difficult to change later. 

No code solutions give the content staff, who understand their needs firsthand, the ability to implement decisions about workflows themselves without help from IT. Enterprises can build a more efficient and flexible solution by empowering content staff to customize workflows.

Many content professionals advocate the goal of providing Content as a Service (CaaS).  The content strategist Sarah O’Keefe says, “Content as a Service (CaaS) means that you make information available on request.” Customers demand specific information at the exact moment they need it.  But for CaaS to become a reality, enterprises must ensure that the information that customers request is available in their repositories. 

Systemic challenges require systemic solutions. As workflow evolves to handle more involved scenarios and provide information on demand, it will need orchestration.  While individuals need to shape the edges of the system, the larger system needs a nervous system that can coordinate the activities of individuals.  Workflow orchestration can provide that coordination.

Orchestration is the configuration of multiple tasks (some may be automated) into one complete end-to-end process or job. Orchestration software also needs to react to events or activities throughout the process and make decisions based on outputs from one automated task to determine and coordinate the next tasks.”  

Orchestration is typically viewed as a way to decide what content to provide to customers through content orchestration (how content is assembled) and journey orchestration (how it is delivered).  But the same concepts can apply to the content teams developing and managing the content that must be ready for customers.  The workflows of other kinds of business operations embrace orchestration. Content workflows must do the same. 

Content teams can’t pause technological change; they must shape it.  A common view holds that content operations are immature because of organizational issues. Enterprises need to sort out the problems of how they want to manage their people and processes before they worry about technology. 

We are well past the point where we can expect technology to be put on hold while sorting out organizational issues. These issues must be addressed together. Other areas of digital transformation demonstrate that new technology is usually the catalyst that drives the restructuring of business processes and job roles. Without embracing the best technology can offer, content operations won’t experience the change it needs.

– Michael Andrews

The post Digital transformation for content workflows appeared first on Story Needle.

Orchestrating the assembly of content

31 januari 2024 - 4:51pm

Structured content enables online publishers to assemble pieces of content in multiple ways.  However, the process by which this assembly happens can be opaque to authors and designers. Read on to learn how orchestration is evolving and how it works.

To many people, orchestration sounds like jargon or a marketing buzzword. Yet orchestration is no gimmick. It is increasingly vital to developing, managing, and delivering online content. It transforms how publishers make decisions about content, bringing flexibility and learning to a process hampered in the past by short-term planning and jumbled, ad-hoc decisions.  

Revealing the hidden hand of orchestration

Orchestration is both a technical term in content management and a metaphor. Before discussing the technical aspects of orchestration, let’s consider the metaphor.  Orchestration in music is how you translate a tune into a score that involves multiple instruments that play together harmoniously. It’s done by someone referred to as an arranger, someone like Quincy Jones. As the New Yorker once wrote: “Everyone knows Quincy Jones’s name, even if no one is quite sure what he does. Jones got his start in the late nineteen-forties as a trumpeter, but he soon mastered the art of arranging jazz—turning tunes and melodies into written music for jazz ensembles.”

Much like music arranging, content orchestration happens off stage, away from the spotlight. It doesn’t get the attention given to UI design. Despite its stealthy profile, numerous employees in organizations become involved with orchestration, often through small-scale A/B testing by changing an image or a headline. 

Orchestration typically focuses on minor tweaks to content, often cosmetic changes. But orchestration can also address how to assemble content on a bigger scale. The emergence of structured content makes intricate, highly customized orchestration possible.

Content assembly requires design and a strategy. Few people consider orchestration when planning how content is delivered to customers. They generally plan content assembly by focusing on building individual screens or a collection of web pages on a website. The UI design dictates the assembly logic and reflects choices made at a specific time.  While the logic can change, it tends to happen only in conjunction with changes to the UI design. 

Orchestration allows publishers to specify content assembly independently of its layout presentation. It does so by approaching the assembly process abstractly: evaluating content pieces’ roles and purposes that address specific user scenarios.

Assembly logic is becoming distributed. Content assembly logic doesn’t happen in one place anymore. Originally, web teams created content for assembly into web pages using templates defined by a CMS on the backend. In the early 2000s, frontend developers devised ways to change the content of web pages presented in the browser using an approach known initially as Ajax, a term coined by the information architect Jesse James Garrett. Today, content assembly can happen at any stage and in any place. 

Assembly is becoming more sophisticated. At first, publishers focused on selecting the right web page to deliver. The pages were preassembled – often hand-assembled. Next, the focus shifted to showing or hiding parts of that web page by manipulating the DOM (document object model).  

Nowadays, content is much more dynamic. Many web pages, especially in e-commerce, are generated programmatically and have no permanent existence.  “Single page applications” (SPAs) have become popular, and the content will morph continuously. 

The need for sophisticated approaches for assembling content has grown with the emergence of API-accessible structured content. When content is defined semantically, rather than as web pages, the content units are more granular. Instead of simply matching a couple of web page characteristics, such as a category tag and a date, publishers now have many more parameters to consider when deciding what to deliver to a user.

Orchestration logic is becoming decoupled from applications. While orchestration can occur within a CMS platform, it is increasingly happening outside the CMS to take advantage of a broader range of resources and capabilities. With APIs growing in coordinating web content, much content assembly now occurs in a middle layer between the back-end storing the content and the front-end presenting it. The logic driving assembly is becoming decoupled from both the back-end and front-end. 

Publishers have a growing range of options outside their CMS for deciding what content to deliver.  Tools include:

  • Digital experience, composition, and personalization orchestration engines (e.g., Conscia, Ninetailed)
  • Graph query tools (e.g., PoolParty)
  • API federation management tools (e.g., Apollo Federation)

These options vary in their aims and motivations, and they differ in their implementations and features. Their capabilities are sometimes complementary, which means they can be used in combination. 

Orchestration inputs that frame the content’s context

Content structuring supports extensive variation in the types of content to present and what that content says. 

Orchestration involves more than retrieving a predefined web page.  It requires considering many kinds of inputs to deliver the correct details. 

Content orchestration will reflect three kinds of considerations:

  1. The content’s intent – the purpose of each content piece
  2. The organization’s operational readiness to satisfy a customer’s need
  3. The customer or user’s intent – their immediate or longer-term goal

Content characteristics play a significant role in assembly. Content characteristics define variations among and within content items. An orchestration layer will account for characteristics of available content pieces, such as:

  • Its editorial role and purpose, such as headings, explanations, or calls to action
  • Topics and themes, including specific products or services addressed
  • Intended audience or customer segment
  • Knowledge level such as beginner or expert
  • Intended journey or task stage
  • Language and locale
  • Date of creation or updating
  • Author or source
  • Size, length, or dimensions
  • Format and media
  • Campaign or announcement cycle
  • Product or business unit owner
  • Location information, such as cities or regions that are relevant or mentioned
  • Version 

Each of these characteristics can be a variable and useful when deciding what content to assemble. They indicate the compatibility between pieces and their suitability for specific contexts.

Other information in the enterprise IT ecosystem can help decide what content to assemble that will be most relevant for a specific context of use. This information is external to the content but relevant to its assembly.

Business data is also an orchestration input. Content addresses something a business offers. The assembled content should link to business operations to reflect what’s available accurately.

The assembled content will be contextually relevant only if the business can deliver to the customer the product or services that the content addresses. Customers want to know which pharmacy branches are open now or which items are available for delivery overnight.  The assembled content must reflect what the business can deliver when the customer seeks it.

The orchestration needs to combine content characteristics from the CMS with business data managed by other IT systems. Many factors can influence what content should be presented, such as:

  • Inventory management data
  • Bookings and orders data
  • Facilities’ capacity or availability
  • Location hours
  • Pricing information, promotions, and discount rules
  • Service level agreement (SLA) rules
  • Fulfillment status data
  • Event or appointment schedules
  • Campaigns and promotions schedule
  • Enterprise taxonomy structure defining products and operating units

Business data have complex rules managed by the IT system of record, not the CMS or the orchestration layer.  For content orchestration, sometimes it is only necessary to provide a “flag,” checking whether a condition is satisfied to determine which content option to show.

Customer context is the third kind of orchestration input. Ideally, the publisher will tailor the content to the customer’s needs – the aim of personalization.  The orchestration process must draw upon relevant known information about the customer: the customer’s context.

The customer context encompasses their identity and their circumstances. A customer’s circumstances can change, sometimes in a short time.  And in some situations, the customer’s circumstances dictate the customer’s identity. People can have multiple identities, for example, as consumers, business customers at work, or parents overseeing decisions made by their children.

Numerous dimensions will influence a customer’s opinions and needs, which in turn will influence the most appropriate content to assemble. Some common customer dimensions include:

  • Their location
  • Their personal characteristics, which might include their age, gender, and household composition, especially when these factors directly influence the relevance of the content, for example, with some health topics
  • Things they own, such as property or possessions, especially for content relating to the maintenance, insurance, or buying and selling of owned things
  • Their profession or job role, especially for content focused on business and professional audiences
  • Their status as a new, loyal, or churned customer
  • Their purchase and support history

The chief challenge in establishing the customer context is having solid insights.  Customers’ interactions on social media and with customer care provide some insights, but publishers can tap a more extensive information store.  Various sources of customer data could be available:

  • Self-disclosed information and preferences to the business (zero-party data or 0PD)
  • The history of a customer’s interactions with the business (first-party data or 1PD) 
  • Things customers have disclosed about themselves in other channels such as social media or survey firms (second-party data or 2PD)
  • Information about a cohort they are categorized as belonging to, using aggregated data originating from multiple sources (third-party data or 3PD)

Much of this information will be stored in a customer data platform (CDP), but other data will be sourced from various systems.  The data is valid only to the extent it is up-to-date and accurate, which is only sometimes a safe assumption.

Content behavior can shape the timing and details assembled in orchestration. Users can signal their intent through their interaction with content. The user’s decisions while interacting with content can signal their intentions.  Some behavior variables include:

  • Source of referral 
  • Previously viewed content 
  • Expressed interest in topics or themes based on prior content consumed
  • Frequency of repeat visits 
  • Search terms used 
  • Chatbot queries submitted
  • Subscriptions chosen or online events booked
  • Downloads or requests for follow-up information
  • The timing of their visit in relation to an offer 

The most valuable and reliable signals will be specific to the context. Many factors can shape intent, so many potential factors will not be relevant to individual customers. Just because some factors could be relevant in certain cases does not imply they will be applicable in most cases. 

Though challenging, leveraging customer intent offers many opportunities to improve the relevance of content. A rich range of possible dimensions is available. Selecting the right ones can make a difference. 

Don’t rely on weak signals to overdetermine intent. When the details about individual content behavior or motivations are scant, publishers sometimes rely on collective behavioral data to predict individual customer intentions.  While occasionally useful, predictive inputs about customers can be based on faulty assumptions that yield uneven results. 

Note the difference between tailoring content to match an individual’s needs and the practice of targeting. Targeting differs from personalization because it aims to increase average uptake rather than satisfy individual goals. It can risk alienating customers who don’t want the proffered content.

Draw upon diverse sources of input. By utilizing a separate layer to manage orchestration, publishers, in effect, create a virtual data tier that can federate and assess many distinct and independent sources of information to support decisions relating to content delivery. 

An orchestration layer gives publishers greater control over choosing the right pieces of content to offer in different situations. Publishers gain direct control over parameters to select,  unlike many AI-powered “decision engines” that operate like a black box and assume control over the content chosen.

The orchestration score

If the inputs are the notes in orchestration, the score is how they are put together – the arrangement. A rich arrangement will sometimes be simple but often will be sophisticated. 

Orchestration goes beyond web search and retrieval. In contrast to a ordinary web search, which retrieves a list of relevant web pages, orchestration queries must address many more dimensions. 

In a web search, there’s a close relationship between what is requested and what is retrieved. Typically, only a few terms need matching. Web search queries are often loose, and the results can be hit or miss. The user is both specifying and deciding what they want from the results retrieved.

In orchestration, what is requested is needs to anticipate what will be relevant and exclude what won’t be. The request may refer to metadata values or data parameters that aren’t presented in the content that’s retrieved. The results must be more precise. The user will have limited direct input into the request for assembled content and limited ability to change what is provided to them.

Unlike a one-shot web search process, in orchestration, content assembly involves a multistage process.  

The orchestration of structured content is not just choosing a specific web page based on a particular content type.  It differs in two ways:

  1. You may be combining details from two (or more) content types.  
  2. Instead of delivering a complete web page associated with each content type (and potentially needing to hide parts you don’t want to show), you select specific details from content items to deliver as an iterative procedure.

Unpacking the orchestration process. Content orchestration consists of three stages:

  1. FIND stage: Choose which content items have relevant material to support a user scenario
  2. MATCH stage: Combine content types that, if presented together, provide a meaningful, relevant experience
  3. SELECT and RETURN stage: Choose which elements within the content items will be most relevant to deliver to a user at a given point in time

Find relevant content items. Generally, this involves searching metadata tags such as taxonomy terms or specific values such as dates. Sometimes, specific words in text values are sought. If we have content about events, and all the event descriptions have a field with the date, it is a simple query to retrieve descriptions for events during a specified time period.

Typically, a primary content type will provide most of the essential information or messages. However, we’ll often also want to draw on information and messages from other content types to compose a content experience. We must associate different types of items to be able to combine their details.

Match companion content types. What other topics or themes will provide more context to a message? The role of matching is to associate related topics or tasks so that complementary information and messages can be included together.

Graph queries are a powerful approach to matching because they allow one to query “edges” (relationships) between “nodes” (content types.)  For example, if we know a customer is located in a specific city, we might want to generate a list of sponsors of events happening in that city.  The event description will have a field indicating the city. It will also reference another content type that provides a profile of event sponsors.  It might look like this in a graph query language like GQL, with the content types in round brackets and the relationships in square brackets.

MATCH (:Event WHERE location=”My City”) - [:SponsoredBy] -> (:SponsorProfile)

We have filtered events in the customer’s city (fictiously named My City) and associated content items about sponsors who have sponsored those events. Note that this query hasn’t indicated what details to present to users. It only identifies which content types would be relevant so that various types of details can be combined. 

Unlike in a common database query, what we are looking for and want to show are not the same. 

Select which details to assemble. We need to decide what information within a relevant content type which details will be of greatest interest to a user. Customers want enough details for the pieces to provide meaningful context. Yet they probably won’t want to see everything available, especially all at once – that’s the old approach of delivering preassembled web pages and expecting users to hunt for relevant information themselves.

Different users will want different details, necessitating decisions about which details to show. This stage is sometimes referred to as experience composition because the focus is on which content elements to deliver. We don’t have to worry about how these elements will appear on a screen, but we will be thinking about what specific details should be offered.

GraphQL, a query language used in APIs, is very direct in allowing you to specify what details to show. The GraphQL query mirrors the structure of the content so that one can decide which fields to show after seeing which fields are available. We don’t want to show everything about a sponsor, just their name, logo, and how long they’ve been sponsoring the event.  A hypothetical query named “local sponsor highlights” would extract only those details about the sponsor we want to provide in a specific content experience.

Query LocalSponsorHighlights {
… on SponsorProfile {
name
logo
sponsorSince
} }

The process of pulling out specific details will be repeated iteratively as customers interact with the content.

Turning visions into versions

Now that we have covered the structure and process of orchestration let’s look at its planning and design. Publishers enjoy a broad scope for orchestrating content. They need a vision for what they aim to accomplish. They’ll want to move beyond the ad hoc orchestration of page-level optimization and develop a scenario-driven approach to orchestration that’s repeatable and scaleable.

Consider what the content needs to accomplish. Content can have a range of goals. They can explicitly encourage a reader to do something immediately or in the future. Or they encourage a reader’s behavior by showing goodwill and being helpful enough that the customer wants to do something without being told what to do.

Content goalImmediate 
(Action outcome)Consequent
 (Stage outcome)Explicit 
(stated in the content)CTA (call to action) conversionContact sales or visit a retail outletImplicit 
(encouraged by the content)Resolve an issue without contacting customer supportRenew their subscription

Content goals must be congruent with the customer’s context. If customers have an immediate goal, then the content should be action-oriented. If their goal is longer-term, the content should focus on helping the customer move from one stage to another.

Orchestration will generate a version of the content representing the vision of what the pieces working together aim to accomplish.

Specify the context.  Break down the scenario and identify which contextual dimensions are most critical to providing the right content. The content should adapt to the user context, reflect the business context, and provide users with viable options. The context includes:

  • Who is seeking content (the segment, especially when the content is tailored for new or existing customers, or businesses or consumers, for example)
  • What they are seeking (topics, questions, requests, formats, and media)
  • When they are seeking it (time of day, day of the week, month, season, or holiday, all can be relevant)
  • Where they are seeking it (region, country, city, or facility such as an airport if relevant)
  • Why (their goal or intent as far as can be determined)
  • How (where they started their journey, channels used, how long have they pursuing task)
Perfecting the performance: testing and learning Leonard Bernstein conducts the New York Philharmonic in a Young People’s Concert. Image: Library of Congress


An orchestral performance is perfected through rehearsal. The performance realized is a byproduct of practice and improvisation.

Pick the correct parameters. With hundreds of parameters that could influence the optimal content orchestration, it is essential that teams not lock themselves into a few narrow ones. The learning will arise from determining which factors deliver the right experience and results in which circumstances. 

Content parameters can be of two kinds:

  1. Necessary characteristics tell us what values are required 
  2. Contingency characteristics indicate values to try to find which ones work best
Specifies in the orchestrationDetermines in the contentOutcome expectedNecessary characteristics

(tightly defined scenarios)What values are required in a given situationWhich categorical version or option the customer getsThe right details to show to a given customer in a given situationContingency characteristics

(loosely defined scenarios)What values are allowed in a given situationWhich versions could be presentedCandidate options to present to learn which most effectively matches the customer’s needs

The two approaches are not mutually exclusive. More complex orchestration (sometimes referred to as “multihop” queries) will involve a combination of both approaches.

Necessary characteristics reflect known and fixed attributes in the customer or business context that will affect the correct content to show. For example, if the customer has a particular characteristic, then a specific content value must be shown. The goal should be to test that the orchestration is working correctly – that the assumptions about the context are correct.  For example, there are no wrong assumptions or missing ones. This dimension is especially important for aspects that are fixed and non-negotiable. The content needs to adapt to these circumstances, not ignore them. 

Contingency characteristics reflect uncertain or changeable attributes relating to the customer’s context. For example, if the customer has had any one of several characteristics now or in the past, try showing any one of several available content values to see which works best given what’s known. The orchestration will prioritize variations randomly or in some ranked order based on what’s available to address the situation.

You can apply the approach to other situations involving uncertainty. When there are information gaps or delays, contingency characteristics can apply to business operations variables and to the content itself.  The goal of using contingency characteristics is to try different content versions to learn what’s most effective in various scenarios.  

Be clear on what content can influence. We have mostly looked at the customer’s context as an input into orchestration. Customers will vary widely in their goals, interests, abilities, and behaviors. A large part of orchestration concerns adapting content to the customer’s context. But how does orchestration impact the customer? In what ways might the customer’s context be the outcome of the content?

Consider how orchestration supports a shift in the customer’s context. Orchestration can’t change the fixed characteristics of the customer. It can sway ephemeral characteristics, especially content choices, such as whether the customer has requested further information.  And the content may guide customers toward a different context. 

Context shifting involves using content to meet customers where they are so they can get where they want to be. Much content exists to change the customer’s context by enabling them to resolve a problem or encouraging them to take action on something that will improve their situation. 

The orchestration of content needs to connect to immediate and downstream outcomes.  Testing orchestration entails looking at its effects on online content behavior and how it influences interactions with the business in other areas. Some of these interactions will happen offline.  

The task of business analytics is to connect orchestration outputs with customer outcomes. The migration of orchestration to an API layer should open more possibilities for insights and learning. 

– Michael Andrews

The post Orchestrating the assembly of content appeared first on Story Needle.

Content models: lessons from LEGO

9 januari 2024 - 10:46pm

Content models can be either a friend or foe.  A content model can empower content to become a modular and flexible resource when developed appropriately. Models can liberate how you develop content and accelerate your momentum.

However, if the model isn’t developed correctly, it can become a barrier to gaining control over your content.  The model ends up being hard to understand, the cause of delay, and eventually an albatross preventing you from pivoting later.  

Models are supposed to be the solution, not the problem

Content modeling can be challenging. Those involved with content modeling have likely heard stories about teams wrestling with their content model because it was too complex and difficult to implement. 

Some common concerns about content modeling include:

  • There’s not enough time to develop a proper content model.
  • We don’t know all our requirements yet.
  • People don’t understand why we need a content model or how to develop one.
  • Most of the content model doesn’t seem relevant to an individual writer.
  • Someone else should figure this out.

These concerns reflect an expectation that a content model is “one big thing” that needs to be sorted out all at once in the correct way, what might be called the monolithic school of content modeling. 

Rather than treat content models as monolithic plans, it is more helpful to think of them as behaving like LEGO. They should support the configuration of content in multiple ways.

Yet, many content models are designed to be monolithic. They impose a rigid structure on authors and prevent organizations from addressing a range of needs.  They become the source of stress because how they are designed is brittle.

In an earlier post, I briefly explored how LEGO’s design supports modularity through what’s called “clutch power.” LEGO can teach us insights about bringing modularity to content models. Contrary to what some believe, content models don’t automatically make content modular, especially when they are missing clutch power. But it’s true that content models can enable modularity. The value of a content model depends on its implementation. 

A complex model won’t simplify content delivery.  Some folks mistakenly think that the content model can encapsulate complexity that can then be hidden from authors, freeing them from the burdens of details and effort. That’s true only to a point.  When the model gets too complex for authors to understand and when it can’t easily be changed to address new needs, its ability to manage details deteriorates. The model imposes its will on authors rather than responding to the desire of authors.

The trick is to make model-making a modular process instead of a top-down, “here’s the spec, take it or leave it” approach. 

Don’t pre-configure your content

LEGO pioneered the toy category of multipurpose bricks. But over time, they have promoted the sale of numerous single-purpose kits, such as one for a typewriter that will “bring a touch of nostalgia to your home office.”  For $249.99, buyers get the gratification of knowing exactly what they will assemble before opening the package.  But they never experience the freedom of creating their own construction.

LEGO typewriter, 2079 pieces

The single-purpose kits contain numerous single-purpose bricks. The kit allows you to build one thing only. Certain pieces, such as the typewriter keys and a black and red ink spool ribbon, aren’t useful for other applications. When the meme gets stale, the bricks are no longer useful.

One of the most persistent mistakes in design is building for today’s problems with no forethought as to how your solution will accommodate tomorrow’s needs. 

Many content models are designed to be single-purpose.  They reflect a design frozen in time when the model was conceived – often by an outside party such as a vendor or committee. Authors can’t change what they are permitted to create, so the model will often be vague and unhelpful to not overly constrain the author. They miss out on the power of true modularity. 

When you keep your model fluid, you can test and learn.

Make sure you can take apart your model (and change it)

Bent Flyvbjer and Dan Gardner recently published a book on the success or failure of large projects called How Big Things Get Done.  They contrast viewing projects as “one big thing” versus “many small things.”

Flyvbjer and Gardner cite LEGO as a model for how to approach large projects. They say: “Get a small thing, a basic building block. Combine it with another and another until you have what you need.”

The textbook example they cite of designing “one big thing” all at once is a nuclear power plant.  

Flyvbjer and Gardner comment: “You can’t build a nuclear power plant quickly, run it for a while, see what works and what doesn’t, then change the design to incorporate the lessons learned. It’s too expensive and dangerous.”

Building a content model can seem like designing a nuclear power plant: a big project with lots of details that can go wrong. Like designing a nuclear power plant, building a content model is not something most of us do very often.  Most large content models are developed in response to a major IT re-platforming.  Since re-platformings are infrequent, most folks don’t have experience building content models, and when a re-platform is scheduled, they are unprepared. 

Flyvbjer and Gardner note that “Lacking experimentation and experience, what you learn as you proceed is that the project is more difficult and costly than you expected.” They note an unwanted nuclear power plant costs billions to decommission.  Similarly, a content model developed all at once in a short period will have many weaknesses and limitations. Some content types will be too simple, others will be too complex.  Some will be unnecessary ultimately, while others will be overlooked.  In the rush to deliver a project, it can be difficult to even reflect on how various decisions impact the overall project.

You will make mistakes with your initial content model and will learn what best supports authoring or frontend interaction needs, enables agile management of core business messages and data, or connects to other IT systems.  And these requirements will change too. 

Make sure you can change your mind.  Don’t let your IT team or vendor tell you the model is set in stone.  While gratuitous changes should be avoided – they generate superfluous work for everyone – legitimate revisions to a content model should be expected.  Technical teams need to develop processes that can address the need to add or remove elements in content types, create new types, and split or merge types. 

Allow authors to construct a range of outputs from basic pieces

In LEGO, the color of bricks gives them expressive potential. Even bricks of the same size and shape can be combined to render an experience.  I recently visited an exhibit of LEGO creations at the National Building Museum in Washington, DC.  

Mona Lisa in LEGO by Warren Elsmore at National Building Museum

Much online content is developed iteratively.  Writers pull together some existing text, modify or update it, add some images, and publish it.  Publishing is often a mixture of combining something old with something new.  A good content model will facilitate that process of composition. It will allow authors to retrieve parts they need, develop parts they don’t yet have, and combine them into a content item that people read, watch, or listen to.

Content modeling is fundamentally about developing an editorial schema.  Elements in content types represent the points authors want to make to audiences. The content types represent larger themes. 

A content model developed without the input of authors isn’t going to be successful. Ideally, authors will directly participate in the content modeling process. Defining a content type does not require any software expertise, and at least one CMS has a UI that allows non-technical users to create content types. However, it remains true that modeling is not easy to understand quickly for those new to the activity. I’m encouraged by a few vendor initiatives that incorporate AI and visualization to generate a model that could be edited. But vendors need to do more work to empower end-users so they can take more control over the content model they must rely on to create content. 

Keep logic out of your model 

LEGO is most brilliant when users supply the creativity rather than rely on instructions from the LEGO corporation.  

The DIY ethos of LEGO is celebrated on the Rebrickable website, where LEGO enthusiasts swap concepts for LEGO creations.

Rebrickable website

Rebrickable illustrates the principle of decoupling content from its assembly. LEGO bricks have an independent existence from assembly instructions. Those who develop plans for assembling LEGO are not the same people who created the bricks.  Bricks can be reused to be assembled in many ways – including ways not foreseen. Rebrickable celebrates the flexibility of LEGO bricks.

A content model should not contain logic. Logic acts like glue that binds pieces together instead of allowing them to be configured in different ways. Decouple your content from any logic used to deliver that content. Logic in content models gets in the way: it complicates the model, making it difficult for authors (and even developers) to understand, and it makes the model less flexible.  

Many Web CMS and XML content models mix together the structure of types with templating or assembly logic. When the model includes assembly instructions, it has already predetermined how modules are supposed to fit together, therefore precluding other ways they might connect. Content models for headless CMS implementations, in contrast, define content structure in terms of fields that can be accessed by APIs that can assemble content in any manner.  

A brittle content model that can’t be modified is also expensive.  Many publishers are saddled with legacy content models that are difficult to change or migrate. They hate what they have but are afraid to decommission it because they are unsure how it was put together. They are unable to migrate off of a solution someone built for them long ago that doesn’t meet their needs.  This phenomenon is referred to as “lock-in.”

A flexible content model will focus only on the content, not how to assemble the content. It won’t embed “views” or navigation or other details that dictate how the content must be delivered. When the model is focused solely on the content, the content is truly modular and portable.  

Focus on basics and learn

LEGO encourages play.  People don’t worry about assembling LEGO bricks the wrong way because there are many valid ways to connect them. There’s always the possibility of changing your mind.

Each LEGO brick is simple.  But by trying combinations over time, the range of possibilities grows.  As Flyvbjer and Gardner say in their book, “Repetition is the genius of modularity; it enables experimentation.”

Start your model with something basic. An author biography content type would be a good candidate. You can use it anytime you need to provide a short profile of an author. It seems simple enough, too.  A name, a photo, and a paragraph description might be all you need.  Congratulations, you have created a reusable content type and are on the path toward content modularity.

Over time, you realize there are other nuances to consider. Your website also features presenters at live events.  Is a presenter biography different from an author biography?  Someone in IT suggests that the author bio can be prepopulated with information from the employee directory, which includes the employee’s job title. The HR department wants to run profiles of employees on their hiring blog and wonders if the employee profile would be like the author bio.  

As new requirements and requests emerge, you start to consider the variations and overlaps among them. You might try to consolidate variations into a single content type, extend a core content type to handle specialized needs or decide that consolidating everything into a single content type wouldn’t simplify things at all. Only through experimentation and learning will the correct choice become apparent.

It’s a good idea to document your decisions and share what you’ve learned, including with outside teams who might be curious about what you are working on – encourage them to steal your ideas.  You can revisit your rationale when you are faced with new requirements to evaluate.

Evolve your model

Embrace the LEGO mindset of starting small and thinking big.

The more experience your team gains with content modeling, the more comprehensive and capable your content model will become.

Flyvbjer and Gardner note: “Repetition also generates experience, making your performance better. This is called ‘positive learning.’”  They contrast it with negative learning, where “the more you learn, the more difficult and costly it gets.” When the model starts off complex, it gets too big to understand and manage. Teams may realize only one person ever understood how the model was put together, and that person moved to another company. 

Content modeling is about harnessing repeating patterns in content. The job of content modeling is to discern these patterns.

Content modeling should be a continuous activity. A content model isn’t a punch-list task that, once launched, is done. The model isn’t frozen. 

The parts that work right will be second nature, while the parts that are still rough will suggest refinement.

While content modeling expertise is still far from mainstream, there are growing opportunities to gain it. Teams don’t have to wait for a big re-platforming project.  Instead, they should start modeling with smaller projects. 

New low-cost and/or low-code tools are making it easier to adopt modular approaches to content. Options include static site generators (SSGs), open-source headless CMSs like Strapi, and no-code web-builders like Webflow. Don’t worry if these tools don’t match your eventual needs.  If you build a model that supports true modularity, it will be easy to migrate your content to a more sophisticated tool later. 

With smaller projects, it will be more evident that the content model can change as you learn new things and want to improve it. Like other dimensions of software in the SaaS era, content models can continuously be released with improvements.  Teams can enhance existing content types and add new ones gradually. 

The evolution of content models will also include fixing issues and improving performance, similar to the refactoring process in software.  You may find that some elements aren’t much used and can be removed. You can simplify or enrich your model.  Sometimes you’ll find similar types that can be folded into a single type – simplification through abstraction.  Other times, you’ll want to extend types to accommodate details that weren’t initially identified as being important. 

Continual iteration may seem messy and inefficient, and in some ways, it is. Those who approach a content model as one big thing wish the model to be born fully mature, timeless, and resistant to aging. In practice, content models require nurture and maintenance. Changing details in a large content model will seem less scary once you’ve had experience making changes on a smaller model.  And by working on them continuously, organizations learn how to make them better serve their needs. 

Regard changes to content models as a learning experience, not a fearful one.

Allow your model to scale up

Flyvbjer and Gardner note: “A block of Lego is a small thing, but by assembling more than nine thousand of them, you can build one of the biggest sets Lego makes, a scale model of the Colosseum in Rome. That’s modularity.”

My visit to the National Building Museum revealed how big a scale small LEGO bricks can build, as shown in this model of London’s St Pancras rail station. 

St Pancras rail station in LEGO by Warren Elsmore at National Building Museum

The magic of LEGO is that all bricks can connect to one another. The same isn’t necessarily true of content models. Many models reflect the idiosyncrasies of specific CMS platforms.  Each model becomes an isolated island that can’t be connected to easily.  That’s why content silos are a pervasive problem.

However, a modular content model focused on content and not assembly logic can easily connect to other modular models. 

A content model can enable content to scale.  But how does one scale the content model?

The good news is that the same process for developing a small model applies to developing a larger one.  When you embrace an iterative, learning-driven approach to modeling, scaling your model is much easier.  You understand how models work: which decisions deliver benefits and which ones can cause problems. You understand tradeoffs and can estimate the effort involved with alternatives.

One key to scaling a model is to play well with other models. In large organizations, different teams may be developing content models to support their respective web properties.  If these models are modular, they can connect.  Teams can share content.

It’s likely when there are two models, there will be overlaps in content types. Each model will have a content type defining a blog post, for example. Such situations offer an opportunity to rationalize the content type and standardize it across the teams. Eventually, separate teams may use a common content model supporting a single CMS. But until then, they can at least be using the same content type specifications.  They can learn from each other.

– Michael Andrews

The post Content models: lessons from LEGO appeared first on Story Needle.

Making it easier to build structured content

17 december 2023 - 4:15pm


A quick note about an amazing UI from Writefull called Sentence Palette that shows how to combine structured content with prompts to write in a structured way. It nicely fleshes out the granular structure of a complex document down to the sentence level. I see broad application for this kind of approach for many kinds of content.

The deep analysis of large content corpora reveals common patterns not just in the conceptual organization of topics but also in the narrative patterns used to discuss them.

Writefull has assessed the academic writing domain and uncovered common text patterns used, for example, in article abstracts. LLMs can draw upon these text fragments to help authors develop new content. This opens up new possibilities for the development of structured content.

— Michael Andrews

The post Making it easier to build structured content appeared first on Story Needle.

How to compare CMSs, objectively

2 november 2023 - 12:32am

There are literally hundreds of CMSs on the market, possibly thousands. So much choice, but often so little satisfaction, judging by the gripes of end-users. Why is the right option so muddled? Old CMS vendors soldier on, and new ones enter the market all the time, promising a better future. How do we make sense of this?

A large part of the answer is the CMS buyer, and CMS users, are different people. The buyer and user have completely different relationships to the CMS. The buyer has either budgetary authority or responsibility for implementing the CMS. The buyer decides what to buy based on budget or infrastructure considerations. They dominate discussions of CMSs during the purchase phase but disappear afterward.

Only after the CMS is purchased do users gain much notice. They now have to “adopt” the CMS and be trained on how to use it. While they may not have had much say in what was purchased, they may nonetheless be hopeful their new solution will be better than the old one. After years of complaining, the user at last enjoys the spotlight. They get a new system and training. However, following a honeymoon period, users may notice the new system has many of the same issues as the one it replaced. Their CMS doesn’t satisfy their needs!

This conflict is formally known as a principal-agent problem.

CMSs are an Enterprise UX issue

CMSs are hardly unique in sparking user complaints. All kinds of enterprise software generate dissatisfaction. These problems stem from a common practice: the buyers of enterprise software are not the users of the software.

Do enterprises care about internal users? The field of enterprise UX emerged in response to a common situation: enterprise software is often less usable than consumer software. One explanation for why consumer software is better than enterprise software is that developers are unsure what consumers want so they test and iterate their designs to ensure people are willing to buy it. For enterprise software, the user base is considered a known and given quantity, especially if the enterprise application is being developed internally.

Enterprise software has changed dramatically over the past decade. It was once common for such software to be developed internally (“homegrown”), or else procured and installed on-premises (“off-the-shelf”). Either way, enterprise software was hard to change. Employees were expected to put up and shut up. Now, much enterprise software is SaaS. In principle, it should now be easier for enterprises to switch software, as firms shouldn’t be locked in. Usability should matter more now.

What’s good enough? Benchmarking usability. The most common usability benchmark is the System Usability Scale (SUS), which has been in use for four decades. Many software vendors, such as GitLab use SUS. A SUS survey yields a score from 0-100 that can be broken into “grades” that reveal how good the usability of the software is compared to other software, as this table from GitLab shows.

The SUS can be used to assess any type of software. It measures general usability, rather than the specific usability of a certain category of software. It matters little who has the best medical claims reconciliation software if all software in that category is below average compared to overall norms.

Employees aren’t consumers. It’s not straightforward to apply consumer usability practices to enterprise software. Many user experience assessment approaches, including the SUS to some degree, rely on measuring user preferences. The SUS asks users if they agree with the statement, “I think that I would like to use this product frequently.” Yet employees are required to use certain software — their preferences have no bearing on whether they use the software or not.

Microsoft, itself a vendor of enterprise software, recognizes the gap in enterprise usability assessment and outcomes. “Current usability metrics in the enterprise space often fail to align with the actual user’s reality when using technical enterprise products such as business analytics, data engineering, and data science software. Oftentimes, they lack methodological rigor, calling into question their generalizability and validity.” Two Microsoft researchers recently proposed a new assessment based on the SUS that focuses on enterprise users, the Enterprise System Usability Scale (ESUS).

The ESUS is readily applicable to assessing CMSs — what in the content strategy discipline is known as the authoring experience, which covers the editorial interface, workflow, analytics, content inventory management, and related end-user tasks. These tasks embody the essential purpose of the software: Can employees get their work done successfully?

ESUS consists of just five questions that cover major CMS issues:

  1. Usefulness – whether the CMS has required functionality and makes it possible to utilize it.
  2. Ease of use – whether the CMS is clear and allows tasks to be completed in a few clicks or steps.
  3. Control – whether the CMS empowers the user.
  4. Cohesion – do the CMS capabilities work together in an integrated manner?
  5. Learnability – can the user make use of the CMS without special training?

The ESUS, shown below, is elegantly simple.

ESUS Items12345How useful is this CMS to you? Not at all usefulSlightly usefulSomewhat usefulMostly usefulVery usefulHow easy or hard was this CMS to use for you?Very HardHardNeutralEasyVery EasyHow confident were you when using this CMS?Not at all confidentSlightly confidentSomewhat confidentMostly confidentVery confidentHow well do the functions work together or do not work together in this CMS?Does not work together at allDoes not work well togetherNeutralWorks well togetherWorks very well togetherHow easy or hard was it to get started with this CMS? Very HardHardNeutralEasyVery EasyMicrosoft’s proposed Enterprise System Usability Scale (ESUS) applied to CMS evaluation by employees How enterprises might use ESUS

The ESUS questionnaire provides quantitive feedback on the suitability of various CMSs, which can be compared.

Benchmark your current state. Enterprises should survey employees about their current CMSs. Benchmark current levels of satisfaction and compare different vendors. Most large enterprises use CMSs from more than one vendor.

Benchmark your desired state. It is also possible to use ESUS for pilot implementations — not vendor demos, but a realistic if limited implementation that reflects the company’s actual situation.

Measure and compare the strengths and weaknesses of different classes of CMSs and understand common tradeoffs. The more separate usability dimensions a vendor tries to maximize, the harder it gets. Much like the iron triangle of project management (the choice of only two priorities among scope, time, and budget), software products also face common tradeoffs. For example, a feature-robust CMS such as AEM can be a difficult-to-learn CMS. Is that tradeoff a given? The ESUS can tell us, using data from real users.

CMSs will vary in their usefulness. Some will have limited functionality, while others will be stuffed with so much functionality that usefulness is compromised. Does what’s out of the box match what users expect? It’s easy to misjudge this. Some vendors overprioritize “simplicity” and deliver a stymied product. Other vendors overemphasize “everythingness” – pretending to be a Swiss Army knife that does everything, if poorly.

CMS difficulty is…difficult to get right. But it matters. Not everyone finds the same things difficult. Developers will find some tasks less onerous than non-developers, for example. But everyone seems to agree when things are easy to do. That’s why consumer software is popular — rigorous user testing has de-bugged its problems, and everyone, no matter their tolerance for nuisance, benefits.

CMSs often fail to give users control — at some point. What’s interesting to look at is where the CMS falls down. Maybe the user feels in control when doing something simple and granular, but is overwhelmed when doing something involving many items at once or a complex task. Conversely, some CMSs are better at batch or exception tasks but impose a rigid process on everyone even to do basic tasks.

Simple CMSs may be coherent, but complex ones often aren’t. Every CMS will be compared to a word processor, which seems simple because it deals with one author at a time. It’s an unfair comparison; it ignores the many other tasks that CMSs support, such as analytics and workflow. But too many CMSs are pointlessly complex. They are mashups of functionality, the shotgun marriage of corporate divisions that don’t collaborate, separate products that were acquired and packaged as a suite, or collections of unrelated products patched together to provide missing functionality.

CMSs vary in their learnability. Some are so complicated that firms hire specialists just to manage the product. Other products require online “academies” to learn them — and possibly certifications to prove your diligence. Still others seem indistinguishable from everyday software we know already until one needs to do some that’s not every day.

Comparing CMSs quantitatively

Over the years, the CMS industry has splintered into more categories with shifting names. It’s become hard to compare CMSs because all want to seem special in their own way. Many categories have become meaningless and obscure what matters.

Remove the qualification “of.” Plenty of sites will claim to be arbiters of what’s best. Analyst firms create “Best of” lists of CMSs based on various criteria. What gets lost in this sorting and filtering is the sense that maybe everyone interested in content management wants many of the same things.

Some analysts focus on the vendor’s projection (positioning) as innovative or market-leading — qualities hard to define and compare. Some other sites rank vendors based on customer surveys, which can reflect whether the customer is in their honeymoon phase or has been incentivized to respond. While these resources can provide some useful information, they fail to provide feedback on things of interest to everyone, such as:

  1. Comparison of CMSs from vendors from different CMS categories
  2. Comparison of the usability of various CMSs

The ESUS can cut through the curation ring fence of “Best of” lists. It’s not beholden to arbitrary categories for classifying content management systems that can prevent comparison between them.

Aim for unfiltered comparison. Imagine if CMS users could get a direct answer to the question, Which CMS has better usability, overall: Adobe Experience Manager, Contentful, Wix, Drupal, WordPress, or Webflow? After all, all these products manage content. Let’s start here, with how well they do the basics.

Many folks would object that it’s an unfair question, like comparing apples with oranges. I believe those objections devalue the importance of usability. Every CMS user deserves good usability. And there’s no evidence that CMS users have different standards of usability — 40 years of SUS results tell us otherwise. Users all want the same experience — even when they want different functional details.

— Michael Andrews

The post How to compare CMSs, objectively appeared first on Story Needle.

Why content and design fall short of the LEGO ideal

3 oktober 2023 - 6:01pm

For many years, the people designing, managing, and delivering user experiences have pursued the LEGO ideal – making experiences modular. 

Content teams have aimed to make content modular so that it can be assembled in multiple ways. UI design teams have worked to make user interfaces modular so they can be assembled in different ways as well.  

More recently, vendors have embraced the LEGO ideal. The IT research firm Gartner labeled this modular approach as “composable” and now scores of SaaS firms endorse composability as the most desirable approach to building user experiences.

The LEGO ideal has become a defining North Star for many digital teams. 

The appeal of LEGO is easy to fathom. LEGO is familiar to almost everyone. 

Though LEGO was not the first construction kit toy that allowed parts to be connected in multiple ways, it has by far been the most successful.  LEGO is now the world’s largest toymaker.

But LEGO’s appeal stems from more than its popularity or the nostalgia of adults for pleasant childhood memories. LEGO teaches lessons about managing systems – though those lessons are often not well understood.

What LEGO figured out: Clutch Power 

What’s been the secret to LEGO’s success? Why has LEGO, more than any other construction toy, achieved sustained global success for decades?

Many people attribute LEGO’s success to the properties of the bricks themselves. The magic comes from how the bricks fit together,  

The Washington Post noted in 1983 the importance of  “the grip that holds one piece to another. Measurements have to be exact down to minute fractions of an inch, which requires high-precision machinery and closely monitored quality control.”

The ability of the bricks to fit together so well has a name: clutch power. 

The fan blog Brick Architect defines clutch power as “Newtons of force to attach or remove the part.”

The Washington Post noted that the bricks’ clutch power translated into “market clutch power”: how solidly the bricks built a grip with consumers.  

Clutch power makes bricks more powerful:

  1. Bricks can connect easily  –  they snap together
  2. Bricks can be disassembled easily by pulling them apart 
  3. Bricks are not damaged or deformed through their repeated use
  4. Bricks are infinitely reusable.

Clutch power is an apt metaphor for how brinks connect. Like the clutch in a car that shifts between gears, clutch power allows bricks of different sizes and roles to work together to deliver a bigger experience. 

What makes content and design LEGO-like?

Truth be told, most content and design modules don’t snap together like LEGOs. Content and design modules rarely exhibit clutch power.  

Even if the original intent was to create a LEGO-like kit of parts, the actual implementation doesn’t deliver a LEGO-like experience.  It’s important to move past the pleasing metaphor of LEGOs and explore what makes LEGOs distinctive.  

LEGO bricks aren’t for very small children – they are a choking hazard. Similarly, some teams figuratively “choke” when trying to manage many small content and design elements.  They are overwhelmed because they aren’t mature enough to manage the details.

Attempts to create modularity in content and design often fall short of the LEGO ideal. They resemble LEGO’s junior sibling, DUPLO, offering simple connections of a limited range of shapes.  In addition to generic bricks, DUPLO includes less general pieces such as specialized shapes and figures. It reduces the choking hazard but limits what can be built.

We find examples of DUPLO-like modularity in many UX processes. A small interaction pattern is reused, but it only addresses a very specific user journey such as a form flow. Small UI “molecules” are defined in design systems, but not more complex organisms. Help content gets structured, but not marketing or app content.

The limitation of DUPLO approach is the modularity isn’t flexible.  Teams can’t create multiple experiences from the pieces. 

When teams can’t create complex experiences out of small pieces, they resort to gluing the pieces together.  Pieces of content and design get glued together – their connections are forced, preventing them from being reused easily. The outputs become one-off, single-use designs that can’t be used for multiple purposes.

Some people glue together LEGO bricks, even though doing so is not considered an approved best practice. They create an edifice that is too fragile and too precious to change. Their design is too brittle to take advantage of the intrinsic clutch power of the bricks. They create a single-use design. They use modularity to build a monolith.

Digital teams routinely build monolithic structures from smaller pieces.  They create content templates or frontend design frameworks that look and behave a certain way but are difficult to change.  They build an IKEA product that can’t be disassembled when you need to move.

So gives content and design clutch power? What allows pieces to connect and be reconfigured?

The digital equivalent of clutch power is systems interface design – how the interfaces between various systems know of and can interact with each other. It determines whether the modules are created in a way that they are “API-first” so that other systems can use the pieces without having to interpret what’s available.  

More concretely, giving content and design modules clutch power involves defining them with models. Models show how pieces can fit together.,

Models define things (resources) and their relationships, highlighting that things have a rich set of potential connections. They can snap together in many ways, not just in one way.

Defining things and their relationships is not easy, which is why the LEGO ideal remains elusive for many digital teams.  It requires a combination of analytic and linguistic proficiency. Relationships are of two kinds:

  • Conceptual relationships that express the properties that things share with each other, which requires the ability to name and classify these properties clearly, at the right granularity (abstraction), to enable connection and comparison with appropriate precision.
  • Logical relationships that express the constraints and requirements of things and their values, which calls for the ability define what is normal, expected, exceptional, and prohibited from the perspective of multiple actors engaged in an open range of scenarios.

Modeling skills transcend the priorities of UI and content “design”, which focus on creating a product intended to support a single purpose. Modeling skills are more akin to engineering, without being cryptic. Modular pieces must be easy to connect, cognitively and procedurally. 

We sometimes find organizations hire content engineers, content architects, information architects, or UI engineers, but most often designers and developers are in charge of implementation.  We need more folks focused on creating clutch power.

What LEGO is still learning – and their lessons for digital teams

LEGO created a system that could grow. It expanded by offering new brick shapes that allow a wider range of items to be built.

LEGO has proved remarkably enduring. But that doesn’t mean it doesn’t need to adapt. To maintain market clutch power, LEGO needs to adapt to a changing market.  Its past formulas for success can no longer be relied upon. 

LEGO’s bricks are made from ABS plastic.  ABS plastic gives the bricks their clutch power. But they are also environmentally bad as they are petroleum-based and have a big carbon footprint.  As the world’s biggest toymaker, producing billions of plastic bricks, LEGO needs to change its model.

LEGO has tried to change the formula for their bricks.  They’ve sought to replace ABS with recycled polyethylene terephthalate (RPET) but found it too soft to provide the needed clutch power. Additives to RPET, which would make it safer and more durable, require too much energy consumption. After intensive research, LEGO is discovering there’s no simple substitute for ABS.

LEGO’s dilemma highlights the importance of creating a system that can adapt to changing priorities. It’s not that clutch power became less important.  But it had to fit in with new priorities of reducing carbon emissions.  

One option LEGO is looking at is how to enable the “recircling” of bricks. How can bricks in one household, when no longer needed, be re-entered into the economy? LEGO is looking at a “circular business model” for bricks.

A circular model is one that digital teams should look at as well. The aim should not just be how a team can reuse their content and design modules, but also how other parts of their organization can reuse them, and perhaps, how outside parties can use them too.  An API-first approach makes recirculation much easier. Better collaboration from vendors would also help.  

– Michael Andrews

The post Why content and design fall short of the LEGO ideal appeared first on Story Needle.

Bridging the divide between structured content and user interface design

25 september 2023 - 6:25pm

Decoupled design architectures are becoming common as more organizations embrace headless approaches to content delivery. Yet many teams encounter issues when implementing a decoupled approach. What needs to happen to get them unstuck?

Digital experts have long advocated for separating or decoupling content from its presentation. This practice is becoming more prevalent with the adoption of headless CMSs, which decouple content from UI design. 

Yet decoupling has been held back by UI design practices. Enterprise UX teams rely on design systems too much as the basis for organizing UIs, creating a labor-intensive process for connecting content with UI components.

Why decoupled design is hard

Decoupled design, where content and UI are defined independently, represents a radical break from incumbent practices used by design teams. Teams have been accustomed to defining UI designs first before worrying about the content. They create wireframes (or more recently, Figma files) that reflect the UI design, whether that’s a CMS webpage template or a mobile app interface.  Only after that is the content developed.

Decoupled design is still unfamiliar to most enterprise UX teams. It requires UX teams to change their processes and learn new skills. It requires robust conceptual thinking, proactively focusing on the patterns of interactions rather than reactively responding to highly changeable details.

The good news: Decoupling content and design delivers numerous benefits. A decoupled design architecture brings teams flexibility that hasn’t been possible previously. Content and UI design teams can each focus on their tasks without generating bottlenecks arising from cross-dependencies. UI designs can change without requiring the content be rewritten. UI designers can understand what content needs to be presented in the UI before they start their designs. Decoupling reduces uncertainty and reduces the iteration cycles associated with content and UI design changes needing to adjust to each other.

It’s also getting easier to connect content to UI designs. I have previously argued that  New tools, such as RealContent, can connect structured content in a headless CMS directly to a UI design in Figma. Because decoupled design is API-centric, UX teams have the flexibility to present content in almost any tool or framework they want.

The bad news: Decoupled design processes still require too much manual work. While they are not more labor intensive than existing practices, decoupled design still requires more effort than it should.

UI designers need to focus on translating content requirements into a UI design.  The first need to look at the user story or job to be done and translate that into an interaction flow. Then, they need to consider how users will interact with content on screen by screen. They need to map the UI components presented in each screen to fields defined in the content model 

When UX teams need to define these details, they are commonly starting from scratch. They map UI to the content model on a case-by-case basis, making the process slow and potentially inconsistent. That’s hugely inefficient and time-consuming.

Decoupled design hasn’t been able to realize its full potential because UX design processes need more robust ways of specifying UI structure. 

UI design processes need greater maturity

Design systems are limited in their scope. In recent years, much of the energy in UI design processes has centered around developing design systems. Design systems have been important in standardizing UI design presentations across products. They have accelerated the implementation of UIs.  

Design systems define specific UI components, allowing their reusability. 

But it’s essential to recognize what design systems don’t do. They are just a collection of descriptions of the UI components that are available for designers to use if they decide to. I’ve previously argued that Design systems don’t work unless they talk to content models.

Design systems, to a large extent, are content-agnostic. They are a catalog of empty containers, such as cards or tiles, that could be filled with almost anything. They don’t know much about the meaning of the content their components present, and they aren’t very robust in defining how the UI works. They aren’t a model of the UI. They are a style guide. 

Design systems define the UI components’ presentation, not the UI components’ role in supporting user tasks. They define the styling of UI components but don’t direct which component must be used. Most of these components are boxes constructed from CSS. 

Unstructured design is a companion problem to unstructured content. Content models arose because unstructured content is difficult for people and machines to manage. The same problem arises with unstructured UI designs.

Many UI designers mistakenly believe that their design systems define the structure of the UI. In reality, they define only the structure of the presentation: which box is embedded in another box.  While they sometimes contain descriptive annotations explaining when and how the component can be used, these descriptions are not formal rules that can be implemented in code. 

Cascading Style Sheets do not specify the UI structure; it only specifies the layout structure. No matter how elaborately a UI component layout is organized in CSS or how many layers of inheritance design tokens contain, the CSS does not tell other systems what the component is about

Designers have presumed that the Document Object Model in HTML structures the UI.  Yet, the structure that’s defined by the DOM is rudimentary, based on concepts dating from the 1990s, and cannot distinguish or address a growing range of UI needs. The DOM is inadequate to define contemporary UI structure, which keeps adding new UI components and interaction affordances. Although the DOM enables the separation of content from its presentation (styling), the DOM mixes content elements with functional elements. It tries to be both a content model and a UI model but doesn’t fulfill either role satisfactorily.

Current UIs lack a well-defined structure. It’s incredible that after three decades of the World Wide Web, computers can’t really read what’s on a webpage. Bots can’t easily parse the page and know with confidence the role of each section.  IT professionals who need to migrate legacy content created by people at different times in the same organization find that there’s often little consistency in how pages are constructed. Understanding the composition of pages requires manual interpretation and sleuthing. 

Even Google has trouble understanding the parts of web pages.  The problem is acute enough that a Google research team is exploring using machine vision to reverse engineer the intent of UI components.  They note the limits of DOMs: “Previous UI models heavily rely on UI view hierarchies — i.e., the structure or metadata of a mobile UI screen like the Document Object Model for a webpage — that allow a model to directly acquire detailed information of UI objects on the screen (e.g., their types, text content and positions). This metadata has given previous models advantages over their vision-only counterparts. However, view hierarchies are not always accessible, and are often corrupted with missing object descriptions or misaligned structure information.” 

The lack of UI structure interferes with the delivery of structured content. One popular attempt to implement a decoupled design architecture, the Blocks Protocol spearheaded by software designer Joel Spolsky, also notes the unreliability of current UI structures. “Existing web protocols do not define standardized interfaces between blocks [of content] and applications that might embed them.”

UI components should be machine-readable

Current UI designs aren’t machine-readable – they aren’t intelligible to systems that need to consume the code. Machines can’t understand the idiosyncratic terminology added to CSS classes. 

Current UIs are coded for rendering by browsers. They are not well understood by other kinds of agents.  The closest they’ve come is the addition of WAI-ARIA code that adds explicit role-based information to HTML tags to help accessibility agents interpret how to navigate contents without audio, visual, or haptic inputs and outputs. Accessibility code aims to provide parity in browser experiences rather than describe interactions that could be delivered outside of a browser context. Humans must still interpret the meaning of widgets and rely on browser-defined terminology to understand interaction affordances. 

The failure of frontend frameworks to declare the intent of UI components is being noticed by many parties.  UI needs a model that can specify the purpose of the UI component so that it can be connected to the semantic content model.  

A UI model will define interaction semantics and rules for the functional capabilities in a user interface. A UI model needs to define rules relating to the functional purpose of various UI components and when they must be used.  A UI model will provide a level of governance missing from current UI development processes, which rely on best-efforts adherence to design guidelines and don’t define UI components semantically. 

When HTML5 was introduced, many UI designers hailed the arrival of “semantic HTML.” But HTML tags are not an adequate foundation for a UI model. HTML tags are limited to a small number of UI elements that are overly proscriptive and incomplete.  HTML tags describe widgets like buttons rather than functions like submit or cancel. While historically, actions were triggered by buttons, that’s no longer true today.  Users can invoke actions using many UI affordances. UI designers may change UI element supporting an action from a button to a link if they change the context where the action is presented, for example. Hard-coding the widget name to indicate its purpose is not a semantic approach to managing UIs. This issue becomes more problematic as designers must plan for multi-modal interaction across interfaces. 

UI specifications must transcend the widget level. HTML tags and design system components fall short of being viable UI models because they specify UI instances rather than UI functions.  A button is not the only way for a user to submit a request. Nor is a form the only way for a user to submit input.

When a designer needs to present a choice to users, the design system won’t specify which UI component to use. Rather it will describe a range of widgets, and it is up to the designer to figure out how they want to present the choice.

Should user choices be presented as a drop-down menu? A radio button?  A slider? Design systems only provide descriptive guidance. The UI designer needs to read and interpret them. Rarely will the design system provide a rule based on content parameters, such as if the number of choices is greater than three, and the choice text is less than 12 characters, use a drop-down.  

UIs should be API-ready. As content becomes more structured, semantically defined, and queriable via APIs, the content needs the UI designs that present it to be structured, too. Content queries need to be able to connect to UI objects that will present the content and allow interaction with the content.  Right now, this is all done on an ad hoc basis by individual designers.

Let’s look at the content and UI sides from a structural perspective.

On the content side, a field may have a series of enumerated values: predefined values such as a controlled vocabulary, taxonomy terms, ordinal values, or numeric ranges. Those values are tracked and managed internally and are often connected to multiple systems that process information relating to the values. 

On the UI side, users face a range of constrained choices. They must pick from among the presented values. The values might appear as a pick list (or a drop-down menu or a spinner). The first issue, noted by many folks, is the naming problem in design systems. Some systems talk about “toasts,” while other systems don’t refer to them. UI components that are essentially identical in their outward manifestations can operate under different names. 

Why is this component used? The bigger structural problem is defining the functional purpose of the UI component.  The component chosen may change, but its purpose will remain persistent. Currently, UI components are defined by their outward manifestation rather than their purpose. Buttons are defined generically as being primary or secondary – expressed in terms of the visual attention they draw – rather than the kind of actions the button invokes (confirm, cancel, etc.)

Constrained choice values can be presented in multiple ways, not just as a drop-down menu.  It could be a slider (especially if values are ranked in some order) or even as free text where the user enters anything they wish and the system decides what is the closest match to enumerated values managed by the system.  

A UI model could define the component as a constrained value option. The UI component could change as the number of values offered to users changed. In principle, the component updating could be done automatically, provided there were rules in place to govern which UI component to use under which circumstances.  

The long march toward UI models

A design system specifies how to present a UI component: its colors, size, animation behaviors, and so on.  A UI model, in contrast, will specify what UI component to present: the role of the component (what it allows users to do) and the tasks it supports. 

Researchers and standards organizations have worked on developing UI models for the past two decades. Most of this work is little known today, eclipsed by the attention in UI design to CSS and Javscript frameworks.  

In the pre-cloud era, at the start of the millennium, various groups looked at how to standardize descriptions of the WIMP (windows, icons, menu, pointers) interface that was then dominant. The first attempt was Mozilla’s XUL. A W3C group drafted a Model-Based User Interfaces specification (MBUI).  Another coalition of IBM, Fujitsu, and others developed a more abstract approach to modeling interactions, the Software & Systems Process Engineering Meta-Model Specification.

Much of the momentum for creating UI models slowed down as UI shifted to the browser with the rise of cloud-based software. However, the need for platform-independent UI specification continues.

Over the past decade, several parties have pursued the development of a User Interface Description Language (UIDL).  “A User Interface Description Language (UIDL) is a formal language used in Human-Computer Interaction (HCI) in order to describe a particular user interface independently of any implementation….meta-models cover different aspects: context of use (user, platform, environment), task, domain, abstract user interface, concrete user interface, usability (including accessibility), workflow, organization, evolution, program, transformation, and mapping.”

Another group defines UIDL as “a universal format that could describe all the possible scenarios for a given user interface.”

Task and scenario-driven UI modeling. Source: OpenUIDL

Planning beyond the web. The key motivation has been to define the user interface independently of its implementation. But even recent work at articulating a UIDL has largely been web-focused. 

Providing a specification that is genuinely independent of implementation requires that it not be specific to any delivery channel.  Most recently, a few initiatives have sought to define a UI model that is channel agnostic.  

One group has developed OpenUIDL, “a user interface description language for describing omnichannel user interfaces with its semantics by a meta-model and its syntax based on JSON.”

UI models should work across platforms.  Much as content models have allowed content to be delivered to many channels via APIs, UI models are needed to specific user interaction across various channels. While responsive design has helped allow a design to adapt to different devices that use browsers, a growing range of content is not browser-based.  In addition to emerging channels such as mixed reality (XR) promoted by Apple and Meta and Generative AI chatbots promoted by Microsoft, Google, OpenAI, and others, the IoT revolution is creating more embedded UIs in devices of all kinds. 

The need for cross-platform UI models isn’t only a future need. It shapes companies’ ability to coordinate decades-old technologies such as ATMs, IVRs, and web browsers. 

A model can support a ‘portable UI.’  A prominent example of the need for portable UIs comes from the financial sector, which relies on diverse touchpoints to service customers.  One recent UI model focused on the financial industry is called Omni-script. It provides “a basic technique that uses a JSON based user interface definition format, called omni-script, to separate the representation of banking services in different platforms/devices, so-called channels….the target platforms that the omnichannel services span over contains ATMs, Internet banking client, native mobile clients and IVR.”

The ideal UI model will be simple enough to implement but flexible enough to address many modes of interaction (including natural language interfaces) and UI components that will be used in various interfaces. 

Abstraction enables modularity.  UI models share a level of abstraction that is missing in production-focused UI specifications.  

The process of abstraction starts with an inventory of UI components a firm has deployed across channels and touchpoints. Ask what system and user functionality each component supports.  Unlike design systems development, which looks to standardize the presentation of components, UI models seek to formalize how to describe the role of each component in supporting a user or system task.  

The abstraction of UI components according to the tasks they support. Source: W3C Model-Based UI XG 

Suppose the functionality is intended to provide help for users. Help functionality can be further classified according to the kind of help offered. Will the functionality diagnose a problem, guide users in making a decision, disambiguate an instruction, introduce a new product feature, or provide an in-depth explanation of a topic?  

A UI model maps relationships. Consider functionality that helps users disambiguate the meaning of content.  We can refer to UI components as disambiguation elements in the UI model (a subset of help elements) whose purpose is to clarify the user’s understanding of terms, statements, assertions, or representations. They would be distinct from confirmation elements that are presented to affirm that the user has seen or heard information and acknowledges or agrees to it.  The model would enumerate different UI elements that the UI design can implement to support disambiguation.  Sometimes, the UI element will be specific to a field or data type. Some examples of disambiguation elements are:

  • Tooltips used in form instructions or labels
  • “Explain” prompt requests used in voice bots
  • Annotations used in text or images
  • Visual overlays used in photos, maps, or diagrams
  • Did-you-mean counter-suggestions used in text or voice search
  • See-also cross-references used in menus, indexes, and headings

The model can further connect the role of the UI element with:

  1. When it could be needed (the user tasks such as content navigation, information retrieval, or providing information) 
  2. Where the elements could be used (context of application, such as a voice menu or a form.)  

The model will show the M:N relationships between UI components, UI elements, UI roles and subroles, user tasks, and Interaction contexts. Providing this traceability will facilitate a rules-based mapping between structured content elements defined in the content model with cross-channel UX designs delivered via APIs. As these relationships become formalized, it will be possible to automate much of this mapping to enable adaptive UI designs across multiple touchpoints. 

The model modularizes functionality based on interaction patterns.  Designers can combine functional modules in various ways. They can provide hybrid combinations when functional modules are not mutually exclusive, as in the case of help. They can adapt and adjust them according to the user context: what information the user knows or has available, or what device they are using and how readily they can perform certain actions. 

What UI models can deliver that’s missing today

A UI model allows designers to focus on the user rather than the design details of specific components, recognizing that multiple components could be used to support users,  It can provide critical information before designers choose a specific UI component from the design system to implement for a particular channel.

Focus the model on user affordances, not widgets. When using a UI model, the designer can focus on what the user needs to know before deciding how users should receive that information. They can focus on the user’s task goals – what the user wants the computer to do for them – before deciding how users must interact with the computer to satisfy that need. As interaction paradigms move toward natural language interfaces and other non-GUI modalities, defining the interaction between users, systems, and content will be increasingly important.  Content is already independent of a user interface, and interaction should become unbound to specificity implementation as well.  Users can accomplish their goals by interacting with systems on platforms that look and behave differently. 

Both content and interactions need to adapt to the user context. 

  • What the user needs to accomplish (the user story)
  • How the user can achieve this task  (alternative actions that reflect the availability of resources such as user or system information and knowledge, device capabilities, and context constraints
  • The class of interaction objects that allow the user to convey and receive information relating to the task

Much of the impetus for developing UI models has been driven by the need to scale UI designs to address complex domains. For UI designs to scale, they must be able to adapt to different contexts

UI models enable UX orchestration. A UI model can represent interactions at an abstract level so that content can be connected to the UI layer independently of which UI is implemented or how the UI is laid out.

For example, users may want to request a change, specify the details of a change, or confirm a change. All these actions will draw on the same information. But they could be done in any order and on various platforms using different modalities. 

Users live in a multi-channel, multi-modal world. Even a simple action, such as confirming one’s identity while online, can be done through multiple pathways: SMS, automated phone call, biometric recognition, email, authenticator apps, etc. 

When firms specify interactions according to their role and purpose, it becomes easier for systems to hand off and delegate responsibilities to different platforms and UIs that users will access.  Currently, this orchestration of the user experience across touchpoints is a major challenge in enterprise UX.  It is difficult to align channel-specific UI designs with the API layer that brokers the content, data, and system responses across devices.

UI models can make decoupled design processes work better

UI models can bring greater predictability and governance to UI implementations. Unlike design systems, UI models do not rely on not voluntary opt-in by individual developers. They become an essential part of the fabric of the digital delivery pipeline and remove inconsistent ways developers may decide to connect UI components to the content model – sometimes derisively referred to as “glue code.” Frontend developers still have options about which UI components to use, provided the UI component matches the role specified in the UI model.  

UI governance is a growing challenge as new no-code tools allow business users to create their UIs without relying on developers. Non-professional designers could use components in ways not intended or even create new “rogue” containers. A UI model provides a layer to govern UIs so that the components are consistent with their intended purpose. 

UI models can link interaction feedback with content. A UI model can provide a metadata layer for UIs.  It can, for example, connect state-related information associated with UI components such as allowed, pending, or unavailable with content fields. This can reduce manual work mapping these states, making implementation more efficient,

An opportunity to streamline API management. API federation is currently complex to implement and difficult to understand.  The ad hoc nature of many federations often means that there can be conflicting “sources of truth” for content, data, and transactional systems of record.

Many vendors are offering tools providing composable front-ends to connect with headless backends that supply content and data.  However, composable frontends are still generally opinionated about implementation, offering a limited way to present UIs that don’t address all channels or scenarios. A UI model could support composable approaches more robustly, allowing design teams to implement almost any front end they wish without difficulty. 

UI models can empower business end-users. Omnichannel previews are challenging, especially for non-technical users. By providing a rule-based encoding of how content is related to various presentation possibilities in different contexts and on various platforms, UI models can enable business users to preview different ways customers will experience content. 

UI models can future-proof UX.  User interfaces change all the time, especially as new conventions emerge. The decoupling of content and UI design makes redesign easier, but it is still challenging to adapt a UI design intended for one platform to present on another. When interactions are grounded in a UI model, this adaptation process becomes simpler.

The work ahead

While a few firms are developing UI models, and a growing number are seeing the need for them, the industry is far from having an implementation-ready model that any firm can adopt and use immediately. Much more work is needed.

One lesson of content models is that the need to connect systems via APIs drives the model-making process. It prompts a rethinking of conventional practices and a willingness to experiment. While the scope of creating UI models may seem daunting, we have more AI tools to help us locate common interaction patterns and catalog how they are presented.  It’s becoming easier to build models.  

–Michael Andrews

The post Bridging the divide between structured content and user interface design appeared first on Story Needle.

The content of evangelism

31 december 2022 - 9:15pm

I’m often asked about my job as a content strategy evangelist. What do I do? What’s the job about?  

Very little has published about the role of evangelism within enterprises and there’s no standard way that evangelism is practiced. Evangelism can easily be confused with other roles, and not everyone who calls themselves an evangelist necessarily embodies of the principles of evangelism.

In my experience at Kontent.ai as their content strategy evangelist over the past several years, I’ve been aware there’s no template for how to do this. Which is very fitting, because evangelism’s focus is on blazing new paths, not in following a checklist of tasks. To be effective at evangelism requires you to define what it is you need to communicate and why it’s important to others.

 Evangelism can bring a unique perspective that other roles don’t address in a sustained way. What makes evangelism special?

Evangelism is about change

My job title of content strategy evangelist is a amalgam of two distinct roles that are rarely combined. To my knowledge, I am the first person in the content management software industry to have this role

My role combines different ends of the communications spectrum.  

As a content strategist, I am focused on the enterprise-wide coordination of digital communications — delivering content at scale.

As an evangelist, I am focused on personal-scale communications, addressing a small cadre of professionals who are responsible for their organization’s content strategy. I communicate to a select group of content leaders, through conference presentations, panel discussions at meet-ups, podcast interviews, and a range of writings. 

Evangelism aims to change the status quo by changing people’s ideas about established practices. The evangelist introduces new ideas to people who might benefit from them.  

I often must distill wide ranging and sometimes nuanced ideas. I see my mission as having two related aspects:

  • To articulate the promise offered by a new way of managing enterprise content
  • To do everything in my power to ensure every stakeholder involved can deliver on that promise

The evangelist’s goals are both lofty and broad. It’s challenging for any single person to direct such change. But I don’t believe you can be successful if your motivation is transactional, such as getting a certain number of people to take a certain action.  

My goal is not to get everyone to agree with me but to help motivated individuals change how they work successfully. Unlike traditional marketing, evangelism can’t be boiled down to a few success metrics, because evangelism aims to influence long-term processes rather than short-term events.

Evangelism differs from traditional corporate roles in numerous ways. For one thing, it involves discussing issues rather than a specific product. The issues typically involve the situational context in which a product might be used. My focus is on rethinking how people do things before getting into the details of what they do. I notice a growing number of people calling themselves “product evangelists” but in most cases these are traditional product marketing and promotion roles dressed up in trendier job title.

Secondly, the evangelist introduces new concepts for addressing an issue. The concept must be new and not simply a small improvement over standard practices (otherwise, it’s not evangelism but ordinary content marketing). It will be unfamiliar to most people and will require explaining. And the concept must be practical, providing tangible benefits. Evangelists are not asking people to sponsor a moonshot venture. They are offering a plausible future that the adopter can use and adapt to suit their needs. 

Business evangelism isn’t new — but it’s growing

 For several decades firms have hired staff evangelists, especially in technical areas. But more recently, the evangelist job title has become more common and diverse. As businesses rethink how they operate and deliver services — a epoch broadly referred to as “digital transformation” — they are looking at and evaluating new options. 

Numerous companies are introducing innovations to enable the transformation of enterprise practices. They need staff who can explain how customers to make use of these innovations. Before you can realize product-market fit, the market needs to understand what you are offering.

Before the current era of digital transformation was the era of computers in every home. Apple is credited with introducing evangelism to the software industry. The first highly visible corporate evangelist was Guy Kawasaki, then an employee at Apple in the 1980s, who donned the title of evangelist for a few years. He subsequently wrote a book, Selling the Dream, marketing his brand of evangelism, which he claimed to have been based upon the preaching style of the Reverend Billy Graham. Kawasaki drew attention to the term “evangelism” — and its contested meaning — outside the religious context. But he was (and is) a bad example of what evangelism should be about. Contrary to Kawasaki, evangelists aren’t selling a nebulous “dream.”  They are raising awareness of innovative solutions to current problems.

Evangelism is about building interest, not building buzz

Seemingly everyone wants to “make noise” these days, which is a problem. It’s a noisy world out there. Noise-making doesn’t accomplish much.

In contrast to vanity driven approaches that emphasize form over substance, evangelism avoids empty public relations or branding tactics. It aims for more enduring changes in how people think about issues.

Apple’s novice evangelist, Guy Kawasaki, acted more like a brand ambassador promoting Apple products than an true evangelist promoting a distinct point of view. His role was channel existing enthusiasm for Apple that was generated by the company’s founder, Steve Jobs. If fans couldn’t meet with Steve, they got Guy as a consolation. Kawasaki was a figurehead: a cheerleader ginning up an a reverential customer base.  For those who remember those days, Apple was far from its behemoth status of today. It was a small company that managed to have a cult-like following. People routinely compared Apple at the time to a cult, with all the baggage that term implies — its fans could be obnoxious zealots prone to demean non-believers with harsh words. The closest current example I can think of are the Tesla fanboys who hero-worship and defend Elon Musk no matter what his shenanigans. It’s a bad template for winning hearts and changing minds.

Evangelism thus needs to be contrasted with evangelicalism or building an army of followers. The goal of evangelism isn’t to promote a rigid set of beliefs or allegiance to a person or brand. Rather, evangelism expresses a conviction about possibilities. It should promote reflection and agency, not doctrinal conformity.

In lieu of doctrine, the essence of evangelism is passion and insight.

To be heard and believed, you need to care deeply about what you talk about. When the legendary chef Julia Child died, her New York Times obituary described her as an evangelist for French food. That passion for the subject, without the self-promoting and personal branding, puts her in stark contrast to today’s “celebrity chefs”, who sell themselves as licensing ventures for spin off products. Julia Child communicated not just her love of French cooking, but let her audience understand why she cared so much about it and why they should as well. Beyond being a talented cook and explainer, she was a brilliant evangelist.

Evangelists must also be contrasted with “influencers” who position themselves as role models or identity-based personalities. Influencers trade on their personal popularity and name recognition, which means that maintaining and growing their popularity is their currency and their goal. 

Unless the influencer has staked their popularity on some position they’ve taken (as contrarian or campaigner), the ideas of the influencer are not the focus of attention. In many cases, the influencer has a tenuous connection to any idea, much like the celebrities who endorse any product that matches the demographic target of their followers. Many brands worry that the influencers they hire could voice problematic opinions that might embarrass the brand.

Evangelism builds on reputation

Everything that an evangelist says and does reflects back on their personal reputation and that of their employer.

Success in evangelism depends on the affinity between the person and the message.  

Evangelism not about a doctrine that can be debated independently of any person, because the ideas are too fresh and evolving to be codified as canonical texts or widely accepted truths. Evangelists discuss the future; they don’t debate the past.

At the same time, evangelism is about more than personal branding, or playing a role (the Marcus Welby phenomenon). An evangelist is more than simply a spokesperson.

The evangelist should be primarily known for the ideas they promote. The famous newscaster Walter Cronkite was widely trusted because he didn’t promote an overt agenda. An evangelist, by contrast, is trusted because of their convictions about an explicit agenda.

Credibility is an essential quality of an evangelist. The evangelist will be recognized as a thought leader. Perhaps the most eminent evangelist is Vint Cerf, considered one of the fathers of the internet in the 1970s who later joined Google as internet evangelist and Vice President. The internet may be over 40 years old, but its dynamism means that much about it is still evolving and undecided. It’s valuable to understand the background that has shaped the junctures we face today, which includes digital governance and the next generation of internet protocols. Evangelism is needed.

Evangelists introduce innovative ideas and highlights why they are appealing  

Other roles may promote points of view. But they differ from the evangelist.

Unlike an advocate, who asks others to support a well established position, an evangelist promotes something less well known and asks the listener not only to support the concept, but to adopt it, practice it, and embrace its consequences. 

Promoting change is hard to do and potentially risky.  If overly assertive, the would-be evangelist comes across as pushy or rigid. 

Evangelism must also be distinguished from activism, where winning can be more important than principles, and opponents can be attacked as villains. There’s little benefit in building a small army of followers if a far larger group thinks you’re a jerk. Evangelists must show they’re on the side of most people.

Evangelists are inclusive

The evangelist champions ideas they expect will have broad appeal for the group they are trying to reach.  

They must avoid setting up a “Marmite taste challenge” for their views, where people react to the ideas as they would to the taste of yeast spread: they either love it or hate it. An evangelist will be inclusive, not polarizing.

While evangelists don’t court controversy — being provocative for the sake of it —they can’t avoid engaging with detractors.  Ideas, by their nature, can be contested. And sometimes, old ideas that have outlasted their usefulness turn out to be the detracting competition.

Evangelists aim to disrupt the status quo because they believe the status quo isn’t working well. Those who defend legacy thinking may want to engage in ideological or theological debates, arguing that the established way anticipated everything that is needed and is therefore better. For them, the stakes might be high: employee skillsets and entire businesses can feel threatened by alternative ways of working championed by the evangelist.  

It’s important to not engage in winner-versus-loser thinking that’s common in much social discourse today. Those who are invested in the status quo will often take the position of “originalists,” contending that the decisions of the past were perfectly informed and have served everyone just fine, thank you. Moreover, the detractors will assert, these practices have been codified as “best practices” or are enshrined in some standard drafted by a committee two or three decades ago. There’s no need to change anything. It’s been written down as doctrine.

Originalists dismiss new alternatives in one of several ways. As:

1. Fads that will be quickly forgotten because they offer no real value

2. Deceptive marketing ploys that are potentially dangerous (and will be remembered in infamy)

3. Phantom novelty that doesn’t deserve attention because it’s a case of putting worthy old wine in modish new bottles

The careful observer will note that these criticisms are mutually inconsistent. 

With the third interpretation — concerning phantom novelty — the original practices supposedly already do everything the new one offers, but people don’t see this because the terminology differs. This stylistics defense can prompt a stylistics update. Defenders of original practices may update their terminology in an effort to make it more appealing and contemporary sounding. When they do this, they manage to do what they complain about: putting old wine in new bottles.  

New ideas invariably invite skepticism, as they should. Not all new ideas are good, much less better. But…

What’s new about a concept can sometimes be hard for industry veterans to recognize. Thery’re invested and habituated in thinking about things a certain way. It’s important to keep highlighting what’s different about new practices, because there can be resistance to accepting that newer ideas are viable options. Few truly novel ideas are instantly recognized as superior. It takes repeated elaboration for the value of an idea to become clear to different people. If you are genuinely convinced of the value of what you propose, there’s nothing disrespectful about evangelizing your position, even if not everyone is ready to accept it. 

A familiar challenge is finding a common ground between old and new practices. It may be tempting to gloss over differences and emphasize similarities, but this approach can hide critical conceptual distinctions that have a big impact.  

Focus instead on how organizations can adapt to changing practices. Sometimes implementing sudden wholesale change is not the best option for an organization.  

Evangelists are generous, not competitive, with their knowledge

Evangelism is not about promoting a cult of expertise. Evangelists shouldn’t be viewed as a know-it-all authority who has solved all issues and has the final answer to everything. Nor should they act like a mansplaining pundit who states the obvious.  

An evangelist is an explorer. Topics they address are on the edge of emerging practices. They advocate a new direction, not a cookie cutter solution.

An evangelist doesn’t work from talking points. They are an reflective thinkers, able to consider each unique situation in light of how they look at issues more generally. They don’t impose an approved way of dealing with these issues.  Rather, they seek new avenues to consider them.

As an evangelist I offer advice freely, without obligation, in contrast my previous work as a consultant. It won’t of course be as specific to a client’s situation as it would be when provided in a consulting engagement — I’m certainly not trying to put consultants out of work. But I do aim to provide the same level of trust in the advice I offer. In so doing, I help our clients become successful. 

Evangelists help people navigate uncertainties associated with transitions

Evangelist aren’t prophets or play the role of “seer.” They won’t forecast the inevitability of certain trends because they understand they don’t control them. A credible evangelist will avoid making bold predictions, which can devolve into “wishcasting” — statements of hopes rather than informed assessments of likelihoods. But they will be aware of current and past trends in their field and can draw on that knowledge to construct plausible scenarios.

An evangelist knows trends because he or she is directly participating in them. They see how early adopters are exploring new possibilities and are aware of the challenges that face the “early majority” who may have less knowledge or passion about new approaches. They also see the potential unlocked by wider adoption, where more infrastructure, learning, and synergies can unlock possibilities difficult to realize at the moment.

This hand-on perspective stands in contrast to the outsider viewpoint of industry analysts. Anyone pronouncing the next wave is probably not directly in the trenches building that wave. Too many industry analysts operate at a level of abstraction that is detached from the practical issues business folks are trying to solve. Hype cycles aren’t helpful. 

Evangelists are authentic

No organization is going to change how they do their daily work based on a short fluffy pitch. Evangelists need more than a 30-second elevator pitch or a 20-slide PechaKucha deck to change long entrenched ways of doing things. But given the transactional orientation of most business communications, many people confuse gaining attention with gaining agreement and commitment.  Evangelism is a long game. Success comes from getting commitment.  

Evangelists must go beyond the TED Talk formula — telling a scripted story about the one event that made the speaker realize something fundamental that would change their life. The first time you hear it, it sounds compelling. The third time you hear it, it sounds contrived — spun-up for the speaking circuit, and repeated because it gets a reaction from the live audience. But those who become interested in the topic start to wonder: what else is there beyond this one shop-worn story? If it sounds too easy or too good, it likely is. Elizabeth Holmes, the recently convicted startup fraudster, mastered the life story pitch. If your message sounds contrived, it will hurt you sooner or later.

Evangelists acknowledge the messiness of change, and the challenges of unlearning legacy habits and practices. 

They can’t gain credibility if they are too rehearsed and tell too tidy a story and gloss over the realities people will confront.   Their authenticity is grounded in realism and based on their first-hand experiences.

Evangelists are trusted

Trust is based on many factors.  As mentioned earlier, you need credibility. People must respect your expertise about an issue. You must have things to say that seem worth listening to.

You gain trust when you are earnest and helpful.  And you won’t be truly helpful if your primary focus is on promoting oneself or your firm’s products. Plenty of sales and marketing roles promote products and can provide useful information for product buyers. The evangelist’s focus is different, and shouldn’t try to duplicate that work.

Evangelists don’t only speak and share information, they listen. I learn important insights through the trust I’ve earned with people. An evangelist belongs to a community who share a common cause in improving how to solve challenging issues by applying emerging practices.  

Evangelists gain trust by acknowledging what they don’t know. That’s easy for me because the issues I talk about are bigger than me individually. I’m often gratified to hear insights I haven’t previously considered during my discussions with peers.

It’s equally important to avoid mystification, such as suggesting that outcomes depend on so many variables that the listener couldn’t possibly understand how they all interact. The evangelist has failed if they ever suggest people won’t be able to understand what they are proposing.

Making changes involves moving into the unknown. Evangelists should inspire confidence that clients can incrementally master the granular issues and details they don’t yet grasp. But they should never make them feel they must buy into an idea that seems too magical to be ever comprehended fully.

Evangelists offer original thinking 

I believe that the quality of ideas has a huge influence on the outcomes that organizations achieve. Yet it’s hard to see much difference in the quality of ideas that various firms voice.

Firms work hard to make their products seem different, but few demonstrate leadership in their approach to the core issues their customers face. They lack a distinct point of view about how to solve issues.

As a content strategist, I’m keenly aware that we live in a content-saturated world that’s burdened by the sameness of group think. The internet is cluttered with copycat content optimized for clicks on social media and search results. Many organizations put much of their energies into continually shifting their tactics to game better results. All this content promises answers but too often disappoints once viewed.

We now face a existential reckoning regarding the tangible value of content. In recent months, we’ve watched the spread of generative AI such as GPT-3+ chatbots into the mainstream. Generative AI can instantly refashion legacy content into new content that superficially appears original but in reality is derivative.  

How can firms expect to be market leaders if their content is derivative copycat content? What’s the value of content if content is now “free”? Why should we care about it?

The burning issue no longer is the value of content in terms of its cost to make — it can be generated by robots on demand at little cost. Now the urgent question is how to develop truly original content that will stand out amidst the sea of synthetic content.

As a content strategist, I think intensively about the distinctive qualities of content and what makes content stand out in the herd of sameness. Good writing, while important, is not is not the decisive factor in how impactful content will be. Rather, it’s the quality of the ideas and information.

Few organizations realize the illusive goal of “thought leadership.” Yet the longing to be recognized as a thought leader highlights the premium that firms place on original thinking.

Readers want to learn things they don’t already know. They want perspectives that will challenge their assumptions so that they can be sure they are making the best choices.

Some companies commission external research agencies to develop original content because they lack that capacity in-house. And some companies will hope that AI will fill their thought leadership deficit. They would do better to hire a proper evangelist.

One sign that my work as an evangelist has been successful is when it gets emulated by competitors.  Once, my firm’s largest direct competitor copied a complex diagram I had created to explain a concept and, after changing the colors to match their branding, published it in their website (it was subsequently taken down). While outright plagiarism is rare, the copycat phenomenon is ever present.

Evangelists show relevance through meaning

An evangelist helps people clarify the complexity that they face.  

When learning about new options, people want to know what does this new practice mean for them, personally? How will they apply it?

Employees need to understand how concepts work in order to be able to apply them. Business history is littered with waves of changes that were driven by outsiders (investors, advisors, outsource consultants, etc) who imposed change on organizations without the rank and file ever fully understanding their details. The leadership in charge never “owned” the change and was never able to direct and control its evolution.

Until someone actually tries a new practice, they must rely on their imagination to envision how it works. They need examples or analogies that will engage their imagination.

Everyone faces different situations and has had distinct experiences, so no single example will fit every person’s unique use case. People will need to translate ideas to apply them to their own situations. Only once people feel they grasp a new concept because they’ve applied their own thoughts and perspectives to it will they be prepared to implement it.

An evangelist must know their audience and identify with their needs. They must be able engage people by sparking their curiosity. They’ll consider how to make the topic personally interesting in some way to the listener. That requires them to apply their own imagination and to develop deep empathy for those they are speaking to.

— Michael Andrews

The post The content of evangelism appeared first on Story Needle.

Your markup is showing

25 mei 2022 - 4:12pm

Markup is supposed to make content better. So why does it frequently make content worse?

Markup helps computers know how to render text in a user interface. Without markup, text is plain — a string of characters. That’s fine for simple communications, but plain text can’t express more complex ideas.

Syntax enables words to become meaningful content. Markup is syntax for computers. But computer syntax is far different from the syntax that writers and readers use.

HTML is the universal markup language for the web. Markdown is positioned as a light-weight alternative to HTML that’s used by some writing apps and publishing systems. Some content developers treat Markdown as hybrid syntax that offers a common language for both humans and machines, a sort of “singularity” for text communication. Sadly, there’s no language that is equally meaningful for both humans and machines. If humans and machines must use the same syntax, both need to make compromises and will encounter unanticipated outcomes.

Markup is a cognitive tax. Code mixed into text interferes with the meaning of the writer’s words, which is why no one writes articles directly in HTML. Text decorated with markup is hard for writers and editors to read. It distracts from what the text is saying by enveloping words with additional characters that are neither words or punctuation. When writers need to insert markup in their text, they are likely to make mistakes that cause the markup to be difficult for computers to read as well.

Each morning, while browsing my iPad, I see the problems that markup creates for authors and for readers. They appear in articles in Apple News, crisply presented in tidy containers.

Apple News format

Apple News publishes text content using a subset of either HTML or Markdown. Apple cautions that authors need to make sure that any included markup is syntactically correct:

Punctuation Is Critical. Incorrect punctuation in your article.json file—even a misplaced comma or a curly quotation mark instead of a straight quote—will generate an error when you try to preview your article.”

That’s the trouble with markup — it depends heavily on its placement. A missing space or an extra one can spell trouble. Developers understand this, but authors won’t expect that the formatting of their writing to present problems in the third decade of the 21st century. They’ve heard that AI will soon replace writers. Surely computers are smart enough to format written text correctly.

Often, markup triggers a collision between computer syntax for text and computer syntax for code. This is especially the case for reserved characters: specific characters that a computer program has decided that it gets to use and that will have priority over any other uses of that character. Computer code and written prose also use some of the same punctuation symbols to indicate meaning. But the intents associated with these punctuation marks are not the same.

Consider the asterisk. It can act like a footnote in text. In computer code, it might signal a function. In Markdown, it can be a bullet or signify the bolding of text. In example below, we see two asterisks around the letter “f”. The author’s goal isn’t clear, but it would appear these were intended to bold the letter, except that an extra space prevented the bolding.

Misfired asterisk

If there was any symbol that logically should be standardized in meaning and use, it would be the quotation mark. After all, quotation marks indicate that the text within them is unmodified or should not be modified. But there are various conventions for expressing quotations using different characters. Among machines and people there’s no agreement about how to express quotation marks and what precisely they convey.

A highly visible failure occurs when quoted text is disrupted. Text in quotes is supposed to be important. The example below attempts to insert quotation marks around a phrase, but instead the Unicode for single quotes are rendered.

Misfired scare quotes

Here’s another example of quotes. The author has tried to tell the code that these quotes are meant to be displayed. But the backslash escape characters show in addition to the quote characters. A quotation mark is not a character to escape in Markdown. I see this problem repeatedly with Reuters posts on Apple News.

The jargon escapes comprehension

Here’s a mystery: “null” starts a new paragraph. Maybe some Javascript code was looking for something it didn’t find. Because the Null follows a link that ends with a quote, it seems likely that part of the confusion was generated by how the link was encoded.

Null results

Here’s another example of link trouble. The intro is botched because the Markdown is incorrectly coded. The author couldn’t figure out where to indicate italics while presenting linked text, and tried too hard.

A not-too-stylish welcome Takeaways

All these examples come from paid media created by professional staff. If publishers dependent on revenues can make these kinds of mistakes, it seems likely such mistakes are even more common among people in enterprises who write web content on a less frequent basis.

Authors shouldn’t have to deal with markup. Don’t assume that any kind of markup is simple for authors. Some folks argue that Markdown is the answer to the complexity of markup. They believe Markdown democratizes markup: it is so easy anyone can use it correctly. Markdown may appear less complex that HTML but that doesn’t mean it isn’t complex. It hides its complexity by using familiar-looking devices such as spaces and punctuation in highly rigid ways.

If authors are in a position to mess up the markup, they probably will. Some formatting scenarios can be complex, requiring an understanding of how the markup prioritizes different characters and how code such as Javascript expects strings. For example, displaying an asterisk in Markdown requires that it be escaped twice with two backslashes in Apple News. That’s not the sort of detail an author on a deadline should need to worry about.

— Michael Andrews

The post Your markup is showing appeared first on Story Needle.

Taxonomy makes the world go round

21 oktober 2021 - 11:05pm

Taxonomy, like the money supply or the ozone layer, is a stealthy — yet fundamental — concept in our lives that, unfortunately, only a few people know much about.  Taxonomy makes the world go around: all kinds of things would stop moving without it.  But only a small number of people — either inside or outside of the world of the web that manages our lives online — can explain what a taxonomy actually does. It’s generally a vague concept to designers and developers.  Business people often have little idea what it is or why it matters. Even many information architects, and other people with backgrounds in library science, tend to ignore the social and economic significance of taxonomies.

Taxonomy matters because it is the basis of numerous decisions that affect us. But it can be misunderstood when seen as being primarily about making individual decisions: specifically, looking up information.  Taxonomy’s true influence comes from how it supports collective, repeatable decisions and enables the aggregation of data and actions.  Rule-based decisions, whether applied by humans or machines, depend on taxonomies.  

Taxonomies shape many of the most important decisions in our world — those that are made again, and again.  Four taxonomies in particular influence the basic dimensions of our lives:

  1. What we eat
  2. How money is made and used
  3. What goods and services are produced
  4. How we evaluate our impact on the environment

All these taxonomies share a common focus on process changes as forming the basis for categorization. Why? When items change state or are susceptible to change, distinctions emerge that are useful to classify.  

Codex Alimentarius: Classifying what we eat

Eating is an elemental need: it supplies nourishment.  Something so basic seems far removed from the abstraction of taxonomy.  And few would expect a taxonomy with a Latin name to be important to the modern world economy.  

The Codex Alimenatrius means “food code.”  It is defined by a United Nations organization based in Rome (a joint effort of the FAO and WHO, in case you care about the acronyms, instead of the Latin).  The “Codex” provides a “classification of foods and feeds.”    

Please don’t get turned off by the jargon — the stakes here are important.

If you have ever wondered how we can safely eat food items produced from around the world, it is thanks to the Codex Alimentarius. It defines in great detail criteria for food commodities, specifying their composition and safety.   The Codex provides a detailed set of guidelines relating to food covering:

  • Standards covering processed, semi-processed or unprocessed foods 
  • Hygienic and technological codes of practice
  • Food additives 
  • Maximum levels for pesticide residues 
  • Guidelines for contaminants

It promotes the harmonization of national regulations and encourages food producers to adopt common standards.  

The scope of food products is enormous.  To provide useful guidance, the Codex should be able to identify the universe of products needing standards.  The Codex Classification of Foods and Animal Feeds provides this needed classification.   It divides food into classes based on three dimensions:

  • How basic or changed, is it? Primary v Processed
  • Who is it for? Food for people v Feed for animals
  • What’s it made of? Plant-origin v Animal-origin

In all, five classes exist:

  1. Primary food commodities of plant-origin 
  2. Primary food commodities of animal-origin
  3. Primary animal feed commodities
  4. Processed food of plant-origin
  5. Processed food of animal-origin

Each of the five classes is divided into types that are “based on physical characteristics and traditional use and to a lesser extent on botanical or zoological associations.”  

Types are subdivided into groups and subgroups “whose members show similarities in their behavior with respect to residues and in the nature of the agricultural practices to which they are subjected.”  

Finally, within each group, different commodities are enumerated with a code identifier having a 2-letter preface followed with a 4-number suffix.   

The standards themselves are narrative documents. But what the standards address is determined by the taxonomic classification.  They can be relevant to individual commodities — but also higher-level classifications.  

Let’s explore how the Codex’s classification works by reviewing how it classifies anise, one of my favorite flavorings, an herb that has been used since ancient Roman times. Anise commodities belong to “Class A: Primary Food Commodities of Plant Origin.” 

Anise commodities more specifically belong, within Class A, to Type 05: “Herbs and Spices.”  

Multiple anise-related commodities exist (a few of them flagged in bold below), which get classified in various ways:

  • Herbs (group 27), whose commodities have a code prefix of “HH”
    • 027A Herbs (herbaceous plants), broken into commodities with 6 character codes, for example:
      • HH 3191 Anise, leaves
    • 027B Leaves of woody plants (leaves of shrubs and trees)
    • 027C Edible flowers
  • Spices (group 28), whose commodities have a code prefix of “HS”
    • 028A Spices, seeds
      • HS 0771 Anise, seed
    • 028B  Spices, fruit or berry
      • HS 3303 Anise pepper
    • 028C Spices, bark
    • 028D Spices, root or rhizome
    • 028E Spices, buds
    • 028F Flower or stigma
    • 028G Spices, aril
    • 028H Citrus peel

The Codex refers to its enumeration as a classification rather than as a taxonomy.  Within the Codex, the term “taxonomy” is reserved for biological descriptions of living plants, animals, and microbes rather than for non-living or man-made food items. Biological taxonomies are also based on identifying distinctions arising from the process of change — distinctions which are increasingly determined through DNA analysis.

Because the Codex is based on science, it is constantly evolving. Experts must account for a growing range of synthetic inputs and outputs that become available, such as lab-grown “meat.”  The classification undergoes revision as needed.

The Codex has been in place for over half a century, achieving something remarkable: it’s made possible the confident trade of food around the world.  We can buy imported food that’s labeled clearly and meets agreed standards. It’s enriched us all, whether we get access to better and less expensive food, or simply have more varied and interesting options.     

The GAAP taxonomy: Is a company profitable and viable?

If you work for a company or have invested your retirement savings in companies, you want to know if a company is profitable.  

And you need to know their statements are reliable. Several high-flying firms, including Wirecard in Germany, Greensill Capital in the UK, and Evergrande in China, have collapsed quite suddenly in recent months.  People need to trust financial data.  

The GAAP (Generally Accepted Accounting Principles) has been the accounting standard for many decades.  But only in 2013 was the GAAP transformed into taxonomy: the US GAAP Financial Reporting Taxonomy (UGT), based on the Extensible Business Reporting Language (XBRL) markup.  

If the GAAP has been around for decades, why is it suddenly a big deal that it’s now a taxonomy?

This development has promoted much greater transparency and global harmonization in financial reporting.    

Accounting, fundamentally, is about the categorization of income and expenses, and of assets and liabilities.  These can be split or lumped according to the granularity sought.  Accounting is taxonomic in its orientation.  Providing precision enables traceability. 

The GAAP taxonomy, when encoded with markup, transforms difficult-to-analyze documents into structured data that can be compared.  

Bob Vause, in his book the Guide to Analyzing Companies, notes the power of the taxonomy to describe individual financial items.  “This involves ‘tags’ — tagging all the individual items of information appearing in financial reports.”

The GAAP taxonomy is being adopted globally, sometimes unofficially, which is promoting a global convergence in approaches.  This is an important development, as different counties sometimes use divergent terminology when describing financial events. Those inconsistencies have hindered the comparison of companies operating in different markets.  

Today, the differences in terminology are less important than in the past because the taxonomy tags provide a standard way to present financial data.  Vause notes: “There will be an international standardised form of presentation; language or terminology will no longer be a barrier to analysis.” He adds: “Moves towards the standardisation of financial report information are leading to significant improvements in the quality, value and accessibility of corporate financial information.”

The GAAP taxonomy helps financial analysts track inputs and outputs, and follow transfers and allocations.  

NAICS taxonomy: what’s the composition of the economy?

It’s difficult to know what’s important in the economy, such as how big a sector is or how fast it is growing.  Some sectors seem to get more media attention than others.  And it’s not always clear what is different about sectors.  For example, is fintech different from banking?  Companies and governments need to understand changes in the economy when making investment decisions.  

Industrial classification taxonomies help analysts dissect the complex composition of an economy.  Since 1997, the United States, Canada, and Mexico have agreed to follow a common system to classify businesses called the NAICS (the North American Industry Classification System). The EU uses a similar classification called the NACE.  Governments define these taxonomies and use them to collect all kinds of data.  By being structured as a  taxonomy, with broader and narrower classifications, it’s possible to aggregate and dissect data about production, orders, hiring, and other activities.  

According to the official NAICS Manual, “NAICS divides the economy into 20 sectors. Industries within these sectors are grouped according to the production criterion.”  But industries could be services as well as manufacturing. For “activities where human capital is the major input… each [is] defined by the expertise and training of the service provider.” 

The NAICS uses hierarchical numeric codes, an approach referred to in the field of taxonomy as “expressive notation.”  These codes don’t look like the more familiar style of taxonomy that uses text labels.  But codes provide many benefits.  They provide a persistent identifier that allows the description to be revised when necessary.  And they enable expansion of the code from 2-6 digits to get the appropriate level of granularity.  They allow the retrieval of either root classes or subordinate classes (subclasses), and the comparison of equally ranked classes.  For example, the taxonomy supports retrieval of data on all performing arts groups, or it can allow the comparison of data for different kinds of performing arts groups (theater companies, dance companies, or music groups). 

The NAICS taxonomy

When you look at the NAICS, you might disagree with how industries are categorized.  You may think the balance or emphasis is wrong.  But the NAICS isn’t based on any single person’s opinion or even a card sort by a group of people.  It’s a highly governed taxonomy.  Persistence in categories is necessary to allow data to be tracked over time.  Even when a sector is losing importance, it is still useful to understand how it has declined.  And sometimes, obscure sectors can be more important than non-specialists realize.  They aren’t scoring headlines in the news, but they still matter.  

The NAICS is the product of a well-defined methodology. It classifies sectors according to how products or services are created.  “Economic units that have similar production processes are classified in the same industry, and the lines drawn between industries demarcate, to the extent practicable, differences in production processes. This supply-based, or production-oriented, economic concept was adopted for NAICS.”  Much like the GAAP, the NAICS allows analysts to track inputs and outputs.  

The Green taxonomy: classifying climate impacts

Lastly, we turn our attention to another thing humans produce, but one that’s less desirable: greenhouse emissions.  

As I write this, world leaders will soon gather in Glasgow for COP26, the UN Climate Change Conference.  Countries and companies are pledging to be carbon-neutral or even carbon-negative by specific dates.  But what do these commitments really mean?  How will emissions and outsets be calculated?  And how will everyone agree these commitments are measured?  

These important questions will require a slightly longer explanation. 

Over the past year, momentum has been growing for climate change-focused taxonomies. These initiatives classify energy usage according to their emissions. They are discussed using various terms:

  • Green taxonomy
  • Climate taxonomy 
  • Sustainable finance taxonomy   

But how can a taxonomy influence the climate?  In many of the same ways that taxonomies influence other human activities such as food, finance, and the economy.  Taxonomies can help individuals understand processes relating to what we produce and consume. And they can help institutions follow common standards to track and direct energy usage behavior.  

A major motivation for green taxonomies is capital allocation: both the capital used in market debt financing of infrastructure projects and the institutional equity investments into energy-consuming enterprises.  Power plants and grids are expensive.  Converting existing infrastructure to be more green forms is as well. The aspiration is to create new financial products such as green bonds or sustainable index funds to promote such investments.

EU environmental taxonomy

Multiple green taxonomy initiatives are underway globally, which vary in their designs.  At the moment, the bulk of media attention has been focused on the EU’s environmental taxonomy, which is currently in draft.  Some reports suggest it may be finalized by the end of 2021 and that China might join the EU initiative in some capacity.  The ASEAN group of Southeast Asian countries is working on their own taxonomy that is heavily modeled on the EU one.

The EU’s environmental taxonomy is based on the NACE, their industrial classification system. It looks at various economic sectors and  identifies “sustainable” activities within each sector.  

The Commission states: “A common language and a clear definition of what is ‘sustainable’ is needed. This is why the action plan on financing sustainable growth called for the creation of a common classification system for sustainable economic activities, or an ‘EU taxonomy’.”

“The Taxonomy sets performance thresholds (referred to as ‘technical screening criteria’) for economic activities.”

Member states and financial institutions within the EU will be required to report on the sustainability of projects they sponsor.  Those activities deemed sustainable will presumably be more attractive to long-term investors.    

The Commission’s “green list” of those activities it has decided are sustainable has created some controversy.

Notably, the Commission has been unable to decide whether nuclear power and natural gas are environmentally desirable.

In the United States, interest in green taxonomies has also been growing.  Compared with the EU, it is more driven by interest from institutional investors, though US government regulators have also called for better and more consistent standards.

The EU’s approach to a green taxonomy is prescriptive — indicating what activities are sustainable.  The US approach, in contrast, is more descriptive — classifying a range of activities that could support or hinder sustainability and promoting reporting of harms and well as benefits.  

The SASB taxonomy in the United States

In the United States, green taxonomies fall under the broader umbrella of “ESG” (Environment, Social, and Governance) — that is, non-financial information relating to company performance.  US investors increasingly favor ESG-positive firms and funds.  But ESG indicators haven’t been standardized across companies, which has triggered concerns from the US General Accountability Office. The ESG landscape has emerged organically, which partly explains their uneven application.  But some enterprises and investment funds have been faulted by environmentalists and financial regulators for using imprecise standards and metrics to engage in “greenwashing.”  

“Investors are using ESG-related information to make investment decisions and to allocate capital more than ever before. They are increasingly looking for sustainable investments, albeit investors have different thoughts about what ‘sustainability’ means.” 

Securities and Exchange Commissioner Caroline A. Crenshaw

Commissioner Crenshaw adds: “To be useful to investors, disclosures need to be meaningful. That’s particularly true for ESG-related disclosures, as they are too often inconsistent and incomparable. What we should be working toward is a clear disclosure regime that yields consistent, comparable, reliable, and understandable ESG disclosures to investors.”

ESG is broader in scope than sustainability.  But environmental metrics so far have been the major driver of ESG interest.

Following a period of competing initiatives in the US, a unified approach has coalesced under the auspices of the Sustainability Accounting Standards Board (SASB), a nonprofit organization that cooperates with several other like-minded organizations.  

The SASB has chosen a different approach to define a green taxonomy.  They have re-imagined the NAICS to focus it on environmental performance.  It has created what it calls the Sustainable Industry Classification System® (SICS®). “The differences between SICS® and traditional industry classification systems can be categorized in three types: (1) new thematic sectors; (2) new industries with unique sustainability profiles; and (3) industries classified in different sectors.”  The SICS classifies 77 industries are grouped into 11 categories.  

The SASB taxonomy

“Unlike other industry classification systems—which use common financial and market characteristics— SICS® uses sustainability profiles to group similar companies within industries and sectors.” 

“SASB Standards identify the subset of environmental, social, and governance issues most relevant to financial performance in each of 77 industries.”

In addition to drawing inspiration from the NAICS, the SASB also modeled its work on the GAAP taxonomy.  

It worked with PwC to convert SASB’s taxonomy into the XBRL format (the “SASB XBRL taxonomy”) to allow data exchange between reporting companies and investors and regulators evaluating the reporting. 

As a recent news report concludes: “SASB Standards Taxonomy in XBRL format…make[s] digital reporting simpler for issuers of environmental, social, and governance (ESG) disclosures and to improve data aggregation and analytics for investors.”

“Having the taxonomy in XBRL, an open standard used in business reporting, will enable reported metrics to be machine-readable via digital tags, and improve usefulness and comparability of ESG reports.”

The next emerging area of focus for green taxonomies is looking at what are called “scope 3” emissions, which consider emissions across the value chain of products.  Instead of looking at individual industries or companies, this work will evaluate the emissions associated with the entire lifecycle of production, consumption, and disposal.  Companies will be more accountable for the actions of their suppliers and customers.  

Taxonomies support exchange

I learned about the value of taxonomies in the late 1980s during my first full-time job after leaving grad school.  I worked as an online researcher at the US Commerce Department.  This was before Tim Berners-Lee launched the World Wide Web — there were no browsers or Google then.  But already online information providers recognized the need to categorize information to make it accessible.  Accessing information at that time was extremely expensive.

Taxonomy is a form of infrastructure, much like the undersea fiber optic cable that transmits internet traffic is.  The scholar Elizabeth Cullen Dunn talks about the role of infrastructure by introducing a Greek word, oikodomi, which she defines as“the infrastructures from which standards emerge and that shape the way they actually effect production.”  Taxonomy is oikodomi, in the sense that it shapes decisions.  

Many people lately are talking about the importance of “systems” (or “systemic” influence) to understand how things work now and how they should in the future.  That’s exactly what taxonomy is doing: defining systems.

Taxonomy standards facilitate global exchange by

  • Promoting harmonization between stakeholders in different places
  • Connecting descriptions between different fields  

Taxonomy standards connect our diverse world. They can thread together different domains: food, which influences health, companies, the economy, and the environment. They provide the transparency that can reveal the relationships between these dimensions through their precise classifications.

— Michael Andrews

The post Taxonomy makes the world go round appeared first on Story Needle.

Revisiting the difference between content and data

23 maart 2021 - 2:30am

This post looks in detail at the differences between conent and data.  

 TL;DR — I understand you’re too busy to ready a 10,000+ word post.  No worries, I’ve pulled out some highlights in the below table.  If this data doesn’t answer your questions, you may have to read further.

ContentDataOpen-domainClosed domainNo restrictions on expressionExpression is restrictedHas an intentDoesn’t have an intentHas an authorOften anonymousComplex valuesSingle, unambiguous valuesTopics and narrativesEntitiesStructure builds resourcesStructure defines boundariesFacts discusses outside of pre-defined relationshipsFacts described through pre-defined relationshipsAssembly structure is coupledAssembly structure is decoupledEditorial compositionLogical compositionHas a specific audienceIs universally relevantNuanced in meaningMeaning is standardizedFocused on audience interestFocused on resource reusabilityMeaning can be independent of contextMeaning is dependent on contextBuild up meaningBreak down meaningThe bigger themeThe detailsOften proprietaryOften public domainUniqueness is valuedRegularity is valuedCares about audience attentionDoesn’t care about audience attentionSome differences between content and data What’s the issue and why does it matter?

New approaches to managing content and data such as headless content management and knowledge graphs are altering how we work with digital resources on a technical level.  Those developments have made it even more important to address what humans need when it comes to content and data, especially since the needs of people so far have not been at the forefront of these technological developments.  Too often, the resource is considered more valuable than the people who might use the resource.  It’s time to define future approaches to accessing content and data in a more human-centered way.   The first step is to be clear about the differences between how people use content and data.    

While the difference between content and data may seem like an idle philosophical question, it goes to the heart of how we conceptualize and imagine our use of resources online. Probing the distinction allows us to examine our sometimes unconscious assumptions about the value of different resources.  Content and data are basic building blocks for communication and understanding.  Yet I’ve been surprised how differently professionals think about their respective value, potential, and limitations.  My thinking about this topic has evolved as well, as I watch changes in the possibilities for working with both but also encounter sometimes idealized notions about what they can accomplish.  

Our mental model of digital resources influences how to work with and value them.  Many people consider content and data as different things, even if they can’t delineate precisely state how they differ.  They often have different perspectives of the value of each.  Some popular tropes illustrate this.  Data is the “new oil” — the value of content is to generate or extract data.  Content is “king” (or queen) – data exists to support content.   In both these perspectives, content or data are seen as raw material — a means to an end — though they disagree on what the end is.   

When we talk about content and data, we rarely define what these terms mean. This situation has been true since the early days of the web.  We’ve made little progress in understanding what various digital resources mean to people and the picture keeps getting more complex.  Content and data are becoming more intertwined, but they aren’t necessarily converging.  

On a technical level, various mental models people have about resources get translated into formal schemas.  How we architect resources influences how people can use them. 

Are content and data different categories of resources?

Professionals who work with digital resources in different roles don’t share the same understanding of how content relates to data.  Until recently that wasn’t a big problem, because content and data lived in separate silos.  People who worked with content could comfortably ignore the details of data, and those with a data focus were indifferent to content.  

Outside of those who create and manage content, content is still largely ignored in the IT world.  There’s always been more of a focus on information or knowledge.  When you avoid discussing content and understanding its role, you can slip into a shaky discussion about delivering information or providing “knowledge” to users without considering what audiences actually need.  

But the historical silos between content and data are slowly coming down, and it’s becoming more important to understand how content and data are related.  Unfortunately, it’s not so simple to express their differences succinctly, because they vary in many different dimensions.  They aren’t just words will simple definitions, but complex concepts.  

Everyday definitions of content and data don’t help us much in locating critical distinctions.  For example, Princeton University’s Wordnet lexical database provides the following short definitions of each:

  • Content: message, subject matter, substance (what a communication that is about something is about)
  • Data: information (a collection of facts from which conclusions may be drawn) “statistical data”

These definitions hint at differences, but also areas of overlap.  The distinction between a “message” and “information” is not obvious.  In everyday speech, we might say “Did you receive the message (alternatively: information) today?”  Similarly, the ideas of “substance” and “facts” seem similar.  It’s tempting to dismiss these problems as the byproduct of sloppy definitions, but I believe many of the difficulties stem from the complex and changing essence of these concepts.  

When concepts are difficult to define, many people look for examples to show what something means.  But canonical examples of familiar resources don’t help us much.  Let’s consider two ink-and-paper products that can be purchased from the US Government Printing Office.  The US Census is a canonical example of data: a compilation of cold, statistical facts.  The US Constitution is a canonical example of content — a series of statements that are rich with meaning — so much so that people passionately debate what every word means. Canonical examples can provide concrete illustrations of concepts, but most of the resources we deal with will not be so well defined.  

In the past, we categorized resources according to end user: data was for machines to process and calculate, while content was for humans to understand. Or we conceptualized content and data by the environments in which we encountered them.  For example, we viewed data as rows and columns of text and numbers in a spreadsheet or relational database.  While these could be presented to readers in a table, the rawness of the source material did not make it seem like content.  As computers began to process all our resources, the picture became more complicated.  Computers could store records, such as my university transcript, which provided an overview of my studies — a story of a sort.  Computers also stored documents.  I began using computers to create documents while in university, even though the delivered document was on paper.  Was the file on that floppy disk content or data? Conversely, I used card stock paper to create data for computers, by filling in the dots on a computer punch card — manually processing data for the benefit a comptuer.  

Based on past experience, people are often inclined to see content and data as different.  It’s also worth considering how they might be related. Earlier this month, a developer I know posted on a content strategy forum that “content is data.” Professionals working with digital resources may view content and data as having a range of relationships:

  1. They are identical (or any avowed distinctions are inconsequential)
  2. They are separate and independent from each other, with no overlap
  3. They overlap in some aspects (either sharing common properties or representing a continuum)
  4. One is a subset of the other (content is a kind of data or data is a kind of content)

I’ve encountered people who promote any one of these views, and among others.  It’s even possible to see data or content as an expendable resource with no inherent value.  For researchers in natural language processing, content is merely a “data set” — a very long string of characters to bend into shape to support different use cases.  

My own view is that content and data are fundamentally different, with only limited overlap. They coexist and complement each other, but they are distinct kinds of resources. In one specific dimension, they are becoming more alike: how they are stored and can be managed.  But they are very different in two other areas: how they are generated and created, and how they are consumed and used.

When viewed solely through the lens of technology, content and data can seem to resemble each other.  Both can be structured by models that are similar in form. Much of what makes content and data different is invisible to technology: they have different purposes, and people relate to them in different ways.  A growing source of confusion arises when individuals assume that content and data should behave the same way because they have similarities in form.  But morphological similarities are not the full story, just as the wings of pigeons, penguins, and ostriches look similar but play different roles.  

How content and data are stored and managed

Digital resources have a material presence. They take up space.  I live only a few miles from where one of the largest concentrations of server farms in the US is hosted and am keenly aware of the land and energy they need.  What’s lurking there?  What are these resources talking about?

Many developers see no intrinsic difference between data and content: both are simply digital objects with IDs.  Different models of storage — branching code repositories, file structures, XML encodings, graph databases, schema-less databases — are simply alternative ways to organize bytes of data.   Some developers refer to content as unstructured data.  According to that perspective, content is a form of data, but in a less perfect form.  The best that content can aspire to is to become “semistructured” data.

Developers encounter the terms content and data in jargon referring to the format of the resource they dealing with.  They deal with “content types” (for HTTP requests) and “data types” (for data values stored or retrieved).  For example, text can be plain or HTML.  In this sense, content and data aren’t too different: they define a discrete payload.  Small wonder developers don’t spend much time pondering the distinctions.

Within a content management system, the term “content types” appears again but in a different sense: they define the fields for an item of content, and each field needs a data type to indicate the kind of value used.  Here, content types are made from data types and might be considered a superset of data types.

In the CMS-specific definition of content types, we see signs of semantics, which break the resource into parts that have names.  The semantics help describes what the resource is about, not just how to process it as a format.  Provided a developer knows the structure of the resources, they can query it to obtain specific elements within the resource to answer questions.  

What do the parts of a resource offer and how can they be used?  These questions are often answered in API documentation, which explains the model of the resource.  Most serious CMSs now have a content API; better ones expose their entire content model. This is having radical consequences.  Content is not tied to any specific display destination such as a website, but instead becomes a dynamic resource.  With GraphQL, a fast-growing API query standard, it’s become easy to combine content from different sources.  Content publishing has the potential to become multisource: distributed and federated.  Content can be exchanged between different sources, which don’t need to have the precise same understanding of what originating source had in mind.  APIs are like a phrasebook that translates between different parties.  People don’t need to speak the identical language (schema) provided that they have the phrasebook.  This is a major shift from the past when the presumption had been that everyone needed to agree to a common schema to share their resources.

Non-technical users gain access to the model of a resource by using a UI that’s connected to it.  With the shift to the cloud, consumers are increasingly unconcerned about where resources are stored.  Many resources are accessed through apps.  Some browsers now don’t display full URLs.  The address of the content — its path and location — is disappearing and along with it a concern about structure.  While the cloud is just a metaphor, it does capture the reality that resources no longer have a fixed address.  Storage containerization means resources move around.  From the consumer’s perspective, the structure is becoming invisible — seamless.  They don’t see or care about how the sausage is made.  They only care what it tastes like.

In terms of access and storage, content and data are becoming more alike.  With APIs, content is becoming more malleable, like data.  And data is getting upgraded with more semantics, becoming more descriptive and content-like. These changes are largely invisible to the consumer, until they don’t work out.  People do notice when things are askew, though they are unsure why they are.  They care not only whether the files can be accessed, but also how coherent the experience is of getting that stream of resources.

Generating and creating resources: expression

What can a digital resource talk about?  Content and data are often discussing different concerns.  Content talks about topics or stories.  Data talks about entities.  While similar in focus, content and data have different expressive potentials.

Data and content have different perspectives about: 

  • What they can mention: the properties that are discussed
  • What can be said about said about those things: the values presented  
Restricting expression

The structuring of resources will influence what resources can discuss.  And they can impose rules on values (controlling values, validating values, or restricting values), which will further limit what can be said.  Structure can limit expression. 

Content is “open domain”: it can talk about anything.  The author decides what topic or story to talk about – they are not restricted to a pre-determined universe of topics.  Once that story or topic is chosen, they can address any aspect they want to, and there are no restrictions about the attributes of the topic.  And they can say anything they want to about; there are no restrictions about the values of those attributes.    

Content, in its untamed form, is open-ended in how it discusses things. Content doesn’t require a pre-defined structure, though it’s possible to structure content to define properties that shape what dimensions get discussed.  These may be specific fields or broader ones.  Even when content is defined into structural elements, the range of values associated with these elements is open-ended (such as free text, video, or photos) and the values can discuss anything without restriction.  Restrictions on content are few: there may be a character limit on a text field or a file size limit on an image.  A few fields will only accept controlled values.  But in general, most content is composed of values created by an author.

 Data is “closed domain”: to be useful, data must be defined with a formal schema of some kind — a set of rules.  This limits what the data can talk about to what the data schema has defined already.  

Digital resources thus vary in their structure and allowed values.  Structure is not inherently good or bad — it involves a range of tradeoffs.  What’s best depends on the intent of the contributor.  We need to consider the goal of the resource.  

With data, there’s no obvious intent.  We don’t know why the data is there, or how it is supposed to be used.  But the presumption is that others will use the data, and to make the data useful, it needs to conform to certain standards relating to its structure and allowed values.

With content, the situation is different.  All content is created with an intent in mind. Sometimes that intent is vague or poorly thought out.  But content generally takes effort to create, which means there must be a motivation behind why it exists.  And by looking at the content, we can often infer its intent.  We know there’s an audience who is expected to view the content and we can make some guesses about what that audience is expecting.  The audience’s needs define both the structure and the values used. 

While data doesn’t have an audience, content does. Data doesn’t have an author, while content does.  These distinctions have implications for how resources can be used.  

Representation and ‘aboutness’ in digital resources

What’s the resource about and what’s it trying to explain?  Again, content and data focus on different aspects.

Content and data differ in what each can represent.  Content can discuss a set of facts that don’t have a predefined relationship.  Authors make decisions about what to include and how to talk about them. Content doesn’t need to be routine in what it expresses. Data is meant to convey a predefined range of facts in a precise way.  Data depends on being routine.    

Both content and data make statements, but the values for these statements are dissimilar.  Content elements hold complex values (the value may contain several ideas.)  Data elements hold simple values (normally one idea or ID per value.)

Content deals with topics and stories that ultimately are about themes, ideas, concepts, life events, and other kinds of things that are open to interpretation.  Data describes entities — concrete things in the real world or human-defined records such as invoices.  Data can only describe properties of things that can be measured according to recognized values.  Its role is to provide a consistent understanding of an entity.

A fundamental difference, then, is the approach that each uses to describe things.  Content describes these with natural language, pictures, or other forms of human communication. Data describes them in terms of their measurable properties, or their relationships to other entities.  

Data values are simple and ideally unambiguous: names, IDs, pre-decided labels, quantities, or dates.  Content is different from those simple values.  A content value can’t be easily evaluated by machines because it contains complex, multipart statements about multiple entities.  Lots of work goes into making natural language understandable to computers — finding the themes and sentiments, or recognizing when entities are mentioned.  Despite impressive progress, machines have trouble interpreting human communication.  That machines don’t find human communication reliable does not mean it’s less accurate.  Content can be both more nuanced and more compact than data.  Content may seem less precise, but it also can represent concepts and statements that data representations can’t hope to.  Data engineers are having difficulty representing even the basic features of laws and regulations, for example.

Data breaks down facts into individual statements.  By doing so, data can manage to be both specific and incomplete.  The building blocks of data allow us to say a lot about entities.  We can map the relationships between various entities, including people.  Data can tell about a couple who marry and later divorce.  But it can’t tell us why they married or why they divorced.  The enumerated values within data models can’t address complex explanations.

Data’s fundamental purpose is to make discrete, unambiguous statements about an entity.  Data will hypostatize or reify an object. The object becomes an entify-able value: its existence is stripped down into its observable and quantifiable qualities. Once an entity is converted into values, it can be evaluated.  Even actions can be treated as entities, provided they can be enumerated into categories that are fixed in meaning.   This restriction limits the data-ification of the descriptions of processes since actions can involve so much variation.  

The ability to make discrete, unambiguous statements depends on having an agreed schema to discuss an object’s properties.  Data schemas can either be opinionated or not opinionated (an opinion being a point of view that’s not universally accepted.)   The semantics (what things mean) may involve coerced agreement — much like the terms of service you must click on to use an internet service.  How one asks questions (the syntax) can involve forced agreement as well.  Schemas can contain a range of opinions:

  • Opinionated: Everyone needs to agree to the same schema to talk about what things mean (you must accept my version of the truth.)  
  • Non-opinionated: people can define their own schemas, though they will need to learn about what others have decided if they want to use someone else’s.
  • Opinionated: everyone needs to make requests in the same way (syntax) about a set of facts defined by a schema.  
  • Non-opinionated: how one asks questions can vary and the answers (truth) are language-neutral, independent of how questions are framed. 

A deep irony — and flaw — of many efforts to structure data with a standardized schema is that these initiatives tend to dictate the use of a specific query language to access and utilize the data.  “Openness” is promoted in the name of interoperability but is done by forcing everyone to adopt a particular standard for data or queries — forcing an opinion about what’s correct and acceptable, resulting in pseudo-openness.

A schema is simply a framework that provides a context to what is being discussed.  People use mental schema to interpret the world, while machines use data schemas to do that.

 Content can have meaning independently of its immediate context, provided the content is unambiguous.  A fragment of content can often stand alone, while a fragment of data can’t.  Content doesn’t need a formal schema: it relies on shared knowledge and shared meaning.  The role of structure within content is to amplify meaning beyond the meaning carried by the words or images.  The structure of content frames  scaffolding of meaning.  This represents a big difference in how content and data approach structure.  With content, the emphasis is on using structure to build up meaning: to enlarge ideas.  With data, the emphasis is on using structure to break down meaning: to locate specific details.  

Data has meaning only within a specific context. When viewed by humans, data makes sense only as part of a record that shows the context for the values presented.   For machines, data makes sense within the hierarchical or lateral relationships defined by a schema.  The innovation of semantic data is its ability to describe the meaning of data independently of a larger record.  Semantic data creates the possibility to recontextualize data.  

Data can supply precise answers to structured questions. But data does poorly when trying to convey the meaning of concepts — the big ideas that motivate and direct our behavior.  Complex concepts are difficult to express as data.  They can be given identifiers to disambiguate them from similar words that refer to different concepts.  But even a universal ID, such as a Wikidata ID, does not make the concept of “love” clear as to its meaning.  It merely tells us we aren’t talking about a band with such a name.

Data can supply a comprehensive description only when an entity can be defined entirely by data. For example, statistical categories can be defined by data.   More often, data must rely on concepts that can only be defined using content.  Even if different resources use the same term to describe a property (such as “name”), they may define those terms differently.   

Combining details to build explanations

Another aspect of expression concerns what can be presented together to create a larger whole.  Here, we aren’t looking at what can be said by a single contributor at a single point in time, but what can be combined from different sources created at different times.

Both content and data are becoming more connected: able to combine with other similar resources.   The elements within both kinds of resources are being defined more specifically with semantics indicating their meaning or purpose.  This allows larger digital resources to be composed semi-automatically, a sort of ghost authorship.

When resources are broken into elements, they can be combined into various combinations to create new resources.  On what basis are combinations made?  It’s useful to distinguish logical composition from editorial composition.  Data is concerned with logical composition, while content is concerned with editorial composition.  

Web content historically hasn’t been structured into pieces that could be separated. All content intended for publication would be created at the same time by the same author within a single body field.  The elements within the content were tightly coupled, part of a common template.   Authors enjoyed great freedom about what they could address within an item of content but had less freedom in how their content could be combined with other content they or others created.  The presentation of a web page was fixed.

Content can be composed by stitching together statements, a process done through human curation (via links) or programmatically (by filtering and gathering similar items.)  In both cases, humans decide the rules for what kinds of things belong together to compose meaningful experiences.  The difference is that hard-coded programmatic rules are applied routinely rather than once.  

Data has always been about decoupling elements to allow them to be presented in different ways.  What the data can say might be restricted, but how it can be presented can vary in many ways.  Intrinsically, data has the flexibility to combine with other data.  It’s able to either be assembled on a one-time basis from a situational query or be generated routinely from a saved query that fires routinely.

How useful are larger resources that are assembled from smaller elements?  Not all assembled resources are equally useful.

 It is easier to break apart content into data than it is to compose content from data.  Designing a successful binding agent that turns data into content is difficult. Automated journalism can generate prose from data, but the resulting content comes across as pro forma and far from engaging.

A key difference in the structure of content and data relates to intent.  Data structure specifies the meaning of a value.  Content structure will often indicate the intent of an element as well as the meaning it conveys. For example, an author has a name, which is a data value.  The author may have a bio, which is a content value.  The bio is intended to provide context about the writer, and perhaps spark interest in what they have to say.  The bio presents some facts about the author, but its purpose is broader.

Data elements don’t have predefined purposes.  This allows them to be displayed in any number of combinations.  Content elements are less independent in how they can be displayed.  When content presents a topic, patterns of elements tend to be grouped together, and hierarchies of elements address broader and more specific aspects.  

To build an explanation, the elements of a resource need to work together to provide a unified understanding of a larger topic.  Content management has moved from a tightly-coupled structure to a loosely-coupled one, especially with the development of headless content models.  Elements can now be remixed into new presentations.  But audiences must perceive these elements as belonging together.  Loose-coupling is done by relating content types: collections of editorially-compatible elements.   

The simplest form of relations defines whether something is “part of” another or is “kind of” another.  Both these relationships enable aggregation of elements discussing smaller entities into discussions of larger ones. 

Both content and data are becoming explicit in indicating what individual elements mean. Despite that similarity, they remain different.  People confuse the concepts of structured content and structured data, yet these concepts are different in their orientation.  Both can be building blocks to generate more sophisticated resources.  But the materiality of those blocks is different.  

The structure of content reflects editorial intent.  That intent is often specific to the publisher, which is one reason why standardizations of content structure across publishers have not materialized. The area of technical documentation is a notable exception: it has tried to standardize the expression of content, with mixed results.  Some transactional content is indeed amenable to standardization: cases where people don’t want to think at all because they don’t have an opinion about the material, they trust the advice, and they don’t worry about the consequences of the advice.  API documentation is an example of documentation where the structure used by different publishers is converging, because of the high degree of similarity in its functional purpose.   

But the trend now seems to move away from making content seem like an interchangeable commodity. Content needs to sound human. The rising prominence of intention-focused approaches such as content design and UX writing reflect a growing public wariness toward cookie-cutter content for even “dry” technical and instructional topics.  Audiences don’t trust content that seems formulaic, because it sounds repetitive — even robotic. When all content follows the same limited patterns, it looks the same and people have trouble noticing what is different about each piece.  They question predefined answers that seem too tidy and lack background explanation.  Content needs to sound conversational if it hopes to garner attention — that’s true even for technical, factually-oriented content.  People hesitate if they feel they’re are being blinded by details — snowed.  Every detail should seem necessary to the larger purpose of what they are seeing. 

Editorial intent is about providing coherence to audiences.  The structure of content needs to support coherence.  It’s the opposite of the data-centric approach of the “mash-up”:  a jarring experience involving a mishmash of items that were never meant to be presented together.  

To assemble content elements successfully into a whole, the pieces need to be designed so they fit together. The pieces should support one another to provide a richer explanation than they would if they were viewed individually.  Generic standards for content elements have never seemed coherent: they have prioritized splitting things apart rather than gluing things together.  Most of them focus on factually heavy content that few people would want to read in total.  It’s possible to combine content from different sources, but the editorial intent of each element needs to fit together. The structure should be seamlessly supporting the larger message.  It shouldn’t be fragmenting the topic to where the relationship among elements is not reinforced.

Public expectations for data are different.  For data to be considered reliable, people expect it to look “regular.”  Consumers don’t want to see an asterisk appended to data, a footnote explaining some nuance.  Data is less trustworthy when posing as solid facts but presented tentatively or teasingly.   It also becomes less trustworthy when it is presented in inconsistent ways.  We expect data to be predictable and wonder what’s being hidden if it is presently in an unexpected manner. Predictable routines are more effective for data.  The regularity in data makes it seem more trustworthy and reliable.  People expect data to offer accuracy, not broader meaning. 

While content and data are different, there’s still a big push to treat content as if it was data.  We’ve seen how earlier attempts to do this, such as mash-ups, were widely unsuccessful because they were incoherent and fragmented the experience for audiences.  More recently, data enthusiasts have promoted the application of semantics to eliminate differences in how publishers describe and use their content.  It’s important to address the potential and limitations of semantics. 

Digital resources rely on semantics. But in the view of some, these resources don’t rely on semantics enough.  Some argue that all digital resources can and should be described with a common model: the RDF data model.   

The RDF data model represents the vision of what the inventor of the internet, Sir Tim Berners-Lee, thought the internet should become: a connected body of commonly described data, or linked data. I’ve seen efforts to use the RDF data model to publish content rather than data.   But I’ve never seen an RDF data model that reflects editorial priorities, structuring content the way audiences expect them.  RDF works well for data but when used for content, it delivers dull, fact-based pages that at best resembles an encyclopedia. An interesting art museum collection gets transformed into a lifeless database dangling with confusing categories and options. These efforts reflect a belief that details are more important than narratives.   

Wikipedia, the most popular encyclopedia on the web, starts with content, rather than data. Wikipedia offers templates to provide an editorial framework for the content, which enables structural uniformity but doesn’t impose it.  Wikipedia is one of the most advanced experiments attempting to harmonize content with a data model, but it highlights the limitations that even a single publisher can have doing that.  It’s possible to extract data from Wikipedia but the complexity of its content has defied efforts to normalize it into a predictable data model.  

How resources are consumed and used

The flip side of expression is interpretation.  The meaning of a resource is not only a function of how it’s represented — the semantics.  Its meaning depends on how it’s received and evaluated — what humans and machines notice and understand.  

 The possibilities for machine-assisted interpretation are growing as data becomes richer, especially when it is highly curated.  People consume data — the fields of data visualization and data storytelling, for example, have exploded in the past decade, and these human-centered experiences can be machine-generated.  And we can no longer assume that stuff traditionally considered content — narrative text, for example — is never consumed by machines.  The range of what machines can do keeps growing, but they remain fundamentally different from humans in what they aim to accomplish.  

Who’s the resource for?

Let’s look at who needs or will want to use a resource. Does everyone need it, or only some people?  Many debates about the importance of standards used to describe resources arise because of different presumptions about who needs them.  Often the audience for the resource is never clearly defined.

Those who advocate for standards believe that everyone needs a shared basis of understanding: a common schema. A related belief is that everyone needs to know the same kinds of things from the resource, so using one common standard will be adequate for everyone.  Those who don’t embrace common standards consider them as extra work, often getting in the way of what’s needed to provide a complete explanation.    

If everyone agrees about the semantics — what to describe and what that signifies —  it can amplify the understanding and the potential utility of resources.  But reaching a universal agreement is difficult to achieve because people in practice want different things from resources.  They have different goals, preferences, priorities, and beliefs about the utility of various resources. What gets represented in standards is a reflection of what the standards makers want to be consumed.  Some people’s priorities don’t fit within what the standards committee is interested in supporting.  And when resources rely on semantic conventions outside of the agreed mainstream, it can potentially limit how those resources are acknowledged by IT systems that rely on these mainstream standards.  Standards are a closed system: they encourage connections within the universe of standards-adopters while indifferently ignoring outsiders.  Common standards can promote shared understanding. But they can also throttle expression by limiting the scope of what can be said.  

Data needs a schema to indicate how different bits of information are related to one another — otherwise, the data is not intelligible. Many data schemas are self-defined to reflect the requirements of the publisher.  Some data only moves around within a single publisher.  But because data is commonly exchanged between different parties, data will often utilize a common schema of some sort.  Externally-defined standards support exchanging resources with other parties. But it’s a mistake to see them as frictionless and cost-free.  

Digital resources that adhere to “open standards” present themselves as universally usable and available.  But in practice, they can become a walled garden that’s not accessible to all, due to stealth technical hurdles or barriers.  To understand this apparent paradox, one first needs to step back and ask how inclusive the standard’s development and adoption are. Standards are created by committees of often like-minded people who have specific agendas about what they want the standard to do.  If a narrow group was responsible for developing a standard, it may look like a consensus but it may never have achieved broad interest and wide adoption.  This can happen when standards are opinionated — they seem to work great, as long as one’s willing to sign on to the concepts, presumptions, and limitations of the standard.  And every standard has these constraints, though few are keen to advertise them.  Practices that are looser in their demands tend to be more flexible (and practical) and hence more widely adopted, and in the process become de-facto standards that are more impactful than many official ones.  

Data relies on APIs to broker what’s needed by different parties.  For people using apps that must access data from various sources, machines need to exchange data and APIs indicate how to do that.  Frequently, only select data needs to be available to a limited number of other parties. In order words, everyone does not need everything. A lot of data is personal to an individual: your streaming music playlist, your car service history, or your vacation plans.  While some people want to broadcast their personal data, many people are reluctant to do that, concerned about their personal privacy and cybersecurity.  

Most data is exchanged using custom APIs, where the provider of data describes its schema. Most data does not conform to a universal standard such as the RDF data model, and only a minority of IT systems support RDF.  

Yet some data is considered public and is made universally available.  Data described with RDF can connect easily to other data that are  described with that model.  RDF is most commonly used for non-proprietary data since there’s an incentive to connect such data widely.  

The audience for RDF-defined data is unique in several respects.  RDF schemas tend to be designed by technical people for other technical people: engineers, scientists, or data analysts. The people creating the schema are largely the same people using the data described by the schema. They share a mental model of what’s important. The data that’s targeted is presumed to be a public good: it should be available to everyone, who can access it by using a commonly adopted schema.  Semantic standards assume that data must have a universal identifier because anyone might need to use the data in its raw form.  This sentiment is most expressed in the notion of “open data”: that data must be downloadable and reusable by anyone — you agree to surrender rights to how what you create is used.  The “authorship” of data, unlike content, is often anonymous.  Data is easily replicated, becoming widely available.  It can quickly become public domain, where it’s not unique enough to merit copyright.

Unlike data, content is neither private nor public.  It’s meant to be widely used but normally it’s neither individually-focused or universally needed.  Not everyone will want or potentially need the same content.  They may want the content from specific sources since the source of the content is part of the context audiences consider when evaluating the content. Content is generally proprietary: it isn’t meant to mix freely with content from other sources.  Content normally has an identifiable individual or institutional author, unlike the case with much data.  To have value, content should be unique, and by extension be subject to copyright.   

For a scientist, any data is potentially useful because you never know ahead of time what facts might be connected to others.  For a teenager viewing a smartphone screen, only some content will ever be of interest.  Content, more than data, is situationally relevant; the requirement of having a universal identifier is not as great.  Content needs an identifier of some sort, but since not everyone in the world needs to be able to access all content, every piece of content doesn’t need a globally unique ID.  Most people only need a way to access the assembled content, such as a web page.  They don’t need access to the individual elements that comprise that web page. That task is delegated to an API, which worries about interpreting what’s wanted with what’s available.

Government-funded organizations often have an obligation to make everything they publish be described using an open standard. Public funding of data tends to drive data openness.  But the data itself may only be of interest to a small group of people.  Just because data utilizes common standards doesn’t imply there’s a universal demand for the data.  Data about the DNA of fruit flies or archeological artifacts from Crete may be of public interest, but they aren’t necessarily of universal interest.

What’s available to use? 

How resources can be used to some extent depends on the relationship between the parts and the whole.  

  • Do the parts have meaning when separated from the whole?  
  • Do different parts when combined resemble a coherent whole?

We can think about digital resources as fulfilling two different scenarios.  First, we have cases where the user is expecting something specific from the resource: the known-knowns.  They have a question to get answered, or a predefined experience they seek, such as listening to a specific music track.  Second, we have cases where the user doesn’t have strong expectations: either known-unknowns or even unknown-knowns.  

The issue more complex than first appears because we haven’t fully defined who the user is.  Does the user directly make their choices on their own, or do they rely on an intermediary (a machine or editor) to guide their choice?

What’s the resource for?

Both data and content can answer questions, but the kinds of questions they can answer are different.  Questions vary:

  • What does the user want to know?  
  • Do they have a specific question in mind?  
  • Does the answer provide a complete picture of what they need to know, or does it hide some important qualification or comparative? 

This distinction between data and content has practical consequences for the design of APIs that offer answers. What can be queried?  Does the value need a label to explain what it is?  Is the author of an API query a curator of content, or the seeker of knowledge?  A curator intermediator will be inclined to read the API documentation, while the knowledge seeker wants direct answers.

Both people and machines consume resources — they try to make sense of them and act on what they say.  When resources are consumed, the identity of what’s being discussed in the resource can be a vexing issue.  What is it talking about, and is it the right thing of interest?  Data IDs can help us be more precise in specifying and locating items, but they can also be misleading.  Both machines and humans presume that identical strings of characters indicate equivalent items.  In the case of humans, this is most true when the string is uncommon and assumed to refer to something unique.  Machines are normally less discriminating about how common something is: they are greedy in making inferences unless programmed not to.  Machines assume every matching set of strings refers to the same thing.  

Of course, just because different items have the same string label or ID, that doesn’t mean they are referring to the same idea.  The hashtags in folksonomies illustrate this problem.  Catchy words or phrases can become popular but have diverging meanings.  

Humans have trouble when talking about concepts, because the same word or phrase may imply different things to different people, what’s known as polysemy.  Nearly everyone has their own idea about what love means, even while nearly everyone is happy to use this four-character string.  Identity (labels or IDs) is not the same as semantics (shared meaning).  In natural language conversation, ambiguity is a recognized problem and to some extent expected, with clarifying questions a common countermeasure.  Suppose a speaker is talking about “personalization.”  The listener may wonder: what does he mean?  The listener may supply her own definition, which may be the same as the speaker’s, or slightly different — or vastly different.  The listener may use a different name to refer to what the speaker considers “personalization” — to the listener, the speaker is talking about “customization.”  It’s also possible that the speaker and listener agree about the definition of personalization, but still conceptualize it differently: one sees it as a specific kind of algorithm while the other considers it a specific kind of online behavior.  Even shared definitions don’t imply shared mental models, which involve assumptions outside of simple definitions.  

 Concepts, whether familiar or technical, often lack precise boundaries or universal definitions.  Humans often encounter others using the same terminology to refer to a concept, only to find that others don’t embrace the same meaning or perspective about it.  

Much of what passes for semantics in the computer realm is more about agreed identifiers than agreed meaning.  A “thing” that can be precisely identified can have multiple identities — depending on who is perceiving.  Machines are unable to distinguish the role of denotation from connotation.  Consider the s an emoji of a facial expression, which has a specific Unicode ID.  The Unicode ID denotes a visual face of some sort.  But the meaning of that facial expression emoji is subject to interpretation.  A single Unicode character can generate polysemy — multiple interpretations.  A Globally Unique ID can spark false confidence that everyone will understand the entity in the same way.

The promise and peril of IDs are that they act as stand-ins for a bunch of statements.  In natural language, we’d call them loaded-phrases: they trigger all kinds of associations. Is the terminology or labels used for a concept the changing, or is the identity of concept changing?   But maybe the ID isn’t precise?  We tend to describe things, and how we describe things tends to define them.  Maybe the descriptive properties aren’t relevant to what’s being said.  Maybe they aren’t useful at all.  

 Shades of truth

Data are about indirect experience.  They provide detached information that’s been recorded by someone or something else.  Data are generally treated as “facts” — they’re considered objective.  Content doesn’t represent itself so absolutely.  It speaks about the author or reader’s direct experience — personally acquired or understood information.  Content is about expression, which involves individual interpretation.  Notionally, data is free of interpretation, while content supplies it.  

As mentioned earlier, data is exchanged between systems and so needs to be defined with enough precision to allow that to happen.  The value of data (in theory at least) is independent of its source.  Its utility and accuracy are presumed so where the data comes from is less a concern.  Do we care who publishes a list of US presidents or where Google gets its answers presented in its knowledge panel box?  If we believe data is intrinsically objective — that data isn’t false or misleading — we don’t.  

Content — subjective and embodying an editorial point of view — is a reflection of its publishing source.  Audiences evaluate content partly by the reputation of the source. They evaluate content not just for its factual accuracy but its completeness, fairness, insight, transparency, and other qualities.   People value content for being unique.  Content can’t always easily measured or compared directly against other content.  People rely on their judgment rather than some calculated comparison.  

The presumption of knowledge graphs is that KNOWLEDGE (yes it’s a big shouty idea) can be reduced to data.  A less glamorous name for knowledge graphs is linked data, the term that Tim Berners-Lee championed two decades ago. The terms knowledge graphs and linked data are nearly synonymous, the main difference being that knowledge graphs involve a curated set of data (curation = editorial choices).  Because knowledge graphs are promoted as the answer to nearly every problem and possibility facing humankind, it’s important to be clear what knowledge graphs can and can’t do.  Knowledge graphs advance our ability to connect different facts and bring transparency to many domains. They are especially useful in supporting internally-focused enterprise use cases where experts share data with their colleagues, who can interpret the meaning and significance based on a shared understanding of its significance. Knowledge graphs are indeed useful in many scenarios.  But we must accept that the “knowledge” aspect of the term is a bit of marketing hyperbole, coined by Google.  Data, by itself, no matter how elaborately connected and explicated, does not generate knowledge.  Because knowledge requires explanation, while data can only show self-evident things.  Content, in contrast, is about presenting information that’s not self-evident to audiences.  Content explains facts.  Knowledge graphs link facts.  

Many possibilities are available to connect items of data together especially when joining tables or federating search across different sources.  But the output of data queries is still limited in what it can express.  Data are related through operators, such as equals (is), comparisons such as more than or before, and inclusion (has).  That’s a small set of expressions compared to natural language.  Semantic data is more expressive than ordinary relational data because its properties function like verbs in natural language.  But the constraint remains. For data to be manageable, the properties must be enumerated into a limited list of verbs.  Nuance is not a forte of data.  

Data’s relevance challenge

How does data establish its relevance to audiences?  Unlike content,  data was never created with a specific audience need in mind.  So how can it become relevant to people?  

Content APIs are designed around basic questions: what content will be relevant to audiences?  The API delivers specific content that provides the answer.  The body of content will be relevant to different audiences for different reasons, though no single individual will necessarily be interested in all of it. The content was created to answer specific questions.  The API’s task is to determine which slice within the content body is needed for a specific context at a specific time.  Answer-responses can be programmatically delivered using a flexible range of simple declarative questions.

Data, being more open-ended in how it can be queried, has more difficulty providing relevance reliably.  Data can answer straightforward questions easily: what’s the cheapest gadget that’s in stock, or the highest-rated gadget?  While the result doesn’t provide much explanation, the answers can point audiences to content items they may want to view content to understand more.

Knowledge graphs are supposed to turn general-purpose data into something that will be understandable and relevant to the casual inquirer. They’re able to do that when the user understands the domain already, but even then the relevance of results is uncertain.  Using knowledge graphs to extract valuable insights from data can be a deeply labor-intensive exercise  — a treasure hunt in an age when consumers expect systems will automatically tell them what they need to know.

Knowledge graphs are generally built from graph databases, which connect different types of entities and reveal their indirect relationships.  These databases have been difficult to translate into audience-facing applications.  People don’t know what to ask when dealing with indirect relationships: what relationships are potentially valuable to explore.  For this reason, we see few consumer applications using the SPARQL, the most commonly used specialized query language for knowledge graphs.  

Instead, most consumer recommendation engines operate using different sort of graph database called property graphs that are based on self-defined semantics rather than universally agreed ones.  Property graphs are optimized to find the shortest path or distance between two objects.    The goal of the recommendation is to highlight the strength of the connection rather than providing a way for the customer to drill into the multifaceted relationships.  Recommendation engineers are fundamentally different in purpose from faceted filtering.  They remove complexity from the user, rather than expose users to it.

The goal of using semantic data to “reason” — to draw non-explicit conclusions — hasn’t materialized in mainstream consumer applications.  Most semantic data is used to answer explicit questions, albeit sometimes highly complex ones.  One benefit of semantic data over traditional data is that the questions asked can be more elaborate because each item of data ultimately is relatable to other data.  The cost of this benefit is high: the answer to any question may be null — because it’s not obvious what questions yield useful answers.    The general public has limited interest in chaining together elaborate queries. They tend to ask straightforward questions.  

When questions become complex and topics are opaquely enmeshed, systems can’t expect audiences to articulate the question: they need to anticipate what’s needed.  While experts are interested in how an answer was arrived at, ordinary users on the web are more interested in getting the answer.  They want to be able to rely on curated queries that ask interesting questions that have meaningful answers.  

Richness and ambiguity in content and data

Whether content or data provides richer expression depends on many factors.  Both can be vague.  Using many words won’t necessarily make things clear.  And presenting many facts doesn’t tell you everything you need to know.

One measure of the value of a resource is its succinctness. A succinct resource can convey much.  Alternatively, it may flatten out important nuances.  Data’s value is associated with its  facticity, where facts speak for themselves and no one entertains deviant ideas about what those facts mean.  Content’s value isn’t about crystalized facts or named entities.  While content is often reduced to being about words, it’s actually about wording — interpretation, understanding, and feeling.  

Data provides confident answers to narrowly defined questions.  Content provides richer but more uncertain answers to broadly-defined ones.  

The limits of decomposition

People learn a lot by focusing on the core facts — nuggets that can be translated into data — within statements in content.  But that focus also carries a risk: the attractive but mistaken idea that all content can be boiled down to data.  People often want direct answers, which we expect can be filtered from facts.  But answers also often involve interpretations, which go beyond agreed facts. Our interpretations can diverge, even if  everyone agrees with what the facts are.

Consider a common, simple question: who is a movie for?  The content of a film can be reduced to a maturity rating. In principle, these are based on clear criteria, but in practice, they can be difficult to apply unambiguously and consistently.  Classification is often based on patterns rather than criteria satisfaction.  And the maturity rating still doesn’t tell us much about who the movie is for, even if we agree the rating is accurate.  

According to the ideas of semantic data, almost anything can be defined in terms of its properties.  An entity should explainable through its data values.  In practice, such descriptions only work for simple entities with regularized parameters — generally human-made things or abstract archetypes.  Complex things that have evolved organically over time, whether products, living beings, or ideas, can’t be easily reduced to a handful of data parameters.  They defy simple data-explication, because of:

  • Tacit qualities that can’t be articulated easily  
  • Complex attribute interactions, including irregular combinations and exceptions
  • The overloading of dimensions, where there are too many dimensions too many to track easily

The mundane act of identifying a bird species illustrates the issue. For many related species, it can be hard to do, as each bird species can have many attributes that vary, and many bird species have similar attributes that generate confusion in identification.  Birders rely on evaluating the “jizz” of birds. Many categories we rely on are concepts that have general tendencies rather than absolute boundaries.  For this reason, criteria-based definitions don’t aren’t satisfactory.  

Data is not also good at representing many processes and how they can change the state of things.  Consider the many ways an onion can be processed resulting in different outcomes with different property values.  The cut of onions (diagonal, minced) and cooking method (steamed, sautéed, stir-fried, deep fried) influences texture, sweetness, and so on.  The input properties influence the contours of the output properties but don’t dictate them.  There are too many subtle variables to model accurately in a data model.  Content is more efficient discussing these nuances.

Food is born from recipes: a structure of inputs yielding an output.  It’s become a stock metaphor for illustrating how content or data works.  Those who believe “content is data” love to cite the example of food recipes.  Recipes are structured, they follow a standard convention, and deal with entities and quantities.  But contrary to what most people think, recipes aren’t really data.  They are content.  They are full of nuance.

A recent semantic data research paper explored how knowledge graphs could allow substitutions in recipes.  The authors, at IBM and the Rensselaer Polytechnic Institute, explored the potential of a “knowledge graph of food.”  They had the goal of creating a database that could allow people to switch out a recipe ingredient to use something more healthful. The authors figured ingredients can be quantified according to their nutritional values: vitamins, calories, fat content, etc.  “We use linked semantic information about ingredients to develop a substitutability heuristic for automatically ranking plausible ingredient substitutions.” While the goal is laudable, the project in many ways was naive.  The authors hoped that a simple substitution would be possible with no second-order effects.  But ingredients interact in complex ways, much like the details of content do. If you want to replace cream in a recipe, there are many options with different tradeoffs. Suppose we think about recipes as formulas of ingredients.  When one ingredient changes, it changes the formula and the outcome.  The problem is that food isn’t only about chemical properties.  It’s about experiential properties.  People react to food according to its sensory qualities: taste foremost, but also texture, aroma, and appearance.  The authors acknowledge that “the quality of an ingredient substitution can be subjective and difficult to concretely determine.”  But they don’t dwell on that because it’s a problem they can’t solve.  Any cookbook writer will tell readers that substitutions are possible, but they will change the character of the dish and may necessitate other ingredient changes and compensating measures.  

When resources discuss things that are important priorities to people, they will address numerous factors they care about.   The whole will be more complex than the sum of its parts.  

Content and Data as Equals

People rely on both content and data.  It’s a mistake to view one as superior to the other.  

From the perspective of user experience, people relate to events in the world on three levels:

  1. Phenomena that we perceive or notice: the content of our experience
  2. How we identify, classify, and characterize that phenomena: the facts or data we ascribe to what we’ve encountered
  3. How we explain and evaluate the phenomena: the content of our personal explanations

Our perceptions translate into properties of an entity.   Our characterizations translate into its values.  Our explanations are complex and open-ended, like content.

Data can’t replace content.  A knowledge graph query will tell you very little about the state of knowledge graph research.  You’ll need to read PDFs of conference papers to learn that.

And modern digital content can’t be viable without drawing on data to explain entities in concrete detail.  Data can highlight what’s available within the content and clarify its details.  Data enhances content.

Standards are valuable when many different people need to do the exact same thing. A basic philosophical disagreement concerns how similar or dissimilar people’s needs are.  On one extreme is the tendency to consider every resource as unique, with no reusability. That’s the old model of web content, where data was a second-class citizen.  On the other extreme is the belief that everyone needs the exact same resources and that these can be described with standardized structure.  People cease being unique.  Content is a second-class citizen.

Content needs to move forward to develop internal publishing standards for structure: internal schemas that allow coherent assembly of different parts into meaningful wholes. That improves the range of combinations that content can present to be able to address the needs of different individuals. But the standardization of content structure can only go so far before it gets homogenized and loses its interest and value to audiences.  It’s not realistic to expect content publishers to adopt external schemas for content: they will lose their editorial voice if they do so.  

For data to increase its utility, it will need to start considering its editorial dimensions, especially who the data is will be for and what do those people need. As public wariness toward data collection by big institutions builds, ordinary people need to feel data can serve their needs and not just the needs of experts or powerful corporations.  Data curators are needed to look at how data can empower audiences.   

Data and content both have structures, which allow them to be managed in similar technical ways. Both content and data can be queried with APIs, and APIs can combine both content and data from different sources.  But despite that commonality, they have different purposes.  The goal should be to make them work together where it makes sense to, but not expect one to be hostage to each other.  

The technical possibilities for transforming content and data have never been greater.  It can be easy to become dazzled by these possibilities and idealize how beneficial they will be for people.  Too little of the discussion so far has focused on what ordinary people need. Experts need to engage more with non-experts to learn what makes digital resources truly meaningful.  

— Michael Andrews

The post Revisiting the difference between content and data appeared first on Story Needle.

XML, Latin, and the demise or endurance of languages

23 november 2020 - 6:02pm

We are living in a period of great fluctuation and uncertainty.  In nearly every domain — whether politics, business, technology, or health policy — people are asking what is the foundation upon which the future will be built.  Even the very currency of language doesn’t seem solid.  We don’t know if everyone agrees what concepts mean anymore or what’s considered the source of truth.

Language provides a set of rules and terms that allow us to exchange information.  We can debate if the rules and terms are good ones — supporting expression.  But even more important is whether other groups understand how to use these rules and terms.  Ubiquity is more important than expressiveness because a rich language is not very useful if few people can understand it.

I used to live in Rome, the eternal city.  When I walked around, I encountered Latin everywhere: it is carved on ancient ruins and Renaissance churches.  No one speaks Latin today, of course. Latin is a dead language.  Yet there’s also no escaping its legacy.  Latin was ubiquitous and is still found in scattered around in many places, even though hardly anyone understands it today.  Widely used languages such as Latin may die off over time but they don’t suddenly disappear.  Slogans in Latin still appear on our public buildings and monetary currency.  

I want to speculate about the future of the XML markup language and the extent to which it will be eternal.  It’s a topic that elicits diverging opinions, depending on where one sits.  XML is the foundation of several standards advocated by certain content professionals.  And XML is undergoing a transition: it’s lost popularity but is still present in many areas of content. What will be the future role of XML for everyday online content?  

In the past, discussions about XML could spark heated debates between its supporters and detractors.  A dozen years ago, for example, the web world debated the XHTML-2 proposal to make HTML compliant with XML. Because of its past divisiveness, discussions comparing XML to alternatives can still trigger defensiveness and wariness among some even now. But for most people, the role of XML today is not a major concern, apart from a small number of partisans who use XML either willingly or unwillingly.  Past debates about whether XML-based approaches are superior or inferior to alternatives are largely academic at this point. For the majority of people who work with web content, XML seems exotic: like a parallel universe that uses an unfamiliar language.   

Though only a minority of content professionals focus on XML now, everyone who deals with content structure should understand where XML is heading. XML continues to have an impact on many things in the background of content, including ways of thinking about content that are both good and bad.   It exerts a silent influence over how we think about content, even for those who don’t actively use it. The differences between XML and its alternatives are rarely directly discussed much now, having been driven under the surface, out of view — a tacit truce to “agree to disagree” and ignore alternatives.  That’s unfortunate, because it results in silos of viewpoints about content that are mutually contradictory.  I don’t believe choices about the structural languages that define communications should be matters of personal preferences, because many consequences result from these choices that affect all kinds of stakeholders in the near and long term.  Language, ultimately, is about being able to construct a common meaning between different parties — something content folks should care about deeply, whatever their starting views.

XML today

Like Latin, XML has experienced growth and decline.  

XML started out promising to provide a universal language for the exchange of content.  It succeeded in its early days in becoming the standard for defining many kinds of content, some of which are still widely used.  A notable example is the Android platform, first released in 2008, which uses XML for screen layouts. But XML never succeeded in conquering the world by defining all content.  Despite impressive early momentum, XML for the past decade seems to be less important each passing year.  Android’s screen layout was arguably the last major XML-defined initiative.  

A small example is XML’s demise of RSS feeds.  RSS was one of the first XML formats for content and was instrumental in the expansion of the first wave of blogging.  However, over time, fewer and fewer blogs and websites actively promoted RSS feeds.  RSS is still used widely but has been eclipsed by other ways of distributing content.  Personally, I’m sorry to see RSS’s decline.  But I am powerless to change that.  Individuals must adapt to collectively-driven decisions surrounding language use.    

By 2010, XML could no longer credibly claim to be the future of content.  Web developers were rejecting XML on multiple fronts:

  • Interactive websites, using an approach then referred to as AJAX (the X standing for XML), stopped relying on XML and started using the more web-friendly data format known as JSON, designed to work with Javascript, the most popular web programming language. 
  • The newly-released HTML5 standard rejected XML compatibility.  
  • The RESTful API standard for content exchange started to take off, which embraced JSON over XML.  

Around the same time, web content creators were getting more vocal about “the authoring experience” — criticizing technically cumbersome UIs and demanding more writer-friendly authoring environments.  Many web writers, who generally weren’t technical writers or developers, found XML’s approach difficult to understand and use.  They preferred simpler options such as WordPress and Markdown.  This shift was part of a wider trend where employees expect their enterprise applications to be as easy to use as their consumer apps. 

The momentum pushing XML into a steady decline had started.  It retreated from being a mainstream approach to becoming one used to support specialized tasks.  Its supporters maintained that while it may not be the only solution, it was still the superior one.  They hoped that eventually the rest of the world would recognize the unique value of what XML offered and adopt it, scrambling to play catch up.  

That faith in XML’s superiority continues among some.  At the Lavacon content strategy conference this year, I continued to hear speakers, who may have worked with XML for their entire careers, refer to XML as the basis of “intelligent content.”  Among people who work with XML, a common refrain is that XML makes content future-ready.  These characterizations imply that if you want to be smarter with content and make it machine-ready, it needs to be in XML.  The myth that XML is the foundation of the future has been around since its earliest days.  Take the now-obscure AI markup language, AIML, created in 2001, which was an attempt to encode “AI” in XML.  It ended up being one of many zombie XML standards that weren’t robust enough for modern implementations and thus weren’t widely used.  Given trends in XML usage, it seems likely that other less mainstream XML-centric standards and approaches will face a similar fate.  XML is not intrinsically superior to other approaches.  It is simply different, having both strengths and weaknesses.  Teleological explanations  — implying a grand historical purpose — tend to stress the synergies between various XML standards and tools that provide complementary building blocks supporting the future. Yet they can fail to consider the many factors that influence the adoption of specific languages.  

The AIML example highlights an important truth about formal IT languages: simply declaring them as a standard and as open-source does not mean the world is interested in using them.  XML-based languages are often promoted as standards, but their adoption is often quite limited.  De facto standards — ones that evolve through wide adoption rather than committee decisions — are often more important than “official” standards.  

What some content professionals who advocate XML seem to under-appreciate is how radically developments in web technologies have transformed the foundations of content.  XML became the language of choice for an earlier era in IT when big enterprise systems built in Java dominated.  XML became embedded in these systems and seemed to be at the center of everything.  But the era of big systems was different from today’s.  Big systems didn’t need to talk to each other often: they tried to manage everything themselves.  

The rise of the cloud (specifically, RESTful APIs) disrupted the era of big systems and precipitated their decline.  No longer were a few systems trying to manage everything.  Lots of systems were handling many activities in a decentralized manner.  Content needed to be able to talk easily to other systems. It needed to be broken down into small nuggets that could be quickly exchanged via an API.   XML wasn’t designed to be cloud-friendly, and it has struggled to adapt to the new paradigm. RESTful APIs depend on easy, reliable and fast data exchanges,” something XML can’t offer. 

A few Lavacon speakers candidly acknowledged the feeling that the XML content world is getting left behind.  The broader organization in which they are employed  — marketing, developers, and writers — aren’t buying into the vision of an XML-centric universe.  

And the facts bear out the increasing marginalization of XML.  According to a study last year by Akamai, 83% of web traffic today is APIs and only 17% is browsers.  This reflects the rise of smartphones and other new devices and channels.  Of APIs, 69% use the JSON format, with HTML a distant second. “JSON traffic currently accounts for four times as much traffic as HTML.” And what about XML?   “XML traffic from applications has almost disappeared since 2014.”  XML is becoming invisible as a language to describe content on the internet.

Even those who love working with XML must have asked themselves: What happened?  Twenty years ago, XML was heralded as the future of the web.  To point out the limitations of XML today does not imply XML is not valuable.  At the same time, it is productive to reality-check triumphalist narratives of XML, which linger long after its eclipse.  Memes can have a long shelf life, detached from current realities.  

XML has not fallen in favor because of any marketing failure or political power play.  Broader forces are at work. One way we can understand why XML has failed, and how it may survive, is by looking at the history of Latin.

Latin’s journey from universal language to a specialized vocabulary

Latin was once one of the world’s most widely-used languages.  At its height, it was spoken by people from northern Africa and western Asia to northern Europe.

The growth and decline of Latin provides insights into how languages, including IT-flavored ones such as XML, succeed and fail.  The success of a language depends on expressiveness and ubiquity.

Latin is a natural language that evolved over time, in contrast to XML, which is a formal language intentionally created to be unambiguous.  Both express ideas, but a natural language is more adaptive to changing needs.  Latin has a long history, transforming in numerous ways over the centuries.

In Latin’s early days during the Roman Republic, it was a widely-spoken vernacular language, but it wasn’t especially expressive.  If you wanted to write or talk about scientific concepts, you still needed to use Greek.  Eventually, Latin developed the words necessary to talk about scientific concepts, and the use of Greek by Romans diminished.  

The collapse of the Romain Empire corresponded to Latin’s decline as a widely-spoken vernacular language.  Latin was never truly monolithic, but without an empire imposing its use, the language fragmented into many different variations, or else was jettisoned altogether.  

In the Middle Ages, the Church had a monopoly on learning, ensuring that Latin continued to be important, even though it was not any person’s “native” language.  Latin had become a specialized language used for clerical and liturgical purposes.  The language itself changed, becoming more “scholastic” and more narrow in expression. 

By the Renaissance, Latin morphed into being a written language that wasn’t generally spoken. Although Latin’s overall influence on Europeans was still diminishing, it experienced a modest revival because legacy writings in Latin were being rediscovered.  It was important to understand Latin to uncover knowledge from the past — at least until that knowledge was translated into vernacular languages.  It was decidedly “unvernacular”: a rigid language of exchange.  Erasmus wrote in Latin because he wanted to reach readers in other countries, and using Latin was the best means to do that, even if the audience was small.  A letter written in Latin could be read by an educated person in Spain or Holland, even if those people would normally speak Spanish or Dutch.   Yet Galileo wrote in Italian, not Latin, because his patrons didn’t understand Latin.  Latin was an elite language, and over time size of the elite who knew Latin became smaller.

Latin ultimately died because it could not adapt to changes in the concepts that people needed to express, especially concerning new discoveries, ideas, and innovations.

Latin has transitioned from being a complete language to becoming a controlled vocabulary.  Latin terms may be understood by doctors, lawyers, or botanists, but even these groups are being urged to use plain English to communicate with the public.  Only in communications among themselves do they use Latin terms, which can be less ambiguous than colloquial ones. 

Latin left an enduring legacy we rarely think about. It gave us the alphabet we use, allowing us to write text in most European languages as well as many other non-European ones.  

XML’s future

Much as the collapse of the Roman Empire triggered the slow decline of Latin, the disruption of big IT systems by APIs has triggered the long term decline of XML.  But XML won’t disappear suddenly, and it may even change shape as it tries to find niche roles in a cloud-dominated world.  

Robert Glushko’s book, The Discipline of Organizing, states: “‘The XML World’ would be another appropriate name for the document-processing world.”  XML is tightly fused to the concept of documents — which are increasingly irrelevant artifacts on the internet.  

The internet has been gradually and steadily killing off the ill-conceived concept of “online documents.”  People increasingly encounter and absorb screens that are dynamically assembled from data.  The content we read and interact with is composed of data. Very often there’s no tangible written document that provides the foundation for what people see.  People are seeing ghosts of documents: they are phantom objects on the web. Since few online readers understand how web screens are assembled, they project ideas about what they are seeing.  They tell themselves they are seeing “pages.” Or they reify online content as PDFs.  But these concepts are increasingly irrelevant to how people actually use digital content.  Like many physical things that have become virtual, the “online document” doesn’t really resemble the paper one.  Online documents are an unrecognized case of skeuomorphism.

None of this is to say that traditional documents are dead.  XML will maintain an important role in creating documents. What’s significant is that documents are returning to their roots: the medium of print (or equivalent offline digital formats).  XML originally was developed to solve desktop publishing problems.  Microsoft’s Word and PowerPoint formats are XML-compatible, as is Adobe’s PDF format. Both these firms are trying to make these “office” products escape the gravity-weight of the document and become more data-like.  But documents have never fit comfortability in an interactive, online world.  People often confuse the concepts of “digital” and “online”.  Everything online is digital, but not everything digital is online or meant to be.  A Word document is not fun to read online.  Most documents aren’t.  Think about the 20-page terms and conditions document you are asked to agree to.  

A document is a special kind of content.  It’s a highly ordered large-sized content item.  Documents are linear, with a defined start and finish.  A book, for example, starts with a title page, provides a table of contents, and ends with an appendix and index  Documents are offline artifacts.  They are records that are meant to be enduring and not change. Most online content, however, is impermanent and needs to change frequently. As content online has become increasingly dynamic, the need for maintaining consistent order has lessened as well.  Online content is accessed non-linearly.  

XML promoted a false hope that the same content could be presented equally well both online and offline — specifically, in print.  But publishers have concluded that print and online are fundamentally different.  They can’t be equal priorities.  Either one or the other will end up driving the whole process.  For example, The Wall Street Journal, which has an older subscriber base, has given enormous attention to their print edition, even as other newspapers have de-emphasized or even dropped theirs.  In a review of their operations this past summer, The Journal found that their editorial processes were dominated by print because print is different.  Decisions about content are based on the layout needs of print, such as content length, article and image placement, as well as the differences in delivering a whole edition versus delivering a single article.  Print has been hindering the Journal’s online presence because it’s not possible to deliver the same content to print and screen as equally important experiences.  As result, the Journal is contemplating de-emphasizing print, making it follow online decisions, rather than compete with them.

Some publishers have no choice but to create printable content.  XML will still enjoy a role in industrial-scale desktop publishing.  Pharmaceutical companies, for example, need to print labels and leaflets explaining their drugs.  The customer’s physical point of access to the product is critical to how it is used — potentially more important than any online information.  In these cases, the print content may be more important than the online content, driving the process for how online channels deliver the content.  Not many industries are in this situation and those that are can be at risk of becoming isolated from the mainstream of web developments.  

XML still has a role to play in the management of certain kinds of digital content.  Because XML is older and has a deeper legacy, it has been decidedly more expressive until recently.  Expressiveness relates to the ability to define concepts unambiguously.  People used to fault the JSON format for lacking a schema like XML has, though JSON now offers such a schema.  XML is still more robust in its ability to specify highly complex data structures, though in many cases alternatives exist that are compatible with JSON.   Document-centric sectors such as finance and pharmaceuticals, which have burdensome regulatory reporting requirements, remain heavy users of XML.  Big banks and other financial institutions, which are better known for their hesitancy than their agility, still use XML to exchange financial data with regulators. But the fast-growing FinTech sector is API-centric and is not XML-focused.  The key difference is the audience.  Big regulated firms are focused on the needs of a tightly knit group of stakeholders (suppliers, regulators, etc.) and prioritize the bulk exchange of data with these stakeholders.  Firms in more competitive industries, especially startups, are focused on delivering content to diverse customers, not bulk uploads.  

XML and content agility

The downside of expressiveness is heaviness.  XML has been criticized as verbose and heavy — much like Victorian literature.  Just as Dickensian prose has fallen out of favor with contemporary audiences, verbose markup is becoming less popular.  Anytime people can choose between a simple way or a complex one to do the same thing, they choose the simple one.  Simple, plain, direct. They don’t want elaborate expressiveness all the time, only when they need it.  

When people talk about content as being intelligent (understandable to other machines), they may mean different things.  Does the machine need to be able to understand everything about all the content from another source, or does it only need to have a short conversation with the content?  XML is based on the idea that different machines share a common schema or basis of understanding. It has a rigid formal grammar that must be adhered to. APIs are less worried about each machine understanding everything about the content coming from everywhere else.  It only cares about understanding (accessing and using) the content it is interested in (a query). That allows for more informal communication. By being less insistent on speaking an identical formal language, APIs enable content to be exchanged more easily and used more widely.  As a result, content defined by APIs more ubiquitous: able to move quickly to where it’s needed.  

Ultimately, XML and APIs embrace different philosophies about content.  XML provides a monolithic description of a huge block of content.  It’s concerned with strictly controlling a mass of content and involves a tightly coupled chain of dependencies, all of which must be satisfied for the process to work smoothly.  APIs, in contrast, are about connecting fragments of content.  It’s a decentralized, loosely coupled, bottom-up approach.  (The management of content delivered by APIs is handled by headless content models, but that’s another topic.)

Broadly speaking, APIs treat the parts as more important than the whole.  XML treats the whole as more important than the parts.  

Our growing reliance on the cloud has made it increasingly important to connect content quickly.  That imperative has made content more open.  And openness depends on outsiders being able to understand what the content is and use it quickly.  

As XML has declined in popularity, one of its core ideas has been challenged.  The presumption has been that the more markup in the content, the better.  XML allows for many layerings of markup, which can specify what different parts of text concern.  The belief was that this was good: it made the text “smarter” and easier for machines to parse and understand.  In practice, this vision hasn’t happened.  XML-defined text could be subject to so many parenthetical qualifications that it was like trying to parse some arcane legalese.  Only the author understood what was meant and how to interpret it.  The “smarter” the XML document tried to be, the more illegible it became to people who had to work with the document — other authors or developers who would do something later with the content.    Compared with the straightforward language of key-value pairs and declarative API requests, XML documentation became an advertisement pointing out how difficult its markup is to use.  “The limitations in JSON actually end up being one of its biggest benefits. A common line of thought among developers is that XML comes out on top because it supports modeling more objects. However, JSON’s limitations simplify the code, add predictability and increase readability.”  Too much expressiveness becomes an encumbrance.  

Like any monolithic approach, XML has become burdened by details as it has sought to address all contingencies.  As XML ages, it suffers from technical debt.  The specifications have grown, but don’t necessarily offer more.  XML’s situation today similar to Latin’s situation in the 18th century, when scientists were trying to use it to communicate scientific concepts.  One commenter asserts that XML suffers from worsening usability: “XML is no longer simple. It now consists of a growing collection of complex connected and disconnected specifications. As a result, usability has suffered. This is because it takes longer to develop XML tools. These users are now rooting for something simpler.”  Simpler things are faster, and speed matters mightily in the connected cloud.  What’s relevant depends on providing small details right when they are needed.

At a high level, digital content is bifurcating between API-first approaches and those that don’t rely on APIs.  An API-first approach is the right choice when content is fast-moving.  And nearly all forms of content need to speed up and become more agile.  Content operations are struggling to keep up with diversifying channels and audience segmentation, as well as the challenges of keeping the growing volumes of online content up-to-date.  While APIs aren’t new anymore, their role in leading how content is organized and delivered is still in its early stages.  Very few online publishers are truly API-first in their orientation, though the momentum of this approach is building.

When content isn’t fast-moving, APIs are less important. XML is sometimes the better choice for slow-moving content, especially if the entire corpus is tightly constructed as a single complex entity.  Examples are legal and legislative documents or standards specifications. XML will still be important in defining the slow-moving foundations of certain core web standards or ontologies like OWL — areas that most web publishers will never need to touch.  XML is best suited for content that’s meant to be an unchanging record.  

 Within web content, XML won’t be used as a universal language defining all content, since most online content changes often.  For those of us who don’t have to use XML as our main approach, how is it relevant?  I expect XML will play niche roles on the web.  XML will need to adapt to the fast-paced world of APIs, even if reluctantly.  To be able to function more agilely, it will be used in a selective way to define fragments of content.  

An example of fragmental XML is how Google uses the SSML standard, an XML-defined speech markup standard to indicate speech emphasis and pronunciation.  This standard predates the emergence of consumer voice interfaces, such as “Hey Google!” Because it was in place already, Google has incorporated it within the JSON-defined schema.org semantic metadata they use.  The XML markup, with its angled brackets, is inserted within the quote marks and curly brackets of JSON.   JSON describes the content overall, while XML provides assistance to indicate how to say words aloud. 

SVG, used to define vector graphics, is another example of fragmental XML.  SVG image files are embedded in or linked to HTML files without needing to have the rest of the content be in XML.

More generally, XML will exist on the web as self-contained files or as snippets of code.  We’ll see less use of XML to define the corpus of text as a whole.  The stylistic paradigm of XML, of using in-line markup  — comments within a sentence — is losing its appeal, as it is hard for both humans and machines to read and parse. An irony is that while XML has made its reputation for managing text, it is not especially good at managing individual words.  Swapping words out within a sentence is not something that any traditional programming approach does elegantly, whether XML-based or not, because natural language is more complex than an IT language processor.  What’s been a unique advantage of XML — defining the function of words within a sentence — is starting to be less important.  Deep learning techniques (e.g., GPT-3) can parse wording at an even more granular level than XML markup, without the overhead.  Natural language generation can construct natural sounding text.  Over time, the value of in-line markup for speech, such as used in SSML, will diminish as natural language generation improves its ability to present prosody in speech.  While deep learning can manage micro-level aspects of words and sentences, it is far from being about to manage structural and relational dimensions of content.  Different approaches to content management, whether utilizing XML or APIs connected to headless content models, will still be important.  

As happened with Latin, XML is evolving away from being a universal language.  It is becoming a controlled vocabulary used to define high specialized content objects.  And much like Latin gave as the alphabet upon which many languages are built, XML has contributed many concepts to content management that other languages will draw up for years to come.  XML may be becoming more of a niche, but it’s a niche with an outsized influence.

— Michael Andrews

The post XML, Latin, and the demise or endurance of languages appeared first on Story Needle.

All models reflect a point of view

12 september 2020 - 7:04pm

We sometimes talk about having a “bird’s eye” perspective of a place or about a topic.  We can see how details connect to form a larger whole.  The distance highlights the patterns that may be hard to see up close.  In many ways, a content model is a bird’s eye view of content addressing topics or activities.  It’s a map of our content landscape. 

This year, many of us — having been shut indoors involuntarily and had our travel limited —  have marveled at the freedom of travel that birds enjoy.  Bird watching has become a popular pastime during the pandemic, encouraging people to consult content about birds to understand what they are newly noticing.

Content about birds provides an excellent view into the structure of content — even for people not interested in birds. Nearly 20 years ago JoAnne Hackos discussed what field guides to birds can teach us about structuring content in her book, Content Management for Dynamic Web Delivery.  Information about birds offers a rich topic to explore content structure.

 Last year, I conducted a workshop exploring content structuring decisions by comparing how field guides describe bird species.  I have a collection of field guides about Indian birds from my time living in India.  They all broadly cover the same material, but the precise coverage of each varies.  Some will talk about habitats, others will discuss diet.  They will get into different levels of detail.

Various guides to Indian birds Various guides to Indian birds

As someone interested in birds, I noticed how these field guides differed when discussing the same bird species.  Even the images of a bird varied greatly: whether they are paintings or photos, if they are in context or not, and whether distinguishing features are described in the text or as annotations.  Image choices have profound implications for what information is conveyed.

All this variation reveals something that’s obvious when you see it: there’s no one way to describe a bird.  People make editorial choices when they structure content.  That’s a very different way of thinking about content structure than the notion of domain modeling, which assumes that things we describe have intrinsic characteristics that can be modeled objectively.  Domain modeling presumes there’s a platonic ideal that we can discover to describe things in our world. It can provide some conceptual scaffolding for identifying the relationships between concrete facts associated with a topic. Domain modeling is best approached from the viewpoint of different personas.  Why do they care about any of these facts?  And importantly, might they care about different facts?

Without doubt, it’s valuable to get outside of our subjective ways of seeing to consider other viewpoints.  But that doesn’t imply there’s a single viewpoint that is definitive or optimal.  Content modeling is not the same thing as data modeling, just as content isn’t simply data.  The limitation of domain modeling is that it doesn’t provide any means to distinguish more important facts from less important ones.  And it treats content as data, where facts are readily reduced to a few concrete objective statements rather than involving descriptive interpretations or analysis.  With a domain model, it’s not obvious why people care about any of the data.  Before we can build experiences from content, we need the basic content to be interesting and relevant. Relevance and interest have been overlooked in many discussions about content modeling.   

While I see richness in small editorial decisions among different field guides, a person not interested in birding may find these distinctions as unimportant.  To a casual viewer, all field guides look similar.  There’s a generic template that field guides seem to follow to describe birds.  Here, the editorial decisions have emerged over time as a common framework that’s widely accepted and expected.  Some people will view the task of structuring content as one of finding an existing framework that’s known, and copying it.  Instead of searching for the intrinsic properties of things that can be described, the search is finding patterns already used in the content.  Adopting vernacular structural patterns has much to commend it.  These patterns are familiar, and in many cases work fine.  But relying on habit can also cause us to miss opportunities to enrich how to describe things.  They may satisfy a basic need, but they don’t necessarily do so optimally.  

The most disruptive development to influence field guides is the smartphone app.  These apps shake up how they approach birds and can incorporate image and audio recognition to aid in identifying a species.   They can be marvelously clever in what they can do, but they too represent editorial decisions: a focus on a transactional task rather than a deeper look into context and comparison.  The more that content exists to support a repetitive transactional task, the more tightly prescribed the content will be in a model.  If you consider content as existing only to support narrow transactional needs, the structure of the model might seem obvious, because what else would someone care about?  Such a model presumes zero motivation on the part of the reader: they only want to view content that is necessary for completing a task.  

Field guides exist to aid the identification of birds.  Deciding what information is important — or is readily available — to aid the identification of birds still involves an editorial judgment.  

Topics don’t have intrinsic structures.  The structure depends in part on the tasks associated with the topic.  And the more that a topic requires the motivation of the reader — their interest, preferences, and choices outside of a narrow task supported within the UI — the more important the editorial dimensions of the content model become.    The model needs to express elements that tell readers why they would care about the topic: what they should learn or see as important and relevant.

Importantly, the discussion of a topic is not limited to a specific genre.  Species of birds can be discussed in ways other than field guides.  Within a genre,  you can change the structure associated with it.  But you can switch genres as well, and embrace a different structure entirely.

When we consider structures within genres, we may be inclined to confuse its form with its intent.  The importance of genre is that it provides a point of view to elucidate a topic.

Picture books provide an alternative genre to present content about birds.  They involve the tight juxtaposition of words and images, often to provide a richer narrative about a topic.  Beyond that, the structure involved is not standardized or well defined.  The genre is most often associated with children’s books and exemplified by outstanding writer-illustrators such as Maurice Sendak, Dr. Seuss, and Quintin Blake.

But picture books are not limited to children’s entertainment.  They can potentially be used for any sort of topic and any audience.

This spring a new picture book, What it’s like to be a Bird, was released that became an instant bestseller.  It was produced by David Sibley, a renowned ornithologist and illustrator.  “My original idea, in the early 2000s, was to produce a bird guide for kids. Then I started thinking about it as a bird guide for beginners of any age.  But having created a comprehensive North American bird guide, the concept of a ‘simplified’ guide never clicked for me. Instead, I wanted to make a broader introduction to birds.”

His goal is to “give readers some sense of what it’s like to be a bird…My growing sense as I worked on this book is that instinct must motivate a bird by feelings — of satisfaction, anxiety, pride, etc…how else do we explain the complex decision that birds make everyday…”  Sibley wants to capture the bird’s experience making decisions on its life journey.  A wonderful backdrop as we think about the reader’s experience on the journey through his book.

“Each essay focuses on one particular detail…they are meant to be read individually, not necessarily in sequence — everything is interconnected, and there are frequent cross-references suggesting which essay to read next.”    We can see how Sibley has planned a content model for his material.

Bolstered by his talents as a subject expert, writer, and illustrator, he’s been able to rethink how to present content about birds.  While he’s also written a conventional field guide to birds, his new book explores species through their behavior.  His is not the only recent book looking at bird behavior, but the perspective he offers is unique.  Rather than organizing content around behavior themes such as mating, he focuses the content on species of birds and then talks about two or three key behaviors they have that are interesting.  The birds act as protagonists in stories about their life situation.  

Sibley’s book profiles various species of birds — something field guides do as well. But Sibley’s profiles explain the bird from its own point of view, instead of from an external viewpoint of concrete properties such as feather markings or song calls.  The stories of these species incapsulate actions that happen over a period involving a motivation and outcome.  They aren’t data.  

He introduces species of birds by presenting two or three stories about each.  Each story is a short essay explaining an illustration of an activity the bird is engaged in.  Frequently, the illustration is puzzling, prompting the reader to want to understand it.  In some cases, he breaks the story into several small paragraphs, each with its own illustration, when he wants to describe a sequence of events over time.

A simplified content model will show how Sibley explains birds.  The diagram reveals a highly connected structure.  It doesn’t look like the hierarchical structure of a book. There’s no table of contents or index.  Though the content is manifested as a book, it could be delivered to alternative platforms and channels.  

A simplified content model for Sibley's What it's like to be a birdA simplified content model for Sibley’s What it’s like to be a bird

On the far left of the diagram, we see themes about birdlife that are explored.  These themes may be broken into sub-themes.  For example, the theme of survival has two sub-themes, which has even more specific sub-themes:

  • Survival
    • Birds and weather
      • Keeping cool
      • Keeping warm
    • Avoiding predators
      • Be inconspicuous
      • Be alert
      • Create a distraction

Each theme or sub-theme presents a range of related factual statements.  For example, he presents a series of facts about how birds create distractions.  These facts represent some important highlights about a theme: an index of knowledge that offers a range of perspectives.  Each fact points to a profile of a bird specifies, where a story essay will provide context about the statement and make it more understandable.

In this example, we see how the theme of how birds use smell is revealed through a series of facts that point to essays about how different species use of smell.

 Sibley's What it's like to be a Bird)Thematically grouped factual highlights about birdlife (source: David Sibley’s What it’s like to be a Bird)

When we visit a profile of a bird species, we encounter several stories, which are a combination of picture and essay.  The illustration shows  a starling holding a cigarette in its beak.  The situation has the makings of a story — we want to know more.  The essay tells us.

 Sibley's What it's like to be a Bird) Illustration and essay providing a story relating to a behavior of a bird species (source: David Sibley’s What it’s like to be a Bird)

Even if the story is enjoyable, a part of us may wonder if it’s just an entertaining yarn. Picture books are most often associated with fantasy, after all.  And unlike a nature program on TV, we don’t see a museum expert in a talking head interview to make it seem more credible.  Instead, we get a list of recent scientific references relating to the issue.  All these references are arranged thematically, like our facts.  They provide an overview of the focus on recent scientific research about birds, giving us a sense of how much scientists are still learning about these ubiquitous creatures that are as old as dinosaurs. 

 Sibley's What it's like to be a Bird)Source references of recent discoveries from scientific research relating to birds, arranged thematically (source: David Sibley’s What it’s like to be a Bird)

The structuring of the content helps us to understand and explore.  The stories make each bird more real: something living we can become interested in.  We can understand their dilemmas and how they seek to solve them.  We are up-close.  But we can also step back and understand the broader behaviors of birds that influence their lives.  

While each species is described through stories, we are not limited to those.  How do other birds work with smell?  What have we learned recently about smell and birds?  These other pathways allow us to follow our interests and find different connections.  

Sibley’s model shows how to transcend the top-down hierarchies that force how to learn about a topic and the bottom-up collections of random facts that leave us with no structure to guide us.  Sibley’s model of content can be approached in different ways, but it is always deliberate.  There’s no feeling of being lost in hyperlinks.

Content models should reflect what readers want to get and how they might want to get it.  They are more than a technical specification.  They are an essential tool in editorial planning.  Developing a content model can be a creative act.

— Michael Andrews

The post All models reflect a point of view appeared first on Story Needle.

Who benefits from schema.org?

23 augustus 2020 - 3:39am

Schema.org shapes the behavior of thousands of companies and millions of web users.  Even though few users of the internet have heard of it, the metadata standard exerts a huge influence in how people get information, and from whom they get it.  Yet the question of who precisely benefits from the standard gets little attention.  Why does the standard exist and does everyone materially benefit equally from it?  While metadata standard may sound like a technical mechanism generating impartial outcomes, metadata is not always fair in its implementation.  

Google has a strong vested interest in the fate of schema.org — but it is not the only party affected.  Other parties need to feel there are incentives to support schema.org.  Should they feel they experience disincentives, that sentiment could erode support for the standard.

As schema.org has grown in influence over the past decade, that growth has been built upon a contradictory dynamic.  Google needs to keep two constituencies satisfied to grow the usage of Google products.  It needs publishers to use schema.org so it has content for its products.  Consumers need that content to be enticing enough to keep using Google products.  But to monetize its products, Google needs to control how schema.org content is acquired from publishers and presented to customers.  Both these behaviors by Google act as disincentives to schema.org usage by publishers and consumers.  

To a large extent, Google has managed this contradiction by making it difficult for various stakeholders to how influences their behavior. Google uses different terminology, different rationales, and even different personas to manage the expectations of stakeholders. How information about  schema.org is communicated does not necessarily match the reality of it in practice.  

 Although schema.org still has a low public profile, more stakeholders are starting to ask questions about it. Should they use schema.org structured data at all?  Is how Google uses this structured data unfair?  

To assess the fairness of schema.org involves looking at several inter-related issues: 

  • What schema.org is
  • Who is affected by it 
  • How it benefits or disadvantages different stakeholders in various ways.  
What kind of entity is schema.org?

Before we can assess the value of schema.org to different parties, we need to answer a basic question is: What is it, really? If everyone can agree on what it is we are referring to, it should be easier to see how it benefits various stakeholders.  What seems like a simple question defies a clear simple answer.  Yet there are multiple definitions of schema.org out there, supplied by schema.org, Google, and the W3C. The schema.org website refers to it as a “collaborative, community activity,” which doesn’t offer much precision. The most candid answer is that schema.org is a chameleon. It changes its color depending on its context.

Those familiar with the schema.org vocabulary might expect that schema.org’s structured data would provide us with an unambiguous answer to that question. A core principle of the schema.org vocabulary is to indicate an entity type.  The structured data would reveal the type of entity we are referring to. Paradoxically, the schema.org website doesn’t use structured data.  While it talks about schema.org, it never reveals through metadata what type of entity it is.

We are forced to ascertain the meaning of schema.org by reading its texts.  When looking at the various ways it’s discussed, we can see schema.org has embodying four distinct identities.  It can be a:

  1. Brand
  2. Website
  3. Organization
  4. Standard
Who is affected by schema.org?

A second basic question: Who are the stakeholders affected by schema.org?  This includes not only who schema.org is supposed to be for, but also who gets overlooked or disempowered.  We can break these stakeholders into segments:

  • Google (the biggest contributor to schema.org and its biggest user of data utilizing the metadata)
  • Google’s search engine competitors who are partners (“sponsors”) in the schema initiative (Microsoft, Yahoo/Verizon, and Russia’s Yandex)
  • Firms that develop IT products or services other than search engines (consumer apps, data management tools) that could be competitive with search engines
  • Publishers of web content, which includes
    •  Commercial publishers who rely on search engines for revenues and in some cases may be competitors to Google products
    •  Non-Commercial publishers (universities, non-profits, religious organizations, etc
  • Consumers and the wider public that encounter and rely on schema.org-described data
  • Professional service providers that advise others on using schema.org such as SEO consultants and marketing agencies
  • The W3C, which has lent its reputation and accommodation to schema.org

By looking at the different dimensions of schema.org and the different stakeholders, we can consider how each interacts.  

Schema.org is very much a Google project — over which it exerts considerable control.  But it cannot appear to be doing so, and therefore relies on various ways of distancing itself from appearing to mandate decisions.  

Schema.org as a brand

Schema.org may not seem like an obvious brand. There’s no logo.  The name, while reassuringly authoritative-sounding, does not appear to be trademarked.  Even how it is spelled is poorly managed.  It is unclear if it is meant to be lower case or uppercase — both are used in the documentation, in some cases within the same paragraph (I mostly use lowercase, following examples from the earliest discussions of the standard.)

Brands are about the identity of people, products, or organizations.  A brand foremost is a story that generates an impression of the desirability and trustworthiness of a tangible thing.  The value of a brand is to attract favorable awareness and interest.   

Like many brands, schema.org has a myth about its founding, involving a journey of its heroes.

 Schema.org’s mythic story involves three acts: life before schema.org, the creation of schema.org, and the world after schema.org.

Life before schema.org is presented as chaotic.  Multiple semantic web standards were competing with one another.  Different search engines prioritized different standards.  Publishers such as Best Buy made choices on their own about which standard to adopt.  But many publishers were waiting on the sidelines, wondering what the benefits would be for them.

The schema.org founders present this period as confusing and grim.  But an alternate interpretation is that this early period of commercializing semantic web metadata was full of fresh ideas and possibilities.  Publishers rationally asked how they would benefit eventually, but there’s little to suggest they feared they would never benefit.  In short, the search engines were the ones complaining about having to deal with competition and diversity in standards. Complaints by publishers were few.

With the formation of schema.org, the search engines announced a new standard to end the existence of other general-coverage semantic web metadata standards.  This standard would vanquish the others and end the confusion about which one to follow. Schema.org subsumed or weeded out competing standards. With the major search engines no longer competing with one another and agreeing to a common standard, publishers would be clear about expectations relating to what they were supposed to do.  The consolidation of semantic web standards into one is presented as inevitable.  This outcome is rationalized with the “TINA” justification: there is no alternative.  And there was no alternative for publishers, once the search engines collectively seized control of the semantic metadata standards process.

After schema.org consolidated the semantic web metadata universe, everyone has benefits, in this narrative.  The use of semantic metadata has expanded dramatically.  The coverage of schema.org has become more detailed over time. These metrics demonstrate its success.  Moreover, the project has become a movement where many people can now participate.   Schema.org positions itself as a force of enlightenment rising about the petty partisan squabbles that bedeviled other vocabularies in the past.   A semi-official history of schema.org states: “It would also be unrecognizable without the contributions made by members of the wider community who have come together via W3C.” 

 The schema.org brand story never questions other possibilities.  It assumes that competition was bad, rather than seeing it as representing a diversity of viewpoints that might have shaped things differently.  It assumes that the semantic web would never have managed to become commercialized, instead of recognizing the alternative commercial possibilities that might have emerged from the activity and interest by other parties.  

Any retrospective judgment that the commercialization semantic web would have failed to happen without schema.org consolidating things under the direction of search engines is speculative history.  It’s possible that multiple vocabularies could have existed side-by-side and could have been understood.  Humans speak many languages.  There’s no inherent reason why machines can’t as well.   Language diversity fosters expressive diversity.  

Schema.org as a website

Schema.org is a rare entity whose name is also a web address. 

If you want to visit schema.org, you head to the website.  There’s no schema.org convention or schema.org headquarters people can visit. If it isn’t clear who runs schema.org or how it works, at least the website provides palpable evidence that schema.org exists.   Even if it’s just a URL, it provides an address and promises answers.

At times, schema.org emphasizes that is just a website — and no more than that: “Schema.org is not a formal standards body. Schema.org is simply a site where we document the schemas that several major search engines will support.”

In its domain level naming, schema.org is a “dot-org,” which Wikipedia notes is “the domain is commonly used by schools, open-source projects, and communities, but also by some for-profit entities.”   Schema.org shares a TLD with such good samaritan organizations such as the Red Cross and the World Wildlife Foundation.  On first impression, schema.org appears to be a nonprofit charity of some sort.  

While the majority of schema.org’s documentation appears on its website, it sometimes has used the “WebSchemas” wiki on the W3C’s domain: https://www.w3.org/wiki/WebSchemas . The W3C is well regarded for its work as a nonprofit organization.  The not-for-profit image of the W3C’s hosting lends a halo of trust to the project.  

In reality, the website is owned by Google.  All the content on the schema.org website is subject to the approval of Google employees involved with the schema.org project.  Google also provides the internal search engine for the site, the Google Programmable Search Engine.

“Who is” for schema.org Schema.org as an organization

Despite schema.org’s disavowal of being a standards body, it does in fact create standards and needs an organizational structure to allow that to happen.  

Schema.org’s formal organization involves two tiers:

  1. A steering group of the four sponsoring companies 
  2. A W3C community group

Once again, the appearances and realities of these arrangements can be quite different.

The steering group

While the W3C community group gets the most attention, one needs to understand the steering group first.  The steering group predates the community group and oversees it.  “The day to day operations of Schema.org, including decisions regarding the schema, are handled by a steering group” notes a FAQ.  The ultimate decision-making authority for schema.org rests with this steering group.

The steering group was formed at the start of the schema.org initiative.  According to steering group members writing in the ACM professional journal, “in the first year, these sponsor companies made most decisions behind closed doors. It incrementally opened up…”  

There are conflicting accounts about who can participate in the steering group.  The 2015 ACM article talks about “a steering committee [sic] that includes members from the sponsor companies, academia, and the W3C.”   The schema.org website focuses on search engines as the stakeholders who steer the initiative: “Schema.org is a collaboration between Google, Microsoft, Yahoo! and Yandex – large search engines who will use this marked-up data from web pages. Other sites – not necessarily search engines – might later join.”  A schema.org FAQ asks: “Can other websites join schema.org as partners and help decide what new schemas to support?” and the answer points to the steering committee governing this.  “The regular Steering Group participants from the search engines” oversee the project.  There have been at least two invited outside experts who have participated as non-regular participants, but the current involvement by outside participants in the steering group is not clear.

Schema.org projects the impression that it is a partnership of equals in the search field, but the reality belies that image. Even though the four search engines describe the steering group as a “collaboration,” the participation by sponsors seems unbalanced. With a 90% market share, Google’s dominance of search is overwhelming, and they have a far bigger interest in the outcomes than the other sponsors.  Since schema.org was formed nearly a decade ago, Microsoft has shifted its focus away from consumer products: dropping smartphones and discontinuing its Cortana voice search — both products that would have used schema.org.  Yahoo has ceased being an independent company and has been absorbed by Verizon, which is not focused on search.  Without having access to the original legal agreement between the sponsors, it’s unclear why either of these companies continues to be involved in schema.org from a business perspective.

The steering group is chaired by a Google employee: Google Fellow R.V. Guha. “R.V. Guha of Google initiated schema.org and is one of its co-founders. He currently heads the steering group,” notes the schema.org website.  Guha’s Wikipedia entry also describes him as being the creator of schema.org. 

Concrete information on the steering group is sparse.  There’s no information published about who is eligible to join, how schema.org is funded, and what criteria it uses to make decisions about what’s included in the vocabulary.  

What is clear is that the regular steering group participation is limited to established search engines, and that Google has been chair.  Newer search engines such as DuckDuckGo aren’t members.  No publishers are members.  Other firms exploring information retrieval technologies such as knowledge graphs aren’t members either.  

The community group

In contrast to the sparse information about the steering group, there’s much more discussion about the W3C community group, which is described as “the main forum for the project.”  

The community group, unlike the steering group, has open membership.  It operates under the umbrella of the W3C, “the main international standards organization for the World Wide Web,” in the words of Wikipedia.  Google Vice President and Chief Internet Evangelist, Vint Cerf, referred to as a “father” of the internet, brokered the ties between schema.org and the W3C.  “Vint Cerf helped establish the relations between Schema.org and the W3C.”  If schema.org does not wish to be viewed as a standard, they choose an odd partner by selecting the W3C.  

The W3C’s expectations for community groups are inspiring: ”Community Groups enable anyone to socialize their ideas for the Web at the W3C for possible future standardization. “  In the W3C’s vision, anyone can influence standards.  

W3C’s explanation of community groups

The sponsors also promote the notion that the community group is open, saying the group “make[s] it easy for publishers/developers to participate.” (ACM)  

The vague word “participation” appears multiple times in schema.org literature:  “In addition to people from the founding companies (Google, Microsoft, Yahoo and Yandex), there is substantial participation by the larger Web community.” The suggestion implied is that everyone is a participant with equal ability to contribute and decide.  

While communities are open to all to join, that doesn’t mean that everyone is equal in decision making in the schema.org’s case — notwithstanding the W3C’s vision.  Everyone can participate, but not everyone can make decisions.

Publishers are inherently disadvantaged in the community process.  Their suggestions are less important than those of search engines, who are the primary consumer of schema.org structured data.  “As always we place high emphasis on vocabulary that is likely to be consumed, rather than merely published.”

Schema.org as a standard

Schema.org does not refer to itself as a standard, even though in practice it is one.  Instead, schema.org relies on more developer-focused terminology: vocabulary, markup, and data models.  It presents itself as a helpful tool for developers rather than as a set of rules they need to follow.  

Schema.org aims to be monolithic, where no other metadata standard is needed or used. The totalistic name chosen — schema.org — suggests that no other schema is required.  “For adoption, we need a simpler model, where publishers can be sure that a piece of vocabulary is indeed part of Schema.org.”

The search engine sponsors discourage publishers from incorporating other semantic vocabularies together with schema.org.  This means that only certain kinds of entities can be described and only certain details.  So while schema aims to be monolithic, it can’t describe many of the kinds of details that are discussed in Wikipedia.  The overwhelming focus is on products and services that promote the usage of search engines.   The tight hold prevents other standards from emerging that are outside of the influence of schema.org’s direction.  

Schema.org’s operating model is to absorb any competing standard that gains popularity.  “We strongly encourage schema developers to develop and evangelize their schemas. As these gain traction, we will incorporate them into schema.org.”  In doing this, schema.org groups avoid the burdens of developing on their own coverage of large domains involving fine details and requiring domain-specific expertise.  Schema.org gets public recognition for offering coverage of these if it decides it would benefit schema.org’s sponsors.   Schema.org has absorbed domain-specific vocabularies relating to autos, finance, and health, which allows search engines to present detailed information relating to these fields.  

How Google shapes schema.org adoption

Google exerts enormous power over web publishing.  Many webmasters and SEO specialists devote the majority of their time satisfying the requirements that Google imposes on publishers and other businesses that need an online presence.  

Google shapes the behavior of web publishers and other parties through a combination of carrots and sticks.

Carrots: Google the persuader

Because Google depends on structured data to attract users who will see its ads, it needs to encourage publishers to adopt schema.org.  The task is twofold: 

  1. Encourage more adoption, especially by publishers that may not have had much reason to use schema.org 
  2. Maintain the use of schema.org by existing publishers and keep up interest

Notably, how schema.org describes its benefits to publishers is different from how Google does.  

According to schema.org, the goal is to “make it easier for webmasters to provide us with data so that we may better direct users to their sites.”  The official rationale for schema.org is to help search engines “direct” users to “sites” that aren’t owned by the search engine.   

“When it is easier for webmasters to add markup, and search engines see more of the markup they need, users will end up with better search results and a better experience on the web.”   The official schema.org rationale, then, is that users benefit because they get better results from their search.  Because webmasters are seeking to direct people to come to their site, they will expect that the search results will direct users there.

“Search engines are using on-page markup in a variety of ways. These projects help you to surface your content more clearly or more prominently in search results.”   Again, the implied benefit of using schema.org is about search results — links people can click on to take them to other websites.  

Finally, schema.org dangles a vaguer promise that parties other than search engines may use the data for the benefit of publishers: “since the markup is publicly accessible from your web pages, other organizations may find interesting new ways to make use of it as well.”  The ability of organizations other than search engines to use scheam.org metadata is indeed a genuine possibility, though it’s one that hasn’t happened to any great extent.  

When Google talks about the benefits, they are far more obtuse.  The first appeal is to understanding: make sure Google understands your content.  “If you add schema.org markup to your HTML pages, many companies and products—including Google search—will understand the data on your site. Likewise, if you add schema.org markup to your HTML-formatted email, other email products in addition to GMail might understand the data.”     Google is one of “many” firms in the mix, rather than the dominant one.  Precisely what the payoff is from understanding is not explicitly stated.

The most tangible incentive that Google dangles to publishers to use schema.org is cosmetic: they gain a prettier display in search.  “Once Google understands your page data more clearly, it can be presented more attractively and in new ways in Google Search.”

Google refers to having a more desirable display as “rich snippets,” among other terms.  It has been promoting this benefit from the start of schema.org: “The first application to use this markup was Google’s Rich Snippets, which switched over to Schema.org vocabulary in 2011.” 

The question is how enticing this carrot is.  

Sticks: Google the enforcer

Google encourages schema.org adoption through more coercive measures as well.  It does so in three ways:

  1. Setting rules that must be followed
  2. Limiting pushback through vague guidelines and uncertainty
  3. Invoking penalties

Even though the schema.org standard is supposedly voluntary and not “normative” about what must be adopted, Google’s implementation is much more directive.  

Google sets rules about how publishers must use schema.org.  Broadly, Google lays down three ultimatums:

  1. Publishers must use schema.org to appear favorably in search
  2. They can only use schema.org and no other standards that would be competitive with it
  3. They must supply data to Google in certain ways  

An example of a Google ultimatum relates to its insistence that only the schema.org vocabulary be used — and no others.  Google has even banned the use of another vocabulary that it once backed: the data vocabulary.  Google prefers to consolidate all web metadata descriptions into the schema.org, which it actively controls.  “With the increasing usage and popularity of schema.org we decided to focus our development on a single SD [structured data] scheme. “  Publishers who continue to use the non-unfavored vocabulary face a series of threats.  Google here is making a unilateral decision about what kinds of metadata are acceptable to it.  

Google imposes a range of threats and penalties for non-compliance.  These tactics are not necessarily specific to schema.org structured data.   Google has used such tactics to promote the use of its “AMP” standard for mobile content.  But these tactics are more significant in the context of the schema.org vocabulary, which is supposed to be a voluntary and public standard.  

Google is opaque how schema.org could influence rankings.  If used incorrectly might your ranking be hurt or even disappear? 

Example of anxiety about search rankings and schema.org usage

Google never suggests that schema.org can positively influence search rankings.  But it leaves open the possibility that not using it could negatively influence rankings.  

Google’s threats and penalties relating to schema.org usage can be categorized into four tactics:

1. Warnings — messages in yellow that the metadata aren’t what Google expects

2. Errors — messages in red that Google won’t accept the structured data

3. Being ignored — a threat that the content won’t be prioritized by Google

4. Manual actions — a stern warning that the publisher will be sanctioned by Google

Manual actions are the death sentence that Google applies.   Publishers can appeal to Google to change its decision.  But ultimately Google decides what it wants and without a reversal of Google’s prior decision, the publisher is ostracized from Google and won’t be found by anyone searching for them.  The publisher becomes persona non grata. 

An example of a “manual action” sanction is if a publisher posts a job vacancy but there’s “no way to apply for the job” via the schema.org mechanism.  That’s doesn’t imply there’s no job —  it simply means that the poster of the job decided not to agree to Google’s terms: that they had to allow Google to let people apply from Google’s product, without the benefit of additional information that Google doesn’t allow to be included.  

While publishers may not like how they are treated, Google makes sure they have no grounds to protest. Google manages publisher expectations around fairness by providing vague guidance and introducing uncertainty.

 A typical Google statement: “Providing structured data to enable an enhanced search feature is no guarantee that the page will appear with that designated feature; structured data only enables a feature to be displayed. Google tries to display the most appropriate and engaging results to a user, and it might be the case that the feature you code for is not appropriate for a particular user at a particular time.” Google is the sole arbiter of fairness.  

In summary, Google imposes rules on publishers while making sure those rules present no obligations on Google.  Its market power allows them to do this.

How Google uses schema.org to consolidate market dominance

While there’s been much discussion by regulators, journalists, economists, and others about Google’s dominance in search, little of this discussion has focused on the role of schema.org in enabling this dominance.  But as we have seen, Google has rigged the standards-making process and pressed publishers to use schema.org under questionable pretenses.  It has been able to leverage the use of structured data based on the schema.org metadata standard to solidify its market position.

Google has used schema.org for web scraping and to build vertical products.  In both these cases, they are taking away opportunities from publishers who rely on Google and who in many cases are customers of Google’s ad business.  

Web scraping 2.0 and content acquisition

Google uses schema.org to harvest content from publisher websites.  It promotes the benefits to publishers of having this content featured as a “rich snippet” or other kinds of “direct answer” — even if there’s no link to the website of the publisher contributing the information.  

For many years governments and publishers around the world have accused Google of scrapping content without consent.  This legally fraught area has landed Google in political trouble.  A far more desirable option for Google would be to scrape web content with implied content.  Schema.org metadata is a perfect vector for Google to acquire vast quantities of content easily. The publishers do the work for providing the data in a format that Google can easily find and process.  And they do this voluntarily — in the belief that it will help them attract more customers to their websites.  In many ways, this is a big con.  There’s growing evidence that publishers are disadvantaged when their content appears as a direct answer.  And it’s troubling to think they are doing much work for Google to take advantage of them.  In the absence of negotiating power toward Google, they consent to do things that may harm them.

In official financial disclosures filed with the US government, Google has acknowledged that wants to avoid presenting links to other firms’ websites in its search results.  Instead, it uses schema.org data to provide “direct answers” (such as Rich Snippets) so that people stay on Google products.  “Instead of just showing ten blue links in our search results, we are increasingly able to provide direct answers — even if you’re speaking your question using Voice Search — which makes it quicker, easier and more natural to find what you’re looking for” Google noted in SEC filing last year

The Markup notes how publishers may be penalized when their content appears as a “rich snippet” if the user is looking for external links: “Google further muddied the waters recently when it decided that if a non-Google website link appears in a scraped “featured snippet” module, it would remove that site from traditional results below (if it appeared there) to avoid duplication.” 

One study by noted SEO expert Rand Fiskin found that half of all Google searches result in zero clicks.

Verticals

Google has developed extensive coverage of details relating to specific “verticals” or industry sectors.  These sectors typically have data-rich transactional websites that customers use to buy services, find things, or manage their accounts.  Google has been interested in limiting consumer  use of these websites and keeping consumers on Google products.  For example, instead of encouraging people who are searching for a job to visit a job listing website, Google would prefer them to explore job opportunities while staying on Google’s search page— whether or not all the relevant jobs are available without visiting another website.  Google is directly competing with travel booking websites, job listing websites, hotel websites, airline websites, and so on.  Nearly all these sites need to pay Google for ads to appear near the top of the search page, even though Google is working to undermine the ability of users to find and access links to these sites.

According to internal Google emails submitted to the US Congress, Google decided to compete directly with firms in specific industry sectors.  To quote these Google emails:

What is the real threat if we don’t execute on verticals?”

a) “Loss of traffic from google.com … .

b) Related revenue loss to high spend verticals like travel.

c) Missing oppty if someone else creates the platform to build verticals.

d) If one of our big competitors builds a constellation of high quality verticals, we are hurt badly.”

Examples of verticals where Google has encouraged schema.org to build out detailed metadata include: 

  • Jobs and employers 
  • Books 
  • Movies 
  • Courses 
  • Events 
  • Products 
  • Voice-enabled content 
  • Subscription content 
  • Appointments at local businesses (restaurants, spas, health clubs)

Google employees are recognized as contributing the key work in the schema.org vocabulary in two areas pertaining to verticals:

  • “Major topics”  — coverage of verticals mentioned previously
  • The “actions vocabulary” which enables the bypassing of vertical websites to do tasks.

The actions vocabulary is an overlooked dimension of Google’s vertical strategy.  Actions let you complete transactions from within Google products without needing to access the websites of the service provider. (Actions in the schema.org vocabulary are not related to Google’s Manual Actions sanctions discussed earlier.) The schema.org website notes: “Sam Goto of Google did much of the work being schema.org’s Actions vocabulary.”  This Google employee, who was responsible for the problem-space of actions, explained on his own websites the barriers to consumers (and Google) to complete actions across websites:

  • “the opentable/urbanspoon/grubhub can’t reserve *any* arbitrary restaurant, they represent specific ones”
  • “Netflix/Amazon/Itunes can’t stream *any* arbitrary movie, there is a specific set of movies available”
  • “Taxis have their own coverage/service area”
  • “AA.com can’t check-in into UA flights”
  • “UPS can’t track USPS packages, etc.”

These transactions target access to the most economically significant service industries.  While consumers will like a unified way to choose things, they still expect options about the platform they can use to make their choices. In principle, the actions could have opened up competition.  But given Google’s domination of search and smartphone platforms, the benefits of actions have filtered up to Google, not down to consumers.  They may not see all the relevant options they need.    Google has decided what choices are available to the user.

There’s a difference between having the option of using a unified access point, and having only one access point.  When a unified access point becomes the only access point, it becomes a monopoly.

Schema.org and the illusion of choice

Google relies on tactics of “self-favoring” — a concept that current antitrust rules don’t regulate effectively.  But a recent analysis noted the need to address the problem: “Google’s business model rests on recoupment against monetization of data blended with the selling of adjunct services. Deprived of access to this, Google might not have offered these services, nor maintained them, or even developed them in the first place.”

Google cares about schema.org because it is profitably to do so — both in the immediate term and in the long term.  Google makes money by directing customer focus to its ad-supported products, and it consolidates its stranglehold on the online market as potential competitors get squeezed by its actions.

But if Google couldn’t exploit its monopoly market position to make unfair profits from schema.org, would it care about schema.org?  That’s the question that needs to be tested.  If it did, it would be willing to fund schema.org with third party oversight, and allow its competitors and others economically impacted by schema.org to have a voice it schema.org decision-making.  It would allow governments to review how Google uses the data it harvests from schema.org to present choices to customers.  

— Michael Andrews

The post Who benefits from schema.org? appeared first on Story Needle.

Time to end Google’s domination of schema.org

9 augustus 2020 - 3:52am

Few companies enjoy being the object of public scrutiny.  But Google, one of the world’s most recognized brands, seems especially averse.  Last year, when Sundar Pichai, Google’s chief executive, was asked to testify before Congress about antitrust concerns, he refused.  They held the hearing without his presence.  His name card was there, in front of an empty chair.

Last month Congress held another hearing on antitrust.  This time, Pichai was in his chair in front of the cameras, remotely if reluctantly.   During the hearings, Google’s fairness was a focal issue.  According to a summary of the testimony on the ProMarket blog of the University of Chicago Business School’s Stigler Center: “If the content provider complained about its treatment [by Google], it could be disappeared from search. Pichai didn’t deny the allegation.”

One of the major ways that content providers gain (or lose) visibility on Google — the first option that most people choose to find information — is through their use of a metadata standard known as schema.org.  And the hearing revealed that publishers are alleging that Google engages in bullying tactics relating to how their information is presented on the Google platform.  How might these issues be related?  Who sits in the chair that decides the stakes?

Metadata and antitrust may seem arcane and wonky topics, especially when looked at together. Each requires some basic knowledge to understand, so it is rare that the interaction between the two is discussed. Yet it’s never been more important to remove the obscurity surrounding how the most widely used standard for web metadata, schema.org influences the fortunes of Google, one of the most valuable companies in the world. 

Why schema.org is important to Google’s dominant market position

Google controls 90% of searches in the US, and its Android operating system powers 9 of 10 smartphones globally.  Both these products depend on schema.org metadata (or “structured data”) to induce consumers to use Google products.  

A recent investigation by The Markup noted that “Google devoted 41 percent of the first page of search results on mobile devices to its own properties and what it calls ‘direct answers.’”  Many of these direct answers are populated by schema.org metadata that publishers provide in hopes of driving traffic to their websites.  But Google has a financial incentive to stop traffic from leaving its websites.  The Markup notes that “Google makes five times as much revenue through advertising on its own properties as it does selling ad space on third-party websites.”  Tens of billions of dollars of Google revenues depend in some way on the schema.org metadata.  In addition to web search results, many Google smartphone apps including Gmail capture schema.org metadata that can support other ad-related revenues.  

During the recent antitrust hearings, Congressman Cicilline told Sundar Pichai that “Google evolved from a turnstile to the rest of the web to a walled garden.”  The walled garden problem is at the core of Google’s monopoly position.   Perversely, Google has been able to manipulate the use of public standards to create a walled garden for its products.  Google reaps a disproportionate benefit from the standard by preventing broader uses of the standards that could result in competitive threats to Google.

There’s a deep irony is that an open W3C metadata standard, schema.org, has been captured by a tech giant not just to promote its unique interests but to limit the interests of others.  Schema.org was supposed to popularize the semantic web and help citizens gain unprecedented access to the world’s information. Yet Google has managed to monopolize this public asset. 

How schema.org became a walled garden

How Google came to dominate a W3C standard requires a little history.  The short history is that Google has always been schema.org’s chief patron.  It created schema.org and promoted making it a W3C standard.  Since then, it has consolidated its hold on it. 

The semantic web — the inspiration for schema.org — has deep roots in the W3C.  Tim Berners Lee, the founder of the World Wide Web, coined the concept and has been its major champion.  The commercialization of the approach has been long in the making. Metaweb was the first venture-funded company to commercialize the semantic web with its product Freebase  The New York Times noted at the time: “In its ambitions, Freebase has some similarities to Google — which has asserted that its mission is to organize the world’s information and make it universally accessible and useful. But its approach sets it apart.”  Google bought Metaweb and its Freebase database in 2010, buying and removing a potential competitor.   The following year (2011) Google launched the schema.org initiative, bringing along Bing and Yahoo, the other search engines that competed with Google.  While the market share of Bing and Yahoo were small compared to Google, the launch initiative raised hopes that more options would be available for search.  Google noted: “With schema.org, site owners can improve how their sites appear in search results not only on Google, but on Bing, Yahoo! and potentially other search engines as well in the future.”  Nearly a decade later, there is even less competition in search than there was when schema.org was created.

In 2015 a Google employee proposed that schema.org become a W3C community group.  He soon became the chair of the group once it was formed.  

By making schema.org a W3C community, the Google-driven initiative gained credibility through its W3C endorsement as a community-driven standard. Previously, only Google and its initiative partners (Microsoft’s Bing, Yahoo, and later Russia’s Yandex) had any say over the decisions that webmasters and other individuals involved with publishing web content needed to follow, a situation which could have triggered antitrust alarms relating to collusion.   Google also faced the challenge of encouraging webmasters to adopt the schema.org standard.  Webmasters had been slow to embrace the standard and assume the work involved with using it.  Making schema.org an open community-driven standard solved multiple problems for Google at once.  

In normal circumstances — untinged by a massive and domineering tech platform — an open standard should have encouraged webmasters to participate in the standards-making process and express their goals and needs. Ideally, a community-driven standard would be the driver of innovation. It could finally open up the semantic web for the benefit of web users.  But the tight control Google has exercised over the schema.org community has prevented that from happening.

The murky ownership of the schema.org standard

From the beginnings of schema.org, Google’s participation has been more active than anyone else, and Google’s guidance about schema.org has been more detailed than even the official schema.org website.  This has created a great deal of confusion among webmasters about what schema.org requires for compliance to the standard, as opposed to what Google requires for compliance for its search results and ranking.  It’s common for an SEO specialist to ask in a schema.org forum a question about Google’s search results.  Even people with a limited knowledge of schema.org’s mandate assume — correctly — that it exists primarily for the benefit of Google.  

In theory, Google is just one of numerous organizations that implements a standard that is created by a third party.  In practice, Google is both the biggest user of the schema.org standard — and also its primary author.  Google is overwhelmingly the biggest consumer of schema.org structured data.  It also is by far the most active contributor to the standard.  Most other participants are along for the ride: trying to keep up with what Google is deciding internally about how it will use schema.org in its products, and what it is announcing externally about changes Google wants to make to the standard.

In many cases, if you want to understand the schema.org standard, you need to rely on Google’s documentation.  Webmasters routinely complain about the quality of schema.org’s documentation: its ambiguities, or the lack of examples.  Parts of the standard that are not priorities for Google are not well documented anywhere.  If they are priorities for Google, however, Google itself provides excellent documentation about how information should be specified in schema.org so that Google can use it.   Because schema.org’s documentation is poor, the focus of attention stays on Google.

The reliance that nearly everyone has on Google to ascertain compliance with schema.org requirements was highlighted last month by Google’s decision to discontinue its Structured Data Testing Tool, which is widely used by webmasters to check that their schema.org metadata is correct — at least as far as Google is concerned.  Because the concrete implementation requirements of schema.org are often murky, many rely on this Google tool to verify the correctness of the data independently of how the data would be used.  Google is replacing this developer-focused tool with a website that checks whether the metadata will display correctly in Google’s “rich results.”  The new “Rich Results Test Tool” acknowledges finally what’s been an open secret: Google’s promotion of schema.org is primarily about populating its walled garden with content.  

Google’s domination of the schema.org community

The purpose of a W3C group should be to serve everyone, not just a single company. In the case of schema.org, a W3C community has been dominated from the start by a single company: Google.

Google has chaired the schema.org community continuously since its inception in 2015.   Microsoft (Bing) and Yahoo (now Verizon), who are minor players in the search business, participate nominally but are not very active considering they were founding members of schema.org.  Google, in contrast, has multiple employees active in community discussions, steering the direction of conversations.  These employees shape the core decisions, together with a few independent consultants who have longstanding relationships with Google.  It’s hard to imagine any decision happening without Google’s consent.  Google has effective veto power over decisions.

Google’s domination of the schema.org community is possible because the community has no resources of its own.  Google conveniently volunteers the labor of its employees to perform duties related to community business, but these activities will naturally reflect the interests of the employer, Google.  Since other firms don’t have the same financial incentives that Google has through its market dominance of search and smartphones in the outcomes of schema.org decisions, they don’t allocate their employees to spend time on schema.org issues.  Google corners the discussion while appearing to be the most generous contributor.

The absence of governance in the schema.org community

The schema.org community essentially has zero governance — a situation Google is happy with.  There are no formal rules, no formal process for proposals and decisions, no way to appeal a decision, and no formal roles apart from the chair, who ultimately can decide everything. There’s no process of recusal.  Google holds sway in part because the community has no permanent and independent staff.  And there’s no independent board of oversight reviewing how business is conducted.

It’s tempting to see the absence of governance as an example of a group of developers who have a disdain for bureaucracy — that’s the view Google encourages.  But the commercial and social significance of these community decisions is enormous and shouldn’t be cloaked in capricious informality.  Moreover, the more mundane problems of a lack of process are also apparent.  Many people who attempt to make suggestions feel frozen out and unwelcome. Suggestions may be challenged by core insiders who have deep relationships with one another.  The standards- making process itself lacks standardization.  

 In the absence of governance, the possibilities of a conflict of interest are substantial.  First, there’s the problem of self-dealing: Google using its position as the chair of a public forum to prioritize its own commercial interests ahead of others.  Second, there’s the possibility that non-Google proposals will be stopped because they are seen as costly to Google, if only because they create extra work for the largest single user of schema.org structured data.  

As a public company, Google is obligated to its shareholders — not to larger community interests.  A salaried Google employee can’t simultaneously promote his company’s commercial interests and promote interests that could weaken his company’s competitive position.  

Community bias in decisions

Few people want an open W3C community to exhibit biases in their decisions.  But owing to Google’s outsized participation and the absence of governance, decision making that’s biased toward Google’s priorities is common.

Whatever Google wants is fast-tracked — sometimes happening within a matter of days.  If a change to schema.org is needed to support a Google product that needs to ship, nothing will slow down that from happening.

 Suggestions from people not affiliated with Google face a tougher journey.  If the suggestion does not match Google priorities, it is slow-walked. They will be challenged as to their necessity or practicality.  They will languish as an open issue in Github, where they will go unnoticed unless they generate an active discussion.  Eventually, the chair will cull proposals that have been long buried in the interest of closing out open issues.

While individuals and groups can propose suggestions of their own, successful ones tend to be incremental in nature, already aligned with Google’s agenda.  More disruptive or innovative ones are less likely to be adopted.

In the absence of a defined process, the ratification of proposals tends to happen through an informal virtual acclamation.  Various Google employees will conduct a public online discussion agreeing with one another on the merits of adopting a proposal or change.  With “community” sentiment demonstrated, the change is pushed ahead.  

Consumer harm from Google’s capture of schema.org

Google’s domination of schema.org is an essential part of its business model.  Schema.org structured data drives traffic to Google properties, and Google has leveraged it so that it can present fewer links that would drive traffic elsewhere.  The more time consumers spend on Google properties, the more their information decisions are limited to the ads that Google sells.  Consumers need to work harder to find “organic” links (objectively determined by their query and involving no payment to Google) to information sources they seek.

A technical standard should be a public good that benefits all.  In principle, publishers that use schema.org metadata should be able to expand the reach of their information, so that apps from many firms take advantage of it, and consumers have more choices about how and where they get their information.  The motivating idea behind semantic structured data such as schema.org provides is that information becomes independent of platforms.  But ironically, for consumers to enjoy the value of structured data, they mostly need to use Google products.  This is a significant market failure, which hasn’t happened by accident.

The original premise of the semantic web was based on openness.  Publishers freely offered information, and consumers could freely access it.  But the commercial version, driven by Google, has changed this dynamic.  The commercial semantic web isn’t truly open; it is asymmetrically open.  It involves open publishing but closed access.  Web publishers are free to publish their data using the schema.org standard and are actively encouraged to do so by Google. The barriers to creating structured data are minimal, though the barriers to retrieving it aren’t.  

Right now, only a firm with the scale of Google is in a position to access this data and normalize it into something useful for consumers.  Google’s formidable ad revenues allow it to crawl the web and harvest the data for its private gain.  A few other firms are also harvesting this data to build private knowledge graphs that similarly provide gated access.  The goal of open consumer access to this data remains elusive.  A small company may invest time or money to create structured data, but they lack the means to use structured data for their own purposes.   But it doesn’t have to be this way.   

Has Google’s domination of schema.org stifled innovation?

When considering how big tech has influenced innovation, it is necessary to pose a counterfactual question: What might have been possible if the heavy hand of a big tech platform hadn’t been meddling?

Google’s routine challenge to suggestions for additions to the schema.org vocabulary is to question whether the new idea will be used.  “What consuming application is going to use this?” is the common screening question.  If Google isn’t interested in using it, why is it worthwhile doing?  Unless the individual making the suggestion is associated with a huge organization that will build significant infrastructure around the new proposal, the proposal is considered unviable.  

The word choice of “consuming applications” is an example of how Google avoids referring to itself and its market dominance.  The Markup recently revealed how Google coaches its employees to avoid phrases that could get it in additional antitrust trouble.  Within the schema.org community group, Google employees strive to make discussion appear objective, where Google seems disinterested in the decision.  

One area where Google has discouraged alternative developments is in discouraging the linking of schema.org data with data using other metadata vocabularies (standards).  This is significant for multiple reasons.  The schema.org vocabulary is limited in its scope, mostly focusing on commercial entities and not on non-commercial entities.  Because Google is not interested in non-commercial entity coverage, publishers need to rely on other vocabularies.  But Google doesn’t want to look at other vocabularies, claiming that it is too taxing for them to crawl data described by other vocabularies.  In this, Google is making a commercial decision that goes against the principles of linked data (a principle of the semantic web), which explicitly encourages the mixing of vocabularies. For publishers, they are forced to obey Google’s diktats.  Why should they supply metadata that Google, the biggest consumer of schema.org metadata, says it will ignore?  With a few select exceptions, Google mandates that only schema.org metadata should be used in web content and no other semantic vocabularies.  Google sets the vision of what schema.org is, and what it does.

To break this cycle, the public should be asking: How might consumers access and utilize information from the information commons without relying on Google?

There are several paths possible.  One might involve opening up the web crawl to wider use by firms of all sizes.  Another would be to expand the role of the schema.org vocabularies in APIs to support consumer apps.  Whatever path is pursued, it needs to be attractive to small firms and startups to bring greater diversity to consumers and spark innovation.

Possibilities for reform: Getting Google out of the way

If schema.org is to continue as a W3C community and associated with the trust conferred by that designation, then it will require serious reform.  It needs governance — and independence from Google.  It may need to transform into something far more formal than a community group.  

In its current incarnation, it’s difficult to imagine this level of oversight.  The community is resource-starved, and relies on Google to function. But if schema.org isn’t viable without Google’s outsized involvement, then why does it exist at all?  Whose community is it?

There’s no rationale to justify the W3C  lending its endorsement to a community that is dominated by a single company.  One solution is for schema.org to cease being part of the W3C umbrella and return to its prior status of being a Google-sponsored initiative.  That would be the honest solution, barring more sweeping changes.

Another option would be to create a rival W3C standard that isn’t tied to Google and therefore couldn’t be dominated by it, but a standard Google couldn’t afford to ignore.  That would be a more radical option, involving significant reprioritization by publishers.  It would be disruptive in the short term, but might ultimately result in greater innovation.  A starting point for this option would be to explore how to popularize Wikidata as a general-purpose vocabulary that could be used instead of schema.org.

A final option would be for Google to step up in order to step down.  They could acknowledge that they have benefited enormously from the thousands of webmasters and others who contribute structured data and they own a debt to them.  They could offer to a payback in kind.  Google could draw on the recent example of Facebook’s funding of an independent body that will provide oversight that company.  Google could fund a truly independent body to oversee schema.org, and financially guarantee the creation of a new organizational structure.   Such an organization would leave no questions about how decisions are made and would dispel industry concerns that Google is gaining unfair advantages.  Given the heightening regulatory scrutiny of Google, this option is not as extravagant as it may first sound.

On a pragmatic level, I would like to see schema.org to realize its full potential.  This issue is important enough to merit broader discussion, not just in the narrow community of people who work on web metadata, but those involved with regulating technology and looking at antitrust.  Google spends considerable sums, often furtively, hiring academic experts and others to dismiss concerns about their market dominance.  The role of metadata should be to make information more transparent.  That’s why this matters in many ways.

— Michael Andrews

The post Time to end Google’s domination of schema.org appeared first on Story Needle.

Lumping and Splitting in Taxonomy

11 januari 2020 - 7:34pm

Creating a taxonomy — noting which distinctions matter — often seems more art than science.  I’ve been interested in how to think about taxonomy more globally, instead of looking at it as a case-by-case judgment call.  Part of my interest here is a spin off from my interest of birding.  I’m no ornithologist, but I try to learn what I can about the nature of birds.  And species of birds, of course, are classified according to a taxonomy.  

The taxonomy for birds is among the most rigorous out there.  It is debated and litigated, sometimes over decades.  The process involves a progression of “lumps” and “splits” that recalibrate which distinctions are considered significant.  Recently the taxonomy underwent a major revision that reordered the kingdom of birds. 

In the mid-2010s, scientists changed the classification of birds to consider not only anatomical features, but DNA.  In the new ordering, eagles and falcons are not as closely related as was previously assumed. Eagles are closer to vultures, while falcons are closer to parrots.  And pigeons and flamingos are more closely related than thought previously.  Appearance alone is not enough on which to base similarity.

More closely related than you might think (Both produce milk to feed their young) Taxonomy and Information Technology

Taxonomy doesn’t receive the attention it deserves in the IT world.  It seems subjective: vague, hard to predict, potentially the source of arguments.  Taxonomy resembles content: it may be necessary, but it is something to work around — “place taxonomy here when ready.”

But taxonomy can’t be avoided. Even though semantic technologies are becoming richer in describing the characteristics of entities, the properties of entities alone may not be enough to distinguish between types of entities.  Many entities share common properties, and even common values, so it becomes important to be able to indicate what type of entity something is.  We can describe something in terms of its physical properties such as weight, height, color and so on, and still have no idea what it is we are describing.  It can resemble the parlor game of twenty questions: a prolonged discourse that’s prone to howlers.

Classification is the bedrock of algorithms: they drive automated decisions.  Yet taxonomies are human designed.  Taxonomies lack the superficial impartiality of machine-oriented linked data or machine learning classification.  But taxonomies are useful because of their perceived limitations. They require human attention and human judgment.  That helps make data more explainable.  

Humans decide taxonomies — even when machines provide assistance finding patterns of similarity. Users of taxonomies need to understand the basis of similarity.  No matter how experienced the taxonomist or sophisticated the text analysis, the basis of a taxonomy should be explainable and repeatable ideally.  Machine-driven clustering approaches lack these qualities.  

To be durable, a taxonomy needs a reasoned basis and justification.  Business taxonomies can borrow ideas from scientific taxonomies.   

Four approaches can us help decide how to classify categories:

  1. Homology
  2. Analogy
  3. Differentia
  4. Interoperability

Homology and analogy deal with “lumping” — finding commonality among different items.  Differentia and interoperability help define “splitting” — where to break out similar things.

Homology: Discovering shared origins

Homology is a phrase taxonomists use to describe when features, while appearing different, have a common origin and original intent.  For example, mammals have limbs, but the limb could be manifested as an arm or as a flipper.  

Homology refers to cases where things start the same but go in different directions.  It can get at the core essence of a feature: what it enables, without worrying so much how it appears or precisely what it does.  Homology is helpful to find larger categories that link together different things.

There are two ways we can use homology when creating a taxonomy. 

First, we can look at the components or features of items.  We look for what they share in common that might suggest a broader capability to pay attention to.  Lots of devices have embedded microprocessors, even though these devices play different roles in our lives.  Microprocessors provide a common set of capabilities of that even allow different kinds of items to interact with one another, such as in the case of the Internet of Things (IoT).  Homology is not limited to physical items.  Many business models get copied and modified by different industries, but they share common origins and drivers. We can speak of a class of businesses using an online subscription model, for example.

Second, we can consider whole items and how they are used.  Homology can be useful when a distinct thing has more than one use, especially when it doesn’t have a single primary purpose.  Baking soda is advertised as having many purposes and some consumers like products that contain baking soda.  Here we have a category of baking soda-derived products.  In the kitchen, there are many small appliances that have a rotator on which one can attach implements.  They may be called a food processor, a blender, a mixer, or some trademarked proprietary name.  What can they do?  Many tasks: chopping vegetables, making dough, making soups, smoothies, spreads…the list is endless.  But the most seem to be about pulverizing and mixing ingredients.  It’s a broad class of gadgets that share many capabilities, though they scatter in what they offer as they seek to differentiate themselves.

But there’s another approach to lumping things: analogy.    

Analogy: Discovering shared functions

We use analogies all the time in our daily conversation.  Taxonomists focus on what analogies reveal.  

Analogy helps identify things that are functionally similar, and might share a category as a result.

Analogy is the opposite of homology. With analogy, two things start from a different place, but produce a similar result.  For example, the wings of bees and wings of birds are analogous.  They are similar in their function, but different in their origin and details.  Analogies capture common affordances: where different things can be used in similar ways

Analogies are most useful when defining mental categories, such as devices to watch video, or places to go on a first date.  It’s the most subjective kind of taxonomy: different people need to hold similar views in order for these categories to be credible.

Contrasting homology and analogy, we can see two concepts, which represent notions of convergence (from differences to similarity) and divergence (from similarity to differences).

The other end of taxonomy is not about lumping things into broader categories, but splitting them into smaller ones.

Differentia: Defining Segments

Taxonomists talk about differentia (Latin for difference), which is broadly similar to what marketers refer to as segmentation.

Aristotle defined humans as animals capable of articulated speech. His formulation provided a structural pattern still used in taxonomy today:

  • A species equals a genus plus differentia

That is, the differences within a genus define individual species.  

To put it in more general terms: 

  • A segment is a group plus its distinguishing characteristics (its epithet)

A group gets divided into segments based on distinguishing characteristics.  The differentia separates members from other members.  

One of the most popular marketing segmentations relates to generational differences. In the United States, people born after the Second World War are segmented into 4 groups by age.  Other countries use similar segments, but it is not a universal segmentation so I will focus specifically on US nationals.  A common segmentation (with the exact years sometimes varying slightly) is:

  • Generation W (aka “Boomers”): American nationals born between 1946 and 1964
  • Generation X: American nationals born between 1965 and 1980
  • Generation Y (aka “Millennials”): American nationals born between 1981 and 1996
  • Generation Z: American nationals born since 1997

Such segmentation has the virtue of creating category segments that are comprehensive (no item is without a category) and mutually exclusive (no item belongs to more than one category).  It’s clean, though it is not necessarily correct — in the sense that the categories identify what most matters.  

Segments won’t be valuable if the distinctions on which they are based aren’t that important.  A segment could comprise things with a common characteristic that are otherwise quite diverse.  It’s possible for segment to be designed around an incidental characteristic that makes different things seem similar.

The point of differentia is to represent a defining characteristic. Differentia is valuable when it helps us think through which distinctions matter and are valid.  For example, we might segment people by eye color.  But that hardly seems an important way to segment people. Such segmentation encourages us to refine the group we are segmenting.  Eye color is of interest to makers of tinted contact lenses.  But even then, eye color is not a defining characteristic of a potential contact lens customer, even if were a relevant one.

While differentia can be hard to define durably, it can play a useful role in taxonomies.  It seems reasonable to segment aircraft according to the number of passengers they carry, for example.  It can capture one key aspect that represents many important issues.

Interoperability: Distinctions within commonality

A related issue is deciding when things are similar enough to say they are the same, and when we can say they are related but different.

Our final perspective comes from nature. The similarity of species is partly defined by their ability to mate.  Some closely related species of birds, for example, will cross breed.  Other pairs of less similar species lack that ability.  

A similar situation exists with languages.  Where are the distinctions and boundaries between similar languages? And when are differences just dialects and not actually different languages?  In language, mutual-intelligibility plays a role.  (Language also involves convergence and divergence — but we’ll consider their interoperability here).

The presence or absence of connection between distinct things is associated with two overlapping but distinct concepts: 

  1. Interoperability 
  2. Substitution

Both these concepts address ways in which distinct things might be consider the “same.”

Interoperability is most often associated with technology, though it can be applied to other areas, for example, cultural norms such as religions as well.  The presence of interoperability — the ability of distinct things to connect together easily because they follow a common standard or code of operation — is an indication of their similarity.  If things interoperate — they require no change in set up to work together — then they belong to the same “family,” even if the things come from different sources. The absence of interoperability is a sign that these things may not belong together and need to be split.   

Being part of the same family does not imply they are the same.   Any distinctions would relate to the role of each thing in the family (same family, different roles).   Things that follow the same standard may be similar (same role), or they may be complements (different roles).  

If things can be substituted — they are interchangeable but require a different set up to use — they may belong to the same category, but that category may need to be broken down further.  Windows, Linux and MacOS computers can be substituted with one another  — they serve the same role — so they belong to the broader personal computer category (same role, different families).  But they are separate categories because they don’t interoperate.

The value of taxonomies

Defining taxonomies is not easy.  Interpretation is needed to spot the differences that make a difference. We can improve the discovery process by using heuristic perspectives for lumping and splitting. 

Taxonomy is valuable because it can provide a succinct way to express the significance of an entity in relation to another entities.  Sometimes we need a quick summary to boil down the essence of a thing: what’s distinctive about it, so we can see how it relates to a given situation.  Taxonomies help us overcome the fragmentation of information.  

— Michael Andrews

The post Lumping and Splitting in Taxonomy appeared first on Story Needle.

3 Tools to Set Tangible Goals for Content

14 oktober 2019 - 10:25pm

How can writers know when content is good enough to satisfy users?  Content quality should not be the subjective judgment of an individual writer, whose opinion may differ from other writers. Content teams should be able to articulate what the content needs to say to satisfy user expectations. I want to explore three tools that content designers and writers can use help them determine how well their content will meet user needs.  Unlike the straight usability testing of content, these tools provide guidance before content is created and diagnostic evaluation after the content has been made.

Good quality content helps people accomplish their tasks and realize their larger goals. To create high quality content, writers need to understand 

  1. The tasks and goals of people
  2. What knowledge and information people need to use the content
  3. Any issues that could arise that hinder people having a satisfactory outcome

Writers can understand what content needs to address by using tools that focus on the user tasks.  Three useful tools are:

  1. Users stories and Jobs-to-be-Done
  2. Behavior driven design
  3. Task analysis 

Each tool offers specific benefits.

Defining goals: User Stories and Jobs-to-be-Done

User stories and Jobs-to-be-Done (JTBD) are two common approaches to planning customer experiences.  User stories are the default way to plan agile IT.  JTBD has gained a big following in the corporate world, especially among product managers.  Content professionals may participate in projects that use these tools.

I’m not going to recap the extensive literature on user stories and JTBD, much of which isn’t focused specifically on content.  Fortunately, Sarah Richards has explored both these approaches in her book Content Design, and she’s done a great job of showing the relevance of each to the work of content professionals.  For my part I want to explore the uses and limitations of user stories and JTBD as it relates to understanding content quality.

Sarah Richards notes: “a user story is a way of pinning down what the team needs to do without telling them how to do it.”

The basic pattern of a user story is: 

  • As a X [kind of person or role], I want to Y [task] so that I can Z [activity or end goal]

The “so that” clause is both the test of success and the motivation for activity.  User stories separate intermediate goals (Y) from end goals (Z).  If the user is able to get to their next step, the goal beyond the immediate one, we assume that the content is successful.   Richards suggests breaking out the “I want to” into separate discrete tasks if the person has several things they want to do in support of a larger goal.  So, if the user wants to do two things, they should be broken into two separate stories.

JTBD or job stories are similar to user stories, except they focus on the job rather than the user.  Richards states: “Job stories are a better choice if you have only one audience to deal with.”   That’s a good point.  And sometimes everyone has the same needs.  People may belong to different segments, but everyone faces a common situation and needs a common resolution.   Everyone on a cancelled flight wants to get booked on another flight that leaves soon, whether or not they are business class or “basic economy” passengers.  

In summary, the difference between user story and job story is the introductory clause:

  • User story: As a X [persona or audience segment]
  • Job story When [a situation]

What this introductory clause tries to do is introduce some context: what people know, what issue they face, or how they are likely to think about an issue.  But the introductory clause is not precise enough to give us much detail about the context.

User and job stories are a helpful way to break down different tasks and goals that needs to bed addressed.  But it is easy to see how these frameworks are so broad that they might fail to provide specific guidance.  For example, a job story could be:

  • “When the power goes off, I want to know who to contact so that I know when the power will be back on.”  

There are several leaps that occur in this story.  We don’t know if the power outage is isolated to the customer or is widespread.  We assume that having a point of contact is what customers need, and that POC will tell the user when the power will be back on.  Even if that job is how a customer expressed their story, it doesn’t mean the building content around the story will provide the customer with a satisfactory outcome.  

User stories and JTBD are loose, even squishy.  Their vagueness provides latitude on how to address a need, but it can also introduce a degree of ambiguity in what needs to happen.  

User and job stories often include “acceptance criteria” so that teams know when they are done.  In the words of Sarah Richards: “Meeting acceptance criteria gives your team a chance to tick things off the to-do list.”  Richards warns against the dangers of acceptance criteria “that puts the solution up front.”  In other words, the acceptance criteria should avoid getting into details of how something is done, though it should indicate exactly what the user is expecting to be able to do.  

As far as I can tell, no universal format exists for writing acceptance criteria.  They may be a list of questions that the story’s writer considers important.  

But even well-written acceptance criteria will tend to be functional, rather than qualitative.  Acceptance criteria are more about whether something is done than whether it is done well.  We don’t know if it was difficult or easy for the customer to do, or whether it took a lot of time or not.  And we never know for sure if satisfying what the customer wants will enable them to do what they ultimately are looking to accomplish.   

User stories and job stories provide a starting point for thinking about content details, but by themselves these approaches don’t reveal everything a writer will want to know to help the user realize their goals.

Specifying Context: Behavior Driven Design

Behavior driven design (BDD) is used in situations where content shapes how people complete a task.  BDD provides functional specifications that indicates a concrete scenario of the before and after state.  This approach can be helpful to someone working as a product content strategist or UX writer who needs to design flows and write content supporting customer transactions.  

The New York Times is one organization that uses BBD.  Let’s look at this example they’ve published to see how it works.  It is written in the Gherkin language, a computer programming language that is easy for non programmers to read.

Description: As a customer care advocate, I want to update a customer’s credit card on file, so that the customer’s new credit card will be charged during the next billing cycle. Scenario:     Change credit card with a valid credit card     Given:     I have a customer with an existing credit card.      When:     I enter a new valid credit card number.      Then:     The service request is processed successfully.   And:     I can see that the customer’s new card is on file. Scenario:     Change credit card with an invalid credit card number      Given:    I have a customer with an existing credit card.      When:     I enter a new credit card number that is not valid.      Then:     An error shows that the credit card number is not valid

As the example shows, multiple scenarios may be associated with a larger situation.  The example presents a primary user (the customer care advocate) who is interacting with another party, a customer.  This level of contingent interaction can flush out many situations that could be missed.   Publishers should never assume that all the information that customers need is readily available, or that customers will necessarily be viewing information that is relevant to their specific situation.  Publishers benefit by listing different scenarios so they understand information requirements in terms of completeness, channel or device availability, and contextual variability.  So much content now depends on where you live, who you are, your transition history, customer status, etc.  BBD can help to structure the content that needs to be presented.

Content must support customer decision making. BDD provides a framework for thinking about what information customers have and lack relating to a decision.  Let’s consider how BDD could help us think about content relating to a  a hotel room.

ScenarioSome determinable user situationGiven some precondition (user knows where they want to holiday)And some other precondition (user knows their budget)When some action by the user(user visits travel options page)And some other action (user compares hotel prices)Then some testable outcome is achieved (user compares hotel prices)And outcome we can check happens too(user books hotel)

This format allows the writer to think about variables addressed by content (decisions associated with hotel prices) and not be overwhelmed by larger or adjacent issues.  It can help the writer focus on potential hesitation by users when comparing and evaluating hotel prices.  If many users don’t compare the prices, something is obviously wrong. If many don’t book a hotel after checking prices, that also suggests issues.  BDD is designed to be testable.  But we don’t have deploy the design to test it.  Some basic guerrilla usability could flag issues with the content design.  These issues might be too much information (scary), missing information (also scary), or information revealed at the wrong moment (which can feel sneaky.)

I believe that BDD is better than JTBD when specifying the user’s situation and how that influences what they need to know.  We can use BDD to indicate: 

  • What knowledge the user knows already,
  • What decisions the user has already made

We can also indicate that more than one action could be necessary for the user to take.  And there may be more than one outcome.

The power of BDD is that it can help writers pin down more specific aspects of the design.

BDD obviously includes some assumptions about what the user will want to do and even how they will do it.  It may not be the approach to start with if you are designing a novel application or addressing a non-routine scenario.  But in situations were common behaviors and conventions are well known and understood, BDD can help plan content and diagnose that it is performing satisfactorily.

Specifying Performance Standards: Task analysis

Task analysis has been around longer than computers.  When I studied human computer interaction nearly two decades ago, I had to learn about task analysis.  Because it isn’t specific to computers, it can help us think how people use any kind of content.

A basic task analysis pattern would be:

  • Activity Verb
  • Performance standards (quantity/quality)
  • Mode

Here’s an example of a task from a review of task analysis.  The writer would need to add content to explain how to perform the task:

 To instruct the reader on how:

  • To make the nail flush …without damaging the surface of a piece of wood……using a hammer.

Design thinking purists might object that this example presupposes the use of a hammer to pound a nail.  Why not consider a shoe instead?  But the task assumes that certain tools will be used.  That’s a reasonable assumption in many scenarios.  If you are writing instructions on how to assemble a table or mend a shirt, you will assume the reader will need access to certain tools to perform the task.

Yet it is possible to change the mode.  There’s more than one way to wash windows without leaving a streak.  A person could use vinegar and a rag, or use an old newspaper.  If both methods were equally effective, the writer could compare how clearly and succinctly the instructions for each could be.  Remember: consuming the content is part of total work involved with completing a task.

What’s nice about the nail example is that it includes problems that the user might not be thinking about.  The user may just want to make the nail flush.  They may not be focused on how they might fail.  Content supporting the task can be tested with real people to determine if they misuse the tool — getting some unintended consequence.  In our complex world, there is plenty of scope for that to happen.  

Checking Quality

Writers are concerned that customers are successful.  There are many reasons why customers may not be.  Content needs to address a range of situations, and at the same time not be too burdensome to read, view or listen to.  Consuming content is part of the work associated with many tasks.  Content needs to facilitate completion of the task, and not detract from it.  

Much of the poor quality in design ultimately stems from bad assumptions.  Designs reflect bad assumptions about user goals, their knowledge, the information they have available, the decision they are prepared to make, and so on.  The three tools covered in this post can help writers to understand these issues more clearly, so that content created is better quality.

— Michael Andrews

The post 3 Tools to Set Tangible Goals for Content appeared first on Story Needle.

Designing Multi-Purpose Content

2 september 2019 - 5:30pm

Publishers can do more with content when content is able to serve more than one purpose.  This post will provide a short introduction to how to structure content so that it’s multi-purpose. 

First let’s define what multi purpose means. Multi-purpose refers to when core information supports more than one content type. A content type is the structure of content relating to a specific purpose.  Each content type should have a distinct structure reflecting its unique purpose. But often certain essential information may be relevant to different content types. A simple example would be a company address.  The address is a content element used in many different content types such as an “About Us” profile or an event announcement about a meetup hosted by the company.  The same content element can be used in different content types. The address is a multi-purpose content element.

Scenarios where purposes overlap

Publishers have many opportunities to use the same content for different purposes. Another simple scenario can show us how this would work.

Imagine a company is about to release a new product to the market. The product is currently in beta.  The company wants to build awareness of the forthcoming product. There are three audience segments who are interested in the product:

  1. Existing customers of the company
  2. People who follow the sector the company is in, such as journalists, industry analysts, or Wall Street analysts
  3. People who are not current customers of the company but who may be interested in knowing about the company’s future plans

All these groups might be interested in information about the new product.  But each of these three groups has a slightly different reason for being interested in the information.  Even though they will all want to see mostly the same content, they each want to see something different as well.  By breaking content into components, we can separate which audience purposes are identical, and which are similar but different.  

Modeling commonalities 

One use of a content model is to indicate what information is delivered to which audience segment. For some aspects of a topic, audiences will see the same information, while for other aspects different audience segments see information that is specific to them.   

A close relationship exists between the segment for whom the content is designed, and the content type which represents the purpose of the content.   A prospective buyer of a product is probably not interested in a troubleshooting page, but an owner of the product might be.  

Even when different audience segments gravitate toward different content types, they may still share common interests and be seeking some of the same information.  

Different audience segments may have different reasons for being interested in the same basic information.  They may need to see slightly different versions because of their differences in their motivations, which could influence messages framing the significant of the information to the audience segment, and differences in the actions they may wan to take.  

Content teams can plan around what different audience segments want to do after reading the content. 

In  our example, the same basic content about the forthcoming product release can be used in three different content types. They can be used in a customer announcement, in a press release, and in a blog post. The descriptive body of each of these will be the same, conveying basic information about the forthcoming product.  

Three different content types drawing on a common, multi-purpose content element Identifying motivations and managing these as components

When designing content, content teams should have a clear idea who is interested in this information and why.

In our example, the content presented to each segment has a different call-to-action at the end of the body. The customer announcement will include a sign-up call-to-action so that customers can try out the beta version. The press release would include a point of contact, which would provide a name, an email and a telephone number that journalist and others could reach.  The blog post wouldn’t include an active call-to-action, but it might embed social media discussion on Twitter concerning the forthcoming product release — perhaps tweets from beta customers crowing about how marvelous the new product is.  

The motivations of each audience segment can also be managed with distinct content elements in the content model.  Content teams can use content elements to plan and manage specific actions or considerations pertaining to different audience segments.

Thinking about purpose globally

Content teams tend to plan content around tasks. But when content is planned individually to support individual tasks, content teams can miss the opportunity to design the content more efficiently and effectively.  They may create content that addresses a specific audience segment and specific task.  But they’ve created single-purpose content that is difficult to manage and optimize.  

Tasks and information are related but not always tightly coupled.  Different audience segments may have common tasks, even though the information they need to support those tasks could vary in coverage or detail.  In such cases, why different segments are interested in a task could be different, or else their level of knowledge or interest could be different.  The instructions describing how to complete task could be global, but the supporting background content would be unique for different audience segments.  

Conversely, different audience segments may rely on the same information to support different tasks, as in our example.  

Content teams have an opportunity to plan the design of content using a common content model, built around common components that could descriptions, explanations, or actions.    A key aspect of designing multi-purpose content is to separate what information everyone is interested in from information that only certain segments are interested in.  Content will need to adjust to different audience segments depending on the motivations of a segment, and the opportunity the segment offers the organization publishing the content.

The design of content should consider two dimensions affecting multi-purpose content elements:

  1.  What brings these readers to view the content?  (The framing of elements that define the content type where information appears) 
  2.  What do these readers want to do next?  (The framing of the call-to-action or task instructions)

When the answers to those questions are specific to a segment, they will be unique element within the content type.  When several segments share common motivations, the component they view will be the same.

In summary, the same content can be useful to different audiences and in different situations.   Multi-purpose content can be considered the flip-side of personalization. We can separate what everyone needs to know (the multi-purpose part) from what only some people need to know (custom-purpose part).  To design multi-purpose content, one is looking for common elements to share with different segments. In personalization one is looking for specific elements targeted at specific segments.  The design of multi-purpose content considers in close detail what different segments need or want to view, and why.

— Michael Andrews 

The post Designing Multi-Purpose Content appeared first on Story Needle.

Pagina's