How to work with WebSked Collections

This document walks you through populating WebSked Collections dynamically and querying for items in the Content API that match a simple tag query or a section query.

For example, if you want to render 10 items from a Collection, but the Collection has been manually populated with only four published items, dynamic Collections lets you backfill those remaining six slots based on a story tag or a section.

WebSked deduplicates content between your backfill and your Collection, which ensures the same item never appears in your backfill if you already manually added it to your Collection.

Workflow

Follow these steps to take advantage of dynamic Collections:

1. Create a Collection in WebSked or through the Collections API.

2. Add a backfill query for either a list of tags or a list of sections to the Collection.

When the Collection is requested through the Content API, the Content API:

1. Checks if there are enough items in the Collection to fulfill the request and return those items.

2. If slots remain, the Content API looks at the query attached to the Collection and queries for matching items.

3. The Content API then appends the query results to the original collection results, de-duplicates any items, and returns that response.

Setting the backfill query

Most organizations can set the backfill query in WebSked, but you can use the Collections API as well. The specified tags or sections that you use to backfill a Collection are saved in the collection’s ANS object under a trait called dynamic_items. The following code sample provides an example of a Collection with dynamic_items:

{
  "_id": "ZYXWVUTSRQPONMLKJIHGFEDCBA",
  "type": "collection",
  "version": "0.10.6",
  "created_date": "2017-06-24T09:50:50.52Z",
  "last_updated_date": "2017-06-24T09:50:50.52Z",
  "canonical_website": "washpost",
  "content_aliases": [ "man-dog-bite-collection", "dog-collection-1" ],
  "dynamic_items": {
    "type": "sections",
    "ids": ["/politics"]
  },
  "credits": {
    "by": [
      {
        "name": "John Q. Reporter",
        "byline": "John Q. Reporter",
        "org": "The Washington Post",
        "type": "author",
        "version": "0.10.6"
      }
    ]
  },
  "language": "en",
  "location": "Washington, D.C.",
  "canonical_url": "/news/dogs/man-bites-dog",
  "short_url": "http://wapo.st/1Crp6bY",
  "headlines": {
    "basic": "The Man Bites Dog collection",
    "twitter": "Lots of Items about Man Biting Dog"
  ...
  ...
}

The dynamic_items trait has two allowed types: sections and tags. The dynamic_items.ids property takes a list of values that you want the query to match by. The previous example backfills your Collection with stories from the /politics section.

The example backfills with stories that are tagged with either dogs or cats:

    ...
     "dynamic_items": {
        "type": "tags",
        "ids": ["dogs", "cats"]
      },
    ...

Most dynamic Collections are configured through WebSked. See Collection API.

The next section explains more about using tags and sections to populate the backfill for more than one website.

Querying the Content API for Collections

Querying the Content API for a dynamic Collection is no different than a static Collection. By default, if the dynamic_items trait of a collection is filled with a query, Content API backfills the Collection with queried items. These items are denoted in the response by additional_properties.backfill: true.

Two endpoints in the Content API can return a Collection:

• /content/v4/collections
• /content/v4.

The /content/v4 endpoint does not support dynamic backfill and can return only the first 20 static items of a Collection. If you want to use dynamic backfill, you must use the /content/v4/collections endpoint, which is better optimized for returning Collections.

The /content/v4/collections endpoint returns 20 items at a time and allows pagination through the rest of the possible static or dynamic items. Arc XP strongly encourages using the /content/v4/collections endpoint for all collections requests.

Endpoint comparison

The following table shows the features and limitations of each of the Collections endpoints.

Feature	/content/v4/collections/	/content/v4/
Returns denormalized items	YES	YES
Dynamic Collections	YES	NO
Pagination	YES	NO
Per page limit	20	20
Total items limit	500	20
Backfilled items limit	100	n/a

ElasticSearch query

When the Content API receives a request for collection with dynamic_items, it turns the dynamic_items into an elasticsearch query. The system uses the ElasticSearch query to find matching results to add to your collection responses.

For dynamic backfilling through tags, the Content API queries against taxonomy.tags.slug for each of the tags in the list. For sections, the Content API queries against taxonomy.sections._id for each of the sections in the list. In both options, the default sort is display_date:desc.

Limit

While a collection can include up to 500 static items, dynamic backfill works only up to 100 total items. For example, if your collection has 60 static items, the Collection API backfills 40 items to get to a total of 100 items. If your collection has 200 static items, the Collection API does not backfill any items.

Published versus unpublished content

The /content/v4/collections endpoint provides two query parameters for accessing published or unpublished content. By default, the /content/v4/collections endpoint returns only published collections and the published stories within. However, you can change these query parameters to get the exact combination of published and unpublished content you want:

?published	?include_unpublished	Response
TRUE (default)	FALSE (default)	Published Collection Published Content within Collection
TRUE	TRUE	Published Collection Published and Unpublished Content within Collection
FALSE	n/a	Unpublished Collection Draft Content within Collection