Migration Center 3.0 - New Features and Key Changes

Handling Referential Content

For v3, we have introduced a new way of managing referential content in Migration Center. POSTing to the Bulk ANS Content endpoint of Migration Center has the ability to handle multiple pieces of content in a single request, utilizing the content body parameter.

{
   "defaultPriority": ...,
   "defaultWebsite": ...,
   "defaultGroupId": ...,
   "content":[]
}

The content array can hold the “parent” content type such as stories, galleries and videos, as well as the references to the main content type such as tags, authors and images. As a matter of fact, major content types can also reference each other — a story may have a video embedded within it, for example. Further, stories can reference each other as related content.

{
   "defaultPriority": ...,
   "defaultWebsite": ...,
   "defaultGroupId": ...,
   "content":[
      {
         "ANS":{
            "_id": ...,
            "type":"story",
            "version": ...,
            "content_elements":[],
            "credits":{
               "by":[
                  {
                     "referent":{
                        "id":"test-author-1",
                        "type":"author"
                     },
                     "type":"reference"
                  }
               ]
            }
         }
      },
      {
         "ANS":{
            "_id":"test-author-1",
            "type":"author",
            "version": ...,
            "firstName": ...,
            "lastName": ...
         }
      }
   ]
}

So if your story has a gallery full of images, this can all be covered in one call. Let’s take a look at a real example:

{
    "defaultPriority": "historical",
    "defaultWebsite": "stagingva",
    "defaultGroupId": "bulk_story_gallery_images_1",
    "content": [
        {
            "ANS": {
                "_id": "2JKKPZNJLRAZTE6SR5SGHAMHE4",
                "canonical_website": "staging-va",
                "content_elements": [
                    {
                        "_id": "HKUHKNM635A5LNEPF3TS7TKMOM",
                        "content": "Welcome to my story about that thing that happened at that place! In this story, we are going to discuss it!",
                        "type": "text"
                    },
                    {
                        "_id": "VUQ6OREKGNDDDD3I4DDW7GC2KY",
                        "referent": {
                            "id": "VUQ6OREKGNDDDD3I4DDW7GC2KY",
                            "provider": "",
                            "referent_properties": {},
                            "type": "gallery"
                        },
                        "type": "reference"
                    }
                ],
                "credits": {
                    "by": [
                        {
                            "referent": {
                                "id": "megan-delaunay",
                                "provider": "",
                                "referent_properties": {},
                                "type": "author"
                            },
                            "type": "reference"
                        }
                    ]
                },
                "display_date": "2021-03-01T18:11:00Z",
                "first_publish_date": "2021-03-25T17:11:18.943Z",
                "headlines": {
                    "basic": "Thing Happens at Place"
                },
                "language": "es",
                "last_updated_date": "2021-03-25T17:11:19.262Z",
                "owner": {
                    "id": "the-herald"
                },
                "publish_date": "2021-03-25T17:11:18.943Z",
                "source": {
                    "system": "old-cms-name"
                },
                "subheadlines": {
                    "basic": "Isn’t it great?"
                },
                "type": "story",
                "version": "0.10.7"
            }
        },
        {
            "ANS": {
                "_id": "VUQ6OREKGNDDDD3I4DDW7GC2KY",
                "copyright": "true",
                "created_date": "2019-01-24T11:43:00Z",
                "headlines": {
                    "basic": "Test for bulk ingestion of gallery + images"
                },
                "last_updated_date": "2019-01-24T14:00:24Z",
                "type": "gallery",
                "version": "0.10.3",
                "additional_properties": {
                    "version": 4,
                    "published": true,
                    "restricted": false
                },
                "content_elements": [
                    {
                        "_id": "YECLMJCM2NH7NEKKXL7UM34EGU",
                        "referent": {
                            "id": "YECLMJCM2NH7NEKKXL7UM34EGU",
                            "provider": "",
                            "type": "image"
                        },
                        "type": "reference"
                    },
                    {
                        "_id": "FDTR3ND3ABDK3JM7XHK42IGXIM",
                        "referent": {
                            "id": "FDTR3ND3ABDK3JM7XHK42IGXIM",
                            "provider": "",
                            "type": "image"
                        },
                        "type": "reference"
                    },
                    {
                        "_id": "76Q4P4MC2BAMTCCPIRT3MQZGXY",
                        "referent": {
                            "id": "76Q4P4MC2BAMTCCPIRT3MQZGXY",
                            "provider": "",
                            "type": "image"
                        },
                        "type": "reference"
                    }
                ]
            }
        },
        {
            "ANS": {
                "version": "0.10.5",
                "additional_properties": {
                    "originalName": "1439386224062.jpg",
                    "originalUrl": "https://img.estadao.com.br/resources/jpg/2/6/1439386224062.jpg",
                    "published": true,
                    "keywords": [
                        "Internacional",
                        "Geral"
                    ]
                },
                "_id": "YECLMJCM2NH7NEKKXL7UM34EGU",
                "url": "https://img.estadao.com.br/resources/jpg/2/6/1439386224062.jpg",
                "type": "image",
                "subtitle": "Os principais Venezuela",
                "caption": "Vignesh Image 1",
                "credits": {
                    "by": [
                        {
                            "byline": "",
                            "name": "",
                            "type": "author"
                        }
                    ]
                }
            }
        },
        {
            "ANS": {
                "version": "0.10.5",
                "additional_properties": {
                    "originalName": "1644258866991.jpg",
                    "originalUrl": "https://img.estadao.com.br/resources/jpg/1/9/1644258866991.jpg",
                    "published": true,
                    "keywords": [
                        "Internacional",
                        "Geral"
                    ]
                },
                "_id": "FDTR3ND3ABDK3JM7XHK42IGXIM",
                "url": "https://img.estadao.com.br/resources/jpg/1/9/1644258866991.jpg",
                "type": "image",
                "subtitle": "Rússia - França",
                "caption": "Vignesh Image 2",
                "credits": {
                    "by": [
                        {
                            "byline": "Sputnik, Kremlin Pool Photo via AP",
                            "name": "Sputnik, Kremlin Pool Photo via AP",
                            "type": "author"
                        }
                    ]
                },
                "created_date": "2022-02-07T15:34:00Z",
                "last_updated_date": "2022-02-07T15:47:01Z"
            }
        },
        {
            "ANS": {
                "_id": "76Q4P4MC2BAMTCCPIRT3MQZGXY",
                "type": "image",
                "version": "0.10.5",
                "source": {
                    "source_id": "683093",
                    "system": "AGILE"
                },
                "owner": {
                    "id": "sandbox.estadao"
                },
                "subtitle": "Egito",
                "caption": "Vignesh Image 3",
                "credits": {
                    "by": [
                        {
                            "byline": "REUTERS/Mohamed Abd El Ghany",
                            "name": "REUTERS/Mohamed Abd El Ghany",
                            "type": "author"
                        }
                    ]
                },
                "url": "https://img.estadao.com.br/resources/jpg/2/6/1483273265562.jpg",
                "publish_date": "2017-01-01T12:21:00Z",
                "display_date": "2017-01-01T12:47:42Z",
                "additional_properties": {
                    "version": 1,
                    "originalName": "1483273265562.jpg",
                    "originalUrl": "https://img.estadao.com.br/resources/jpg/2/6/1483273265562.jpg",
                    "published": true,
                    "keywords": [
                        "Cidades",
                        "Geral"
                    ]
                }
            }
        }
    ]
}

Source Content Is Optional

In previous versions, ANS Content could not be added to the Migration Center pipeline without first registering its source content. Source content is any content that has come in from a source CMS that has not yet been transformed to ANS. In v3, registering or “archiving” source content is optional and ANS content can be added to the pipeline without any association with its source content.

In previous versions, sourceId and sourceType should be added to each ANS content endpoint’s request body, as shown below:

{
    sourceId: ...,
    sourceType: ...,
    ANS: ...,
    circulations: ...,
    operations: ...
}

For v3, sourceId and sourceType can be omitted from the ANS content endpoint’s request body, as shown below:

{
    ANS: ...,
    circulations: ...,
    operations: ...
}

New Record Count API Endpoint

For v3, we have introduced a new API endpoint that allows developers to understand and retrieve the volume of documents added to the Migration Center pipeline. This can be beneficial for a number of reasons, including:

• Understand the document volume added to the pipeline that have successfully migrated to Arc.
• Understand the document volume staged in the pipeline waiting to be processed.
• Understand the document volume that errored during migration.

curl --location --request GET 'https://api.sandbox.staging.arcpublishing.com/migrations/v3/report/status/count' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer <TOKEN>' \
--data-raw ''

You can also specify the time period for which you need the document counts (using the startDate and endDate parameters).

Arc Additional Properties Request Model

In v3, if a story should be published once it makes it to Arc, requests to ingest ANS should have all fields relevant to the schema of the content type to be published. Also note that the parameter has been renamed from additionalProperties to arcAdditonalProperties.

{
    ANS: ...,
    arcAdditionalProperties: {
        story: {
            publish: true
        }
    }
}

Publishing of videos is accomplished by setting a specific ANS property within the video ANS itself.

{
  ANS: {
     "type": "video",
     "version": "0.8.0",
     "_id": ...,
     "revision": {"published": "true"},
     ...
  }
}

Publishing photos and galleries requires a different property to be set within the ANS.

{
  ANS: {
     "type": photo or gallery
     "version": ...,
     "_id": ...,
     "additional_properties: { "published": true},
     ...
  }
}

Reset API Endpoint Deprecation

The /reset endpoint have been deprecated in 3.0. As the Migration Center platform has evolved since its launch in 2020, the reset feature has been seldom used by our customers due to the uncertainty around its operation and business value. Therefore, we have made a decision to remove access to this API endpoint in the 3.0 release. However, we intend to take a closer look at the reset use case and the overall migration status lifecycle to design a better experience to manage the workflow, so stay tuned for further updates.

Migration Pipeline Status Lifecycle Changes

The Validated status has been deprecated in v3 and replaced by the existing Queued status. In other words, Queued now implies that the ANS content has successfully validated and has been placed on the queue to send to Arc XP. Additionally, a new status called Processing has been added in v3. Processing implies that the ANS content has picked up from the queue and is currently being migrated to Arc XP. The new status lifecycle in v3 is as follows:

1. Pending - If you send source content using the archive API endpoint, this status implies that this content is archived in the pipeline but is pending an ANS Transformation.
2. Queued - This transformed content ANS has successfully validated and has been placed on the queue to send to Arc.
3. Processing - The ANS content has been picked up from the queue and is currently being migrated to Arc.
4. Success - The ANS content has been successfully created in Arc.
5. Circulated - If required, the content has been successfully circulated in Arc.
6. Published - If required, the content has been successfully published in Arc.

Service SLAs

Migration Center makes the following SLA commitments for the Core Platform APIs.

SLA Defintion	SLA Metric (Per Tenant)
Inbound Requests Throughput	800 requests per minute
Outbound Documents Throughput	110 stories per minute* 180 images/videos per minute 180 authors/tags per minute
Average API Response Time	600 ms
Average Error Rate for Inbound Requests	1%

Note

Migration Center JSON containing story ANS is picked up off of the queue at this rate unless your organization has acquired a higher Draft API rate limit. If your Technical Account Manager acquired a higher Draft API rate limit for your organization, that higher rate limit can be communicated to Migration Center and results in more story ingestions per minute.