How To Send Dataset Dynamically to A REST Service Using ADF using Copy Activity
Published Aug 15 2022 12:24 AM 875 Views
Microsoft

Introduction


In this article, we will build together a pipeline that will allow us to send a dataset as an input to a PUT request using the Copy activity


Main Idea

In this example, we will modify a sample JSON document and send a change via a PUT request.  To observe how this works, we will change the title in the JSON string (https:/https://techcommunity.microsoft.com/jsonplaceholder.typicode.com/posts/1) where the ID is 1.

 

 

{
    "userId": 1,
    "id": 1,
    "title": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
    "body": "quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nostrum rerum est autem sunt rem eveniet architecto"
}

 

 

 

 

 

 

 

 

Pre-requisites

You will require:

Services:
You will need:

  • An Azure Storage Account (Blob storage):
    You will need to create a blob storage account. In this demo, I created an account and saved the metadata.json file in the blob storage. 
  • ADF:
    Added a linked dataset that points to the file (metadata.json) we created in the Blob Linked Service connection with the ADF instance; this will be our dynamic dataset.
    dataset file name: metadata.json

Sally_Dabbah_0-1659346917019.png



Screenshot 2022-08-01 122737.png

Read more here:

 

 

ADF Pipeline

  • Drag a Copy Activity to the pipeline.
  • Specify the path to the JSON document that we saved in the Blob storage as a dataset like so:

image.png

  •  Copy Activity: Add the above dataset as a source, in Sink:
    Sink DataSet : 
    New -> Rest -> Create new linkedService like so:


    image.png

  • Sink Request Method: PUT
    Sally_Dabbah_1-1659347004165.png

     

 Output:
Here we can see that the request was successfully executed, and the JSON object is modified.

Sally_Dabbah_1-1659348543088.png

 

 

 

 

 

 

 






Co-Authors
Version history
Last update:
‎Aug 15 2022 12:18 AM
Updated by: