Many times, we have a requirement to delete the older logs from $logs as the Retention days is not supported at the moment for ADLS gen2 storage account
Please refer to article https://docs.microsoft.com/en-us/azure/storage/common/manage-storage-analytics-logs?tabs=azure-porta... for more information
Deletion of logs from the $logs can be achieved from the following options
This blog demonstrates how we can use ADF for deleting logs from $logs
Create an azure data factory resource from azure portal. If you are new to ADF, please refer this link on how to create one: https://docs.microsoft.com/en-us/azure/data-factory/quickstart-create-data-factory-portal
Open Azure data factory studio in the overview
On the data factory portal click on the pen item -> + > Create pipeline
Click on the Pipeline that has been created -> Under Activities -> General-> select ‘Delete’ and drag it to the pipeline area.
Under Source -> Select New for Datasource =-> choose Azure Data Lake Storage Gen2
In the New linked service, you can use the name that’s by default or customize as per requirement
If the account selection method is azure subscription you can choose the storage account, else you can enter manually too.
Then click on ‘Create’.
In the file path: enter the container as $logs and folder name(according to your requirement)
Coming back to the pipeline, you can set the below properties as per the image :
Step 9 :
Click on Debug, to run the pipeline:
Note : Can make use of Add Trigger to Trigger the pipeline manually
Also you can make use of Add Trigger to Trigger the pipeline on a schedule
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.