What's new: ASIM File Activity schema

Published Aug 04 2021 04:52 AM 3,552 Views
Microsoft

Hello everyone,

 

Continuing our normalization journey, we now add the file activity schema.

 

Retweet, Re-share on LinkedIn

 

Why should you care?

 

In addition to ASIM advantages: cross source analytics, source agnostic rules, and ease of use, the File Activity Schema lets you write rules that span endpoint, server, and cloud activity. We have included parsers for Sysmon, Microsoft 365 Defender for Endpoint, SharePoint, OneDrive, and Azure Storage. For example:

 

  • Analyzing file activity is instrumental for ransomware detection. Now your on-prem ransomware analytics can secure cloud workloads.

 

  • When looking for malware leftovers, you will find them on the affected endpoints and on cloud services that may have served to spread them.
Ofer_Shezaf_0-1628077271391.png

 

Learn more

 

Read more about Azure Sentinel Information Model and the File Activity schema, and deploy the File Activity parser packs in a single click using an ARM template

 

Join us to learn more about the Azure Sentinel information model in two webinars:

 

  • The Information Model: Understanding Normalization in Azure Sentinel: Presentation, YouTube.
  • Deep Dive into Azure Sentinel Normalizing Parsers and Normalized Content: next week, register here.

 

Special thanks to @Yaron Fruchtmann, who made all this possible.

 

Why normalization, and what is the Azure Sentinel Information Model?

 

Working with various data types and tables together presents a challenge. You must become familiar with many different data types and schemas, write and use a unique set of analytics rules, workbooks, and hunting queries for each, even for those that share commonalities (for example, DNS servers). Correlation between the different data types necessary for investigation and hunting is also tricky.

 

The Azure Sentinel Information Model (ASIM) provides a seamless experience for handling various sources in uniform, normalized views. ASIM aligns with the Open-Source Security Events Metadata (OSSEM) common information model, promoting vendor agnostic, industry-wide normalization. ASIM:

  • Allows source agnostic content and solutions
  • Simplifies analyst use of the data in sentinel workspaces

 

The current implementation is based on query time normalization using KQL functions. And includes the following:

  • Normalized schemas cover standard sets of predictable event types that are easy to work with and build unified capabilities. The schema defines which fields should represent an event, a normalized column naming convention, and a standard format for the field values.
  • Parsers map existing data to the normalized schemas. Parsers are implemented using KQL functions.
  • Content for each normalized schema includes analytics rules, workbooks, hunting queries, and additional content. This content works on any normalized data without the need to create source-specific content.

Thanks!

1 Comment
Occasional Contributor

Ofer -

How can we maintain these ASIM parsers going forward? It looks like these are under active development, so when they change, how do we make sure we get all the new updates installed as soon as possible after release? When they are updated, do we just click the associated 'Deploy to Azure' buttons again? Is there a way to track what versions of each parser are currently installed? And when these are eventually incorporated into the standard Sentinel install (as I assume is the intent) will the new code transparently overwrite what we had previously installed from GitHub? Also, I see that there are a variety of rule templates in Sentinel that already depend on these (which is actually where I heard about ASIM in the first place). As new templates are added, will there be some kind of reference to the required version of the associated parsers?
%3CLINGO-SUB%20id%3D%22lingo-sub-2609732%22%20slang%3D%22en-US%22%3EWhat's%20new%3A%20ASIM%20File%20Activity%20schema%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2609732%22%20slang%3D%22en-US%22%3E%3CP%3EHello%20everyone%2C%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EContinuing%20our%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fazure-sentinel%2Fwhat-s-new-asim-authentication-process-registry-and-enhanced%2Fba-p%2F2502268%22%20target%3D%22_blank%22%3Enormalization%20journey%3C%2FA%3E%2C%20we%20now%20add%20the%20%3CA%20href%3D%22https%3A%2F%2Faka.ms%2FAzSentinelFileEventDoc%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3Efile%20activity%20schema%3C%2FA%3E.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSTRONG%3E%3CA%20href%3D%22https%3A%2F%2Ftwitter.com%2Foshezaf%2Fstatus%2F1422889505871306752%3Fs%3D20%22%20target%3D%22_self%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3ERetweet%3C%2FA%3E%2C%20%3CA%20href%3D%22https%3A%2F%2Fwww.linkedin.com%2Fposts%2Foshezaf_whats-new-asim-file-activity-schema-activity-6828655435965313024-fptY%22%20target%3D%22_self%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3ERe-share%20on%20LinkedIn%3C%2FA%3E%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--385541880%22%20id%3D%22toc-hId--385541879%22%20id%3D%22toc-hId--385541879%22%20id%3D%22toc-hId--385541879%22%3EWhy%20should%20you%20care%3F%3C%2FH2%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CTABLE%20border%3D%220%22%20width%3D%22100%25%22%3E%0A%3CTBODY%3E%0A%3CTR%3E%0A%3CTD%20width%3D%2250%25%22%20style%3D%22border-style%3A%20none%3B%20width%3A%2040%25%3B%22%3E%3CP%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EIn%20addition%20to%20ASIM%20advantages%3A%20cross%20source%20analytics%2C%20source%20agnostic%20rules%2C%20and%20ease%20of%20use%2C%20the%20File%20Activity%20Schema%20lets%20you%20write%20rules%20that%20span%20endpoint%2C%20server%2C%20and%20cloud%20activity.%20We%20have%20included%20parsers%20for%26nbsp%3B%3C%2FSPAN%3E%3CSTRONG%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3ESysmon%2C%20Microsoft%20365%20Defender%20for%20Endpoint%2C%20SharePoint%2C%20OneDrive%2C%20and%20Azure%20Storage%3C%2FSPAN%3E%3C%2FSTRONG%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3E.%20For%20example%3A%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3E%26nbsp%3B%3C%2FSPAN%3E%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EAnalyzing%20file%20activity%20is%20instrumental%20for%20ransomware%20detection.%20Now%20your%20on-prem%20ransomware%20analytics%20can%20secure%20cloud%20workloads.%3C%2FSPAN%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3E%26nbsp%3B%3C%2FSPAN%3E%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3EWhen%20looking%20for%20malware%20leftovers%2C%20you%20will%20find%20them%20on%20the%20affected%20endpoints%26nbsp%3B%3C%2FSPAN%3E%3CSTRONG%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3Eand%26nbsp%3B%3C%2FSPAN%3E%3C%2FSTRONG%3E%3CSPAN%20data-preserver-spaces%3D%22true%22%3Eon%20cloud%20services%20that%20may%20have%20served%20to%20spread%20them.%3C%2FSPAN%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3C%2FTD%3E%0A%3CTD%20width%3D%2250%25%22%20style%3D%22border-style%3A%20none%3B%20width%3A%2060%25%3B%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Ofer_Shezaf_0-1628077271391.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F300420iA7280CF8422A727D%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Ofer_Shezaf_0-1628077271391.png%22%20alt%3D%22Ofer_Shezaf_0-1628077271391.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FTD%3E%0A%3C%2FTR%3E%0A%3C%2FTBODY%3E%0A%3C%2FTABLE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-2101970953%22%20id%3D%22toc-hId-2101970954%22%20id%3D%22toc-hId-2101970954%22%20id%3D%22toc-hId-2101970954%22%3ELearn%20more%3C%2FH2%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ERead%20more%20about%20%3CA%20href%3D%22https%3A%2F%2Faka.ms%2FAzSentinelNormalization%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3EAzure%20Sentinel%20Information%20Model%3C%2FA%3E%20and%20the%20%3CA%20href%3D%22https%3A%2F%2Faka.ms%2FAzSentinelFileEventDoc%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3EFile%20Activity%20schema%3C%2FA%3E%2C%20and%20deploy%20the%26nbsp%3BFile%20Activity%26nbsp%3Bparser%20packs%20in%20a%20single%20click%20using%20an%20%3CA%20href%3D%22https%3A%2F%2Faka.ms%2FAzSentinelFileEvent%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3EARM%20template%3C%2FA%3E.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EJoin%20us%20to%20learn%20more%20about%20the%20Azure%20Sentinel%20information%20model%20in%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fforms.office.com%252FPages%252FResponsePage.aspx%253Fid%253Dv4j5cvGGr0GRqy180BHbR_0A4IaJRDNBnp8pjCkWnwhUQVdTOFVQN1JNMFZaR0FLQkFLVENXQVNDMS4u%26amp%3Bdata%3D04%257C01%257COfer.Shezaf%2540microsoft.com%257C0718c8e4caa24b5e275e08d93032332d%257C72f988bf86f141af91ab2d7cd011db47%257C1%257C0%257C637593815232343810%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3DI7sGVrOcX8uAhyBdefSkpWRuCUGpMAx5xsbBsSqq60I%253D%26amp%3Breserved%3D0%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Etwo%20webinars%3C%2FA%3E%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3E%3CSTRONG%3EThe%20Information%20Model%3A%20Understanding%20Normalization%20in%20Azure%20Sentinel%3A%20%3CA%20href%3D%22https%3A%2F%2F1drv.ms%2Fb%2Fs!AnEPjr8tHcNmjDY1cro08Fk3KUj-%3Fe%3DmurYHG%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3EPresentation%3C%2FA%3E%2C%20%3CA%20href%3D%22https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DWoGD-JeC7ng%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3EYouTube%3C%2FA%3E.%3C%2FSTRONG%3E%3C%2FLI%3E%0A%3CLI%3E%3CSTRONG%3EDeep%20Dive%20into%20Azure%20Sentinel%20Normalizing%20Parsers%20and%20Normalized%20Content%3A%20next%20week%2C%20register%20%3CA%20href%3D%22https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fforms.office.com%252FPages%252FResponsePage.aspx%253Fid%253Dv4j5cvGGr0GRqy180BHbR_0A4IaJRDNBnp8pjCkWnwhUQVdTOFVQN1JNMFZaR0FLQkFLVENXQVNDMS4u%26amp%3Bdata%3D04%257C01%257COfer.Shezaf%2540microsoft.com%257C0718c8e4caa24b5e275e08d93032332d%257C72f988bf86f141af91ab2d7cd011db47%257C1%257C0%257C637593815232343810%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3DI7sGVrOcX8uAhyBdefSkpWRuCUGpMAx5xsbBsSqq60I%253D%26amp%3Breserved%3D0%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehere%3C%2FA%3E.%3C%2FSTRONG%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%3CSTRONG%3E%26nbsp%3B%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3ESpecial%20thanks%20to%26nbsp%3B%3CA%20href%3D%22mailto%3AYaron.Fruchtmann%40microsoft.com%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3E%40Yaron%20Fruchtmann%3C%2FA%3E%2C%20who%20made%20all%20this%20possible.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-294516490%22%20id%3D%22toc-hId-294516491%22%20id%3D%22toc-hId-294516491%22%20id%3D%22toc-hId-294516491%22%3EWhy%20normalization%2C%20and%20what%20is%20the%20Azure%20Sentinel%20Information%20Model%3F%3C%2FH2%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EWorking%20with%20various%20data%20types%20and%20tables%20together%20presents%20a%20challenge.%20You%20must%20become%20familiar%20with%20many%20different%20data%20types%20and%20schemas%2C%20write%20and%20use%20a%20unique%20set%20of%20analytics%20rules%2C%20workbooks%2C%20and%20hunting%20queries%20for%20each%2C%20even%20for%20those%20that%20share%20commonalities%20(for%20example%2C%20DNS%20servers).%20Correlation%20between%20the%20different%20data%20types%20necessary%20for%20investigation%20and%20hunting%20is%20also%20tricky.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThe%20Azure%20Sentinel%20Information%20Model%20(ASIM)%20provides%20a%20seamless%20experience%20for%20handling%20various%20sources%20in%20uniform%2C%20normalized%20views.%20ASIM%20aligns%20with%20the%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fossemproject.com%252Fintro.html%26amp%3Bdata%3D04%257C01%257COfer.Shezaf%2540microsoft.com%257C0718c8e4caa24b5e275e08d93032332d%257C72f988bf86f141af91ab2d7cd011db47%257C1%257C0%257C637593815232353805%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3DuX8QbedyVN4%252BblDiXoHlWnJFLt6m28cZtlNlt3%252Fm5fg%253D%26amp%3Breserved%3D0%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3EOpen-Source%20Security%20Events%20Metadata%20(OSSEM)%3C%2FA%3E%26nbsp%3Bcommon%20information%20model%2C%20promoting%20vendor%20agnostic%2C%20industry-wide%20normalization.%20ASIM%3A%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3EAllows%20source%20agnostic%20content%20and%20solutions%3C%2FLI%3E%0A%3CLI%3ESimplifies%20analyst%20use%20of%20the%20data%20in%20sentinel%20workspaces%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThe%20current%20implementation%20is%20based%20on%20query%20time%20normalization%20using%20KQL%20functions.%20And%20includes%20the%20following%3A%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3E%3CSTRONG%3ENormalized%20schemas%3C%2FSTRONG%3E%26nbsp%3Bcover%20standard%20sets%20of%20predictable%20event%20types%20that%20are%20easy%20to%20work%20with%20and%20build%20unified%20capabilities.%20The%20schema%20defines%20which%20fields%20should%20represent%20an%20event%2C%20a%20normalized%20column%20naming%20convention%2C%20and%20a%20standard%20format%20for%20the%20field%20values.%3C%2FLI%3E%0A%3CLI%3E%3CSTRONG%3EParsers%3C%2FSTRONG%3E%26nbsp%3Bmap%20existing%20data%20to%20the%20normalized%20schemas.%20Parsers%20are%20implemented%20using%20%3CA%20href%3D%22https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fdocs.microsoft.com%252Fen-us%252Fazure%252Fdata-explorer%252Fkusto%252Fquery%252Ffunctions%252Fuser-defined-functions%26amp%3Bdata%3D04%257C01%257COfer.Shezaf%2540microsoft.com%257C0718c8e4caa24b5e275e08d93032332d%257C72f988bf86f141af91ab2d7cd011db47%257C1%257C0%257C637593815232358803%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3D1gdREgYahRpYVWFZ1Dshjbmoqcmu3Zx7WBnlvUgvIt4%253D%26amp%3Breserved%3D0%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3EKQL%20functions%3C%2FA%3E.%3C%2FLI%3E%0A%3CLI%3E%3CSTRONG%3EContent%20for%20each%20normalized%20schema%3C%2FSTRONG%3E%26nbsp%3Bincludes%20analytics%20rules%2C%20workbooks%2C%20hunting%20queries%2C%20and%20additional%20content.%20This%20content%20works%20on%20any%20normalized%20data%20without%20the%20need%20to%20create%20source-specific%20content.%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3EThanks!%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-2609732%22%20slang%3D%22en-US%22%3E%3CP%3EFind%20traces%20of%20malware%20and%20ransomware%20also%20in%20cloud%20storage.%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2609732%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EWhat's%20New%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2641057%22%20slang%3D%22en-US%22%3ERe%3A%20What's%20new%3A%20ASIM%20File%20Activity%20schema%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2641057%22%20slang%3D%22en-US%22%3E%3CP%3EOfer%20-%3C%2FP%3E%3CDIV%3EHow%20can%20we%20maintain%20these%20ASIM%20parsers%20going%20forward%3F%20It%20looks%20like%20these%20are%20under%20active%20development%2C%20so%20when%20they%20change%2C%20how%20do%20we%20make%20sure%20we%20get%20all%20the%20new%20updates%20installed%20as%20soon%20as%20possible%20after%20release%3F%20When%20they%20are%20updated%2C%20do%20we%20just%20click%20the%20associated%20'Deploy%20to%20Azure'%20buttons%20again%3F%20Is%20there%20a%20way%20to%20track%20what%20versions%20of%20each%20parser%20are%20currently%20installed%3F%20And%20when%20these%20are%20eventually%20incorporated%20into%20the%20standard%20Sentinel%20install%20(as%20I%20assume%20is%20the%20intent)%20will%20the%20new%20code%20transparently%20overwrite%20what%20we%20had%20previously%20installed%20from%20GitHub%3F%20Also%2C%20I%20see%20that%20there%20are%20a%20variety%20of%20rule%20templates%20in%20Sentinel%20that%20already%20depend%20on%20these%20(which%20is%20actually%20where%20I%20heard%20about%20ASIM%20in%20the%20first%20place).%20As%20new%20templates%20are%20added%2C%20will%20there%20be%20some%20kind%20of%20reference%20to%20the%20required%20version%20of%20the%20associated%20parsers%3F%3C%2FDIV%3E%3C%2FLINGO-BODY%3E
Co-Authors
Version history
Last update:
‎Aug 04 2021 05:00 AM
Updated by:
www.000webhost.com