The CEF Log format is used by ArcSight.

Here is the sample log:

CEF:0|Check Point|FireWall-1|4.1|accept|CP FW In Action:accept
Service:telnet Rule:5 ( Sec Log)|Low| eventId=116
externalId=arcsightDemo:54 proto=TCP customerURI=/All Customers/ArcNet
Customers/west.arcnet categorySignificance=/Normal categoryBehavior=/Access
categoryDeviceGroup=/Firewall catdt=Firewall categoryOutcome=/Success
categoryObject=/Host/Application/Service art=1398755279514 act=accept
rt=1398755279514 deviceDirection=0 shost=node9774.dslzn23.pacbell.net
src= sourceZoneURI=/All Zones/System Zones/Private Address
Space spt=2814 dhost=w2ksj101.sj1.west.arcnet.com dst=
destinationZoneURI=/All Zones/ArcNet Zones/west.arcnet.com - external
destinationTranslatedAddress= destinationTranslatedZoneURI=/All
Zones/ArcNet Zones/sj1.west.arcnet.com - internal dproc=telnet
fileType=security cs1=/Pass/Accept cs2=eth-s1p4c0 cs3=inbound cs4=5 cn2=0
cn3=0 cs1Label=v2.x ArcSight Category cs2Label=v2.x Custom String
cs3Label=v2.x Custom String cs4Label=v2.x Custom String cs5Label=v2.x
Custom String cs6Label=v2.x Custom String cn1Label=v2.x Custom Number
cn2Label=v2.x Custom Number cn3Label=v2.x Custom Number
deviceCustomDate1Label=v2.x Custom Date deviceCustomDate2Label=v2.x Custom
Date ahost=fe80:0:0:0:d12a:31e3:8dca:9d20%11 agt=
agentZoneURI=/All Zones/ArcNet Zones/sj2.west.arcnet.com - internal
av= atz=America/Chicago aid=3XPpfc0UBABCAAUZDy8Vfdw\=\=
at=checkpointfirewall_opsec dvchost=cpfwsj104.sj1.west.arcnet.com
dvc= deviceZoneURI=/All Zones/ArcNet Zones/sj2.west.arcnet.com -
internal dtz=America/Chicago deviceInboundInterface=eth-s1p4c0 _cefVer=0.1

The delimiter are mixed by pipeline, colon, tag (ex:cs1, cs2, src, dat,

But both Pig and Hive have to use the same delimiter to parse logs.

If I just need to extract specific tag(or value) for calculating (ex:src,
dat), like counting Top10 connection IP pairs, is there any idea to do this?

Thanks all!

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
postedApr 29, '14 at 5:07p
activeApr 29, '14 at 5:07p

1 user in discussion

Ivan Hsueh: 1 post



site design / logo © 2023 Grokbase