The CEF Log format is used by ArcSight.
Here is the sample log:
CEF:0|Check Point|FireWall-1|4.1|accept|CP FW In Action:accept
Service:telnet Rule:5 ( Sec Log)|Low| eventId=116
externalId=arcsightDemo:54 proto=TCP customerURI=/All Customers/ArcNet
Customers/west.arcnet categorySignificance=/Normal categoryBehavior=/Access
categoryDeviceGroup=/Firewall catdt=Firewall categoryOutcome=/Success
categoryObject=/Host/Application/Service art=1398755279514 act=accept
rt=1398755279514 deviceDirection=0 shost=node9774.dslzn23.pacbell.net
src=192.168.10.138 sourceZoneURI=/All Zones/System Zones/Private Address
Space spt=2814 dhost=w2ksj101.sj1.west.arcnet.com dst=209.128.98.149
destinationZoneURI=/All Zones/ArcNet Zones/west.arcnet.com - external
destinationTranslatedAddress=10.0.20.21 destinationTranslatedZoneURI=/All
Zones/ArcNet Zones/sj1.west.arcnet.com - internal dproc=telnet
fileType=security cs1=/Pass/Accept cs2=eth-s1p4c0 cs3=inbound cs4=5 cn2=0
cn3=0 cs1Label=v2.x ArcSight Category cs2Label=v2.x Custom String
cs3Label=v2.x Custom String cs4Label=v2.x Custom String cs5Label=v2.x
Custom String cs6Label=v2.x Custom String cn1Label=v2.x Custom Number
cn2Label=v2.x Custom Number cn3Label=v2.x Custom Number
deviceCustomDate1Label=v2.x Custom Date deviceCustomDate2Label=v2.x Custom
Date ahost=fe80:0:0:0:d12a:31e3:8dca:9d20%11 agt=192.168.217.129
agentZoneURI=/All Zones/ArcNet Zones/sj2.west.arcnet.com - internal
av=2.1.0.3401.0 atz=America/Chicago aid=3XPpfc0UBABCAAUZDy8Vfdw\=\=
at=checkpointfirewall_opsec dvchost=cpfwsj104.sj1.west.arcnet.com
dvc=10.0.112.3 deviceZoneURI=/All Zones/ArcNet Zones/sj2.west.arcnet.com -
internal dtz=America/Chicago deviceInboundInterface=eth-s1p4c0 _cefVer=0.1
The delimiter are mixed by pipeline, colon, tag (ex:cs1, cs2, src, dat,
etc.).
But both Pig and Hive have to use the same delimiter to parse logs.
If I just need to extract specific tag(or value) for calculating (ex:src,
dat), like counting Top10 connection IP pairs, is there any idea to do this?
Thanks all!
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
