The CEF Log format is used by ArcSight.

Here is the sample log:

CEF:0|Check Point|FireWall-1|4.1|accept|CP FW In Action:accept
Service:telnet Rule:5 ( Sec Log)|Low| eventId=116
externalId=arcsightDemo:54 proto=TCP customerURI=/All Customers/ArcNet
Customers/west.arcnet categorySignificance=/Normal categoryBehavior=/Access
categoryDeviceGroup=/Firewall catdt=Firewall categoryOutcome=/Success
categoryObject=/Host/Application/Service art=1398755279514 act=accept
rt=1398755279514 deviceDirection=0 shost=node9774.dslzn23.pacbell.net
src= sourceZoneURI=/All Zones/System Zones/Private Address
Space spt=2814 dhost=w2ksj101.sj1.west.arcnet.com dst=
destinationZoneURI=/All Zones/ArcNet Zones/west.arcnet.com - external
destinationTranslatedAddress= destinationTranslatedZoneURI=/All
Zones/ArcNet Zones/sj1.west.arcnet.com - internal dproc=telnet
fileType=security cs1=/Pass/Accept cs2=eth-s1p4c0 cs3=inbound cs4=5 cn2=0
cn3=0 cs1Label=v2.x ArcSight Category cs2Label=v2.x Custom String
cs3Label=v2.x Custom String cs4Label=v2.x Custom String cs5Label=v2.x
Custom String cs6Label=v2.x Custom String cn1Label=v2.x Custom Number
cn2Label=v2.x Custom Number cn3Label=v2.x Custom Number
deviceCustomDate1Label=v2.x Custom Date deviceCustomDate2Label=v2.x Custom
Date ahost=fe80:0:0:0:d12a:31e3:8dca:9d20%11 agt=
agentZoneURI=/All Zones/ArcNet Zones/sj2.west.arcnet.com - internal
av= atz=America/Chicago aid=3XPpfc0UBABCAAUZDy8Vfdw\=\=
at=checkpointfirewall_opsec dvchost=cpfwsj104.sj1.west.arcnet.com
dvc= deviceZoneURI=/All Zones/ArcNet Zones/sj2.west.arcnet.com -
internal dtz=America/Chicago deviceInboundInterface=eth-s1p4c0 _cefVer=0.1

The delimiter are mixed by pipeline, colon, tag (ex:cs1, cs2, src, dat,

But both Pig and Hive have to use the same delimiter to parse logs.

If I just need to extract specific tag(or value) for calculating (ex:src,
dat), like counting Top10 connection IP pairs, is there any idea to do this?

Thanks all!

To unsubscribe from this group and stop receiving emails from it, send an email to scm-users+unsubscribe@cloudera.org.

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 1 of 1 | next ›
Discussion Overview
groupscm-users @
postedApr 29, '14 at 5:07p
activeApr 29, '14 at 5:07p

1 user in discussion

Ivan Hsueh: 1 post



site design / logo © 2022 Grokbase