SMF to one line for each event



All about SAF, RACF, encryption, Firewall, Risk assessment and integrity concepts

SMF to one line for each event

Postby frogerc » Tue Jun 08, 2010 10:51 pm

Hello All.

I'm analyzing the SMF, but as you all know, the format is multiline.

Does someone know how to convert it to one line for each event?

This way I can use SQL and it would be easier to understand.

Thanks
frogerc
 
Posts: 6
Joined: Wed May 12, 2010 7:47 pm
Has thanked: 0 time
Been thanked: 0 time

Re: SMF to one line for each event

Postby enrico-sorichetti » Tue Jun 08, 2010 11:32 pm

I'm analyzing the SMF, but as you all know, the format is multiline.

please clarify what You mean by multiline , not a common term for MF datasets
cheers
enrico
When I tell somebody to RTFM or STFW I usually have the page open in another tab/window of my browser,
so that I am sure that the information requested can be reached with a very small effort
enrico-sorichetti
Global moderator
 
Posts: 2994
Joined: Fri Apr 18, 2008 11:25 pm
Has thanked: 0 time
Been thanked: 164 times

Re: SMF to one line for each event

Postby Robert Sample » Tue Jun 08, 2010 11:56 pm

You would be better off checking with your site support group to see if they are using MICS or MXG or another SMF analysis tool that accumulates the data for you. Not only are there multiple records that relate to a single event (such as the various type 30 records for a job), there are also single records that contain data on many things (such as disk pack data). It is not a simple task to properly decode the SMF records.
Robert Sample
Global moderator
 
Posts: 3719
Joined: Sat Dec 19, 2009 8:32 pm
Location: Dubuque, Iowa, USA
Has thanked: 1 time
Been thanked: 279 times

Re: SMF to one line for each event

Postby frogerc » Wed Jun 09, 2010 12:03 am

Thanks enrico-sorichetti I'll try to explain myself.

When You look at an SMF file, what you see are events, but the SMF uses 4, 5 or more lines for each event. Something like this:

1 010.144 00:01:10 PROD STCRESPC STCGROUP 0 2 0 JOBID=(RESPCATU 10.144 00:00:00),USERDATA=(),OWNER=SITOAMP
USER BK CATALOGOS AUTH=(OPERATIONS),REASON=(SPECIAL/OPERATIONS)
SESSION=STARTED PROCEDURE,TOKEN STATUS=(
CREATED BY PRE 1.9 RACF CALL)
DATASET=TELMEX.CATALOG.SYSIN,GENPROF=TELMEX.CATALOG.**,VOLUME=SOPTE0,
LEVEL=00,INTENT

Instead... I am trying to convert this to only one line for each event, like in Excel.

1 010.144 00:01:10 PROD STCPROC STCGROUP JOBID=(RESPCATU USER BK CATALOGOS AUTH=(OPERATIONS),REASON=
2 010.144 00:01:11 PROD STCRESPC STCGROUP JOBID=(RESPCATU USER BK CATALOGOS AUTH=(OPERATIONS),REASON=
3 010.144 00:01:13 DES STCERSC STCGROUP JOBID=(RESPCATU USER BK CATALOGOS AUTH=(OPERATIONS),REASON=

Hope someone can help.
frogerc
 
Posts: 6
Joined: Wed May 12, 2010 7:47 pm
Has thanked: 0 time
Been thanked: 0 time

Re: SMF to one line for each event

Postby dick scherrer » Wed Jun 09, 2010 12:14 am

Hello,

That looks more like "syslog" entries than smf data. . .

Most of the "raw" smf data is not displayable. . .
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6268
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 93 times

Re: SMF to one line for each event

Postby Bill Dennis » Wed Jun 09, 2010 12:27 am

There is a RACF utility (exit?) that can be used during SMF dumps to process records from RACF. Perhaps this a report out of that utility?
Regards,

Bill Dennis

Disclaimer: My comments on this forum are my own and do not represent the opinions or suggestions of any other person or business entity.
Bill Dennis
 
Posts: 278
Joined: Thu May 15, 2008 9:45 pm
Has thanked: 0 time
Been thanked: 0 time

Re: SMF to one line for each event

Postby Joe Babcock » Wed Jun 09, 2010 9:05 am

Let's start over. What record or records are you trying to look at. If you have RACF you will probably want the 80,81 and 83 records. If you're using ACF2 you will use a record greater than 127 for that record type. For example we use TYPE 229. SMF is not event records per say. You are correct in saying they have several event per record, and you may find there are different types of data in the same record if you are trying to relate them to a relational model for a DB.

MXG is closest to what you are trying to do, using SAS to create tables of the records that are stored as SAS files. There are lots of utilities to use against SMF data, but you need to be more spacific in what your objective is before you can get much help with the details of how to get there.

By the way I parse about .75 TerraBytes of SMF per day for a bank. We have both ACF2 and RACF in multiple SYSPLEX's. :)
Joe Babcock
 
Posts: 2
Joined: Wed Jun 09, 2010 8:16 am
Has thanked: 0 time
Been thanked: 0 time

Re: SMF to one line for each event

Postby frogerc » Thu Jun 10, 2010 5:27 am

Thanks to all for the answers.

Dick Scherrer: It is a real SMF register.. it is 8 Mb and it is only of 2 days.
Bill Dennis: I'm not sure what you meen with an EXIT? but I asked IBM people in México and as Robert Sambple sugested... they told me that I can use MICS or MXG.
The problem is that their expensive and I'm not sure we are going to use it that much.
Joe Babcock: It is an SMF 30 register.

I'm trying to find out the users that are trying to do things they are not permited... I meen people who are trying to UPDATE when they can only READ.
And maybe the users that just READ and for some reason they have permission to ALTER.

tHANKS AGAIN.
frogerc
 
Posts: 6
Joined: Wed May 12, 2010 7:47 pm
Has thanked: 0 time
Been thanked: 0 time

Re: SMF to one line for each event

Postby Robert Sample » Thu Jun 10, 2010 7:17 am

8 MB in 2 days? Either your shop is so small as to be almost non-existent, or someone is extracting the data for you already. Typically SMF generates hundreds of megabytes to gigabytes per day.

Why are you even considering SMF? Wouldn't it make more sense to get a RACF (assuming that RACF is your security package) report giving what you want?
Robert Sample
Global moderator
 
Posts: 3719
Joined: Sat Dec 19, 2009 8:32 pm
Location: Dubuque, Iowa, USA
Has thanked: 1 time
Been thanked: 279 times

Re: SMF to one line for each event

Postby dick scherrer » Thu Jun 10, 2010 7:45 am

Hello,

I'm analyzing the SMF, but as you all know, the format is multiline.
Not really. . . What you are trying to do is use some data that is no longer in SMF format. Even if you had something that would do what you want with the raw SMF data, it would not work for the "file" that was posted.

And maybe the users that just READ and for some reason they have permission to ALTER.
There are people who only do an update every year or 2. . . They read all of the time, but only update for some extroardinary process. Suggest someone review everyone who has "alter" permission - there should be very few for production datasets.

Sounds like you would be better served using other than SMF for what you want.
Hope this helps,
d.sch.
User avatar
dick scherrer
Global moderator
 
Posts: 6268
Joined: Sat Jun 09, 2007 8:58 am
Has thanked: 3 times
Been thanked: 93 times

Next

Return to Mainframe Security

 


  • Related topics
    Replies
    Views
    Last post