bot statistics for iknow

[dns_server]

i created this little script to parse my log files and retrieve some simple statistics from them.

the log files are in a text file with the format: day (as a long) space message ie

1128599852564: [PM from person: message]

#!/bin/bash
echo parseing logs
rm test
cat *.txt|grep  "PM from"|awk '{print $4 " "$5 " " $6}'|sed -e 's/\]//g' -e 's/\://g'|tr 'a-z' 'A-Z'>test
echo createing users list, to the file 'users'
rm users
cat test|awk '{print $1}'| sort | uniq -c | sort -nr>users
echo createing a command list to the file 'command'
rm command
cat test|awk '{print $2}'| sort | uniq -c | sort -nr>command
echo createing file 'stats' to store the player with the most requested stats
rm stats
cat test  |awk '{print $2 " " $3}'|grep "STATS"|awk '{print $2}'|sort | uniq -c | sort -nr>stats

echo createing file 'skills' to store the player with the most requested skills
rm skills
cat test  |awk '{print $2 " " $3}'|grep "SKILLS"|awk '{print $2}'|sort | uniq -c | sort -nr>skills

 

here's the top 10 from each file with the links to the full files. There where 75416 pm's to iknow over several months.

 

==> users <==

3631 DEADFACE

1707 ASGNNY

1168 GRYFLET

1080 RAEN

958 YASERHAMEED

910 ECLYPSE

842 MASKA007PL

751 DAMAGEDTMX

740 BERLYSAM

685 JORMANE

 

==> command <==

53889 STATS

6123 SKILLS

2122 PONG

1362 SKILL

1116 PING

1085 HELP

942 ON

898 STAT

728 SEEN

627 JOKER

 

==> stats <==

522 CRUS

374 SHIVAR

303 DONPEDRO

294 BELALI

283 ALCHMASTER

279 MATESS

277 ZIARA

275 WEXY

271 BENHE

270 TOOMASS

 

==> skills <==

46 DONPEDRO

45 KIBORA

34 ENKI

34 MASTERPITER

33 FELO

31 BEP

27 SKILLS

26 ANYONE

24 FRASSE

23 MASA

Edited November 11, 2005 by dns_server

[Placid]

Hahahaeh, nice one DNS. Actually quite interesting to see :-)

 

I especially enjoyed reading the stats bit.

[crus]

OMG lol..

==> stats <==

522 CRUS

what does this means?..

That ppl wanted to see my stats 522 times?..

*faints*

[LadyWolf]

OMG lol..

==> stats <==

522 CRUS

what does this means?..

That ppl wanted to see my stats 522 times?..

*faints*

 

You feel special now dont you crus

 

 

W00t! i made the list

[crus]

I kind of feel wanted .. not so special ))..

[WoodeH]

Hehe.

 

That's clever, nice dns

[Algorn]

if anyone knows how to convert the Unix long time to a human readable date that can be passed thru grep --count, please chime in. i have been wracking my brain over how to do it(i am not a scripting guru), and i would like to be able to parse the stats by month, since the above stats were totals for all logged months from iknow.

 

if you have any questions about what i'm wanting to do, please feel free to drop me a PM here at the forum(i won't be ingame until at least around xmas due to some hardware failure in my linux box )

 

 

<sidenote> some of you may have noticed some delay in iknow's response around 8am GMT (iknow's server is set to GMT time) to aprox 9am GMT. this is due to the server's backup script running at that time and it seems to occupy quite a bit of the processor time. over the next few days or so i will try to tweak the backup script so it doesn't consume so much of the processor's resources<sidenote>

[trollson]

if anyone knows how to convert the Unix long time to a human readable date that can be passed thru grep --count, please chime in. i have been wracking my brain over how to do it(i am not a scripting guru), and i would like to be able to parse the stats by month, since the above stats were totals for all logged months from iknow.

 

On GNU/Linux, something like:

date '+%s' | awk '{ print strftime ("%F-$T",$1) }'

 

"date '+%s'" is just for demostration, and returns the current time as seconds since the epoch (GNU extension to 'date', you'll need to check whether this is the same value you have).

 

The first argument of 'strftime' is a format string, as per the C API strftime (man 3 strftime). You can change this to something to suit your requirements. For example, "%m" should just print out the month number (01-12).

Edited November 14, 2005 by trollson

[Algorn]

On GNU/Linux, something like:

date '+%s' | awk '{ print strftime ("%F-$T",$1) }'

 

"date '+%s'" is just for demostration, and returns the current time as seconds since the epoch (GNU extension to 'date', you'll need to check whether this is the same value you have).

 

The first argument of 'strftime' is a format string, as per the C API strftime (man 3 strftime). You can change this to something to suit your requirements. For example, "%m" should just print out the month number (01-12).

 

 

okay, i don't currently have a linux box running to test this with, however iknow runs on a NetBSD 2.0 server.

 

now the quandry i am having is when i perform the above command as you have it, it returns the current and correct date(with th exception of attaching "-$T" at the end of the date string.

 

now, if i pass one of the long times from one of the logs thru | awk '{ print strftime ("%F-$T",$1) }', all dates return as 1901-12-13-$T (the $T being consistent with previous behavior mentioned and therefore is easily removable once i figure out how to get the long times converted correctly). example:

bash-3.00# echo 1132031808843  | awk '{ print strftime ("%F-$T",$1) }'
1901-12-13-$T

this behavior is also persistent if i

cat $log_name | awk '{ print strftime ("%F-$T",$1) }'

whereby all dates from within the log are returned as the previously mentioned 1901 date.

any suggestions on why this would be?

[trollson]

Your time stamp is in milliseconds since the epoch. The 'strftime' function expects it to be in seconds since the epoch.

 

Try:

echo 1132031808843  | awk '{ print strftime ("%F-%T",$1/1000) }'

 

Typo: The '$T' my previous post should have been a '%T'.

 

If you just want to get the month out, and dont care about the rest of the timestamp, change the format to "%b", or "%B", or "%m". Format strings are as given the man page for 'strftime' (C API, "man 3 strftime" or similar).

[tuXico]

Hi,

 

since I enjoy shell scripts, I dont want you to stop to play with date

 

but i think it would be better to rotate the log (I dont know if logrotate is available on others linux than debian by default...)

so you ll get tarballs of previous month (checkable with zgrep, zcat...) and your script would have less info to parse (only those of the current (or last if you addapt your script with 'z' commands) month.

 

Thanks for iknow btw =)

 

CitizenKane

Edited November 15, 2005 by tuXico

[Algorn]

Hi,

 

since I enjoy shell scripts, I dont want you to stop to play with date

 

but i think it would be better to rotate the log (I dont know if logrotate is available on others linux than debian by default...)

so you ll get tarballs of previous month (checkable with zgrep, zcat...) and your script would have less info to parse (only those of the current (or last if you addapt your script with 'z' commands) month.

 

Thanks for iknow btw =)

 

CitizenKane

 

 

iknow's logs are rotated as part of his inherent functions (ie, dns coded him to rotate his own logs). we're only interested in extracting usage stats from those logs and trying to break it down by monthly usage.

 

trollson, thanx for the help. i believe i can work with the advice you've given me, i just need to find an efficient way to parse the logs and get the monthly counts.

 

<update>

and with the wonderful assisstance provided to me from trollson, i present to you some usage stats for iknow as of 2005-11-16 06:20am GMT (adjust to your timezone):

Number of Occurrences of 'PM from'
83868

Number of Occurrences of 'PM from deadface'
7266
Total Number of PMs per Month(includes PMs from deadface)
43410 oct
22575 sep
13264 nov
Total Number of PMs from deadface per month
3582 sep
 49 oct

 

trollson, again thank you very much for your assisstance. this geek couldn't have done this without your help.

Edited November 16, 2005 by Algorn