VladimirPootin's Mafia Log Parser v5.0: Linux compatible!

For stuff related to KoLmafia, KoLproxy, Ascension Log Visualizer, and greasemonkey scripts. Maybe other stuff too.
kain
Pie of the Sky
Posts: 23
Joined: Wed May 16, 2007 2:30 pm

Post by kain » Tue Sep 16, 2008 8:46 am

many kudos for this incredible script.

Has there been any thought to tallying black cat actions? I can't count the number of stupid swashbuckling pants mine has cost me.

*grrr*

User avatar
QuantumNightmare
AFH
Posts: 1827
Joined: Mon Jan 29, 2007 10:28 pm
Location: Montreal

Post by QuantumNightmare » Sun Sep 21, 2008 1:51 pm

I have yet to see a log from a black cat run, but that could be done. Won't be a very high priority, though.

I've released a new version, which will track navel ring retreats. It post something like this:

Code: Select all

[71-74] Outskirts of The Knob
     +> [74] Got Knob Goblin encryption key
     &> 6 / 8 free retreats

Out of 4 turns spent: 1 was the encryption key, 1 was a combat that I did normally, and 2 were failed retreats.

Note: This might be confused by tattered scraps and divine crackers. It tracks the number of times you run away without using an adventure, while a navel ring is equipped.

User avatar
QuantumNightmare
AFH
Posts: 1827
Joined: Mon Jan 29, 2007 10:28 pm
Location: Montreal

Post by QuantumNightmare » Sun Oct 12, 2008 10:57 am

So far the log parser has been searching for strings that mafia displays on ascension, but mafia has been changing these around quite a bit lately. While I still search for some of the more popular strings that are displayed on ascension, the log parser also checks your turncount and makes educated guesses as to when as ascension happened.

I use string matching on logs that begin on turn 1, and educated guesses on logs that begin on turn 2-99. If this method works, I might eliminate string matching entirely and go to a much safer method of tracking turncount for ascension recognition.

Sample Output:

Code: Select all

===Day 1===

[1-74] Missing Data
[75-78] Guano Junction


EDIT: Mafia changed it's strings again. I've given up, and track ascensions purely by turncount.

StDoodle
Just Plain Old Pie
Posts: 7
Joined: Wed Oct 15, 2008 12:54 am

Post by StDoodle » Thu Oct 30, 2008 12:26 am

Tried using this today, and it looks impressive, but I'm running in to lots of problems. My Black Cat run has an inflated run and is counting the two days so far of my following hc/oxy DB run, the previous run has a long string of html below the Pirates line in the adventures summary, possibly etc. but I'm tired.

Anyway, just let me know if I can help by providing my BC logs or anything. I imagine some of the recent mafia changes may have made things break rather badly. But don't let that stop you! The work you've (both) done is greatly appreciated!

User avatar
Kirkpatrick
Just Plain Old Pie
Posts: 11
Joined: Sat Oct 25, 2008 2:56 am

Post by Kirkpatrick » Fri Oct 31, 2008 9:32 am

I started using this a little while ago, and I wanted to drop a line to thank you for the fantastic script!

StDoodle: If you happen to have an older version, I was having very similar problems with the the parser, because of mafia messing about with its beginning-of-ascension marking, but updating fixed it right up...Old or new, it sounds like it's that kind of issue - not that I can be of any further help :P

User avatar
QuantumNightmare
AFH
Posts: 1827
Joined: Mon Jan 29, 2007 10:28 pm
Location: Montreal

Post by QuantumNightmare » Fri Oct 31, 2008 9:57 am

I've been updating the ascension recognition a few times a week now, as kirkpatrick said I'd download the latest version and see if the problems still occur.

If things are still messing up, you can zip up your problem mafia logs and send them to quantum.afh at gmail dot com.

Puttan Esca
Just Plain Old Pie
Posts: 6
Joined: Thu May 08, 2008 7:53 am

Post by Puttan Esca » Tue Nov 11, 2008 10:35 pm

Hi downloaded newest parser, seems to work wonders...
Thanks Vlad and Quantum!

Got a strange thing in my log for BM though:

Code: Select all

Disembodied Hand : 15 combat turns (1.5%) 


Must be the first 15 combats without a familiar?
Last run was 100% disembodied hand HCO BM lead-in (fastest of its type ever btw!).

While on the topic of logs, I encountered weird semi-rare issues. I would happy to give my raw log to anyone interested in tracking down why the fortune counter was out by 5 turns in either direction twice. Have heard about this happening in BM before.
Edit: I am trying to to work out the semi-rare thing myself, but I thought if there was more than one log where this had happened it might help.
Last edited by Puttan Esca on Tue Nov 11, 2008 11:09 pm, edited 1 time in total.

alacrity
AFH
Posts: 501
Joined: Sun May 27, 2007 6:55 am
Location: Santa Barbara, CA
Contact:

Post by alacrity » Tue Nov 11, 2008 11:09 pm

Puttan Esca wrote:

Code: Select all

Disembodied Hand : 15 combat turns (1.5%) 


Must be the first 15 combats without a familiar?
Last run was 100% disembodied hand HCO BM lead-in (fastest of its type ever btw!).

Indeed, congrats on a fantastic run! I'm definitely looking forward to reading your log. Yeah, it looks like the variable for current familiar was picked up from your pre-ascension turns and then didn't reset like it is supposed to when the parser hit the beginning of your run. A quick fix would be to run the parser over an altered copy of your log files, with the first day's log cut down to just data after you ascended.

Puttan Esca wrote:While on the topic of logs, I encountered weird semi-rare issues. I would happy to give my raw log to anyone interested in tracking down why the fortune counter was out by 5 turns in either direction twice. Have heard about this happening in BM before.

I've done a fair amount of fortune cookie data collection and log parsing, and I'd be happy to take a look at your logs. I'll send you a private message with my email address because I hate spam and this is a public forum. :)

User avatar
lostcalpolydude
Forum Maestro
Posts: 2940
Joined: Wed Apr 23, 2008 6:54 pm

Post by lostcalpolydude » Wed Nov 12, 2008 12:03 am

Puttan Esca wrote:While on the topic of logs, I encountered weird semi-rare issues. I would happy to give my raw log to anyone interested in tracking down why the fortune counter was out by 5 turns in either direction twice. Have heard about this happening in BM before.
Edit: I am trying to to work out the semi-rare thing myself, but I thought if there was more than one log where this had happened it might help.

I would count on it being mafia losing track. I've actually had a case where I ate a cookie, had some numbers show up, and mafia displayed them correctly from the cookie. I immediately used the 'counters' command and they had shifted by 3 when I did absolutely nothing in between eating the cookie and checking mafia's counter.
bmaher: Softcore character
lostcalpolydude: Hardcore character (abandoned)

User avatar
deusnoctum
Statistically likely to return
Posts: 820
Joined: Wed May 07, 2008 4:00 pm

Post by deusnoctum » Wed Nov 12, 2008 7:08 am

Not sure if it's a related issue, but mafia also screwed up the GMoB. I summoned him for what mafia said was the third time, and got the combat.

User avatar
QuantumNightmare
AFH
Posts: 1827
Joined: Mon Jan 29, 2007 10:28 pm
Location: Montreal

Post by QuantumNightmare » Mon Nov 17, 2008 11:49 pm

The idea for this update comes from Deus.

You can now write comments directly into the gCLI, and they will appear in your parsed log. If you type "echo Note: This is my comment", then the parsed log will contain the following text:
[/code]This is my comment[code]
This formats nicely with the [code] tags which are used on the AFH forum. You can search for the [code] tags in the source and replace them with whatever formatting is better for your forum of choice.

A useful command (again thanks to deus) to to type into the gCLI once is:
alias nb => echo Note: %%
After this is typed into the gCLI, you just need to type
nb This is my comment
and it will be included into your parsed log.

Much easier than keeping a seperate text file open to record any comments you have as the run is progressing.

User avatar
NardoLoopa
Market Manipulator
Posts: 1201
Joined: Tue Jun 19, 2007 10:37 pm

Post by NardoLoopa » Tue Nov 18, 2008 6:33 am

I would like to make a small request:

Could you please replace all the:

Code: Select all

chop();
chop();


lines with the following:

Code: Select all

chomp();


The first way chops two characters off of all lines. But lines in Linux are terminated by a single \n, not the MS-DOS \r\n. And so every line ends up missing its last character in Linux. The latter way takes care of this on both systems by only taking off the line-terminator(s) where appropriate for the system.

I have to make this change each time I DL a new version.

Thanks.
Image

Kovacs
AFH
Posts: 116
Joined: Sun May 11, 2008 2:55 pm
Location: UK

Post by Kovacs » Thu Dec 11, 2008 5:44 am

I apologize if this is obvious & on the to-do list already, but: now that leaderboard count stops on breaking the prism, most people will spend some turns/days in aftercore, which messes up the end-of-run statistics. Would it be possible to modify the parser to include only pre-prism turns in that overview?

User avatar
QuantumNightmare
AFH
Posts: 1827
Joined: Mon Jan 29, 2007 10:28 pm
Location: Montreal

Post by QuantumNightmare » Thu Dec 11, 2008 11:49 am

By default, the parser now ignores everything after you fight the NS's third form the first time. It doesn't care if you beat her or not, but assumes you do. If you spend turns after this point, they will be recorded as such:

Code: Select all

[1042] Sorceress Tower: Naughty Sorceress
[1043-3145] Aftercore
If you want the parser to display details about your time spent in aftercore, use the new "-aftercore" option. So run the parser with "./mafialogparser.pl -all -aftercore YourNameHere".

And I don't bite, feel free to suggest more improvements in the future.

Kovacs
AFH
Posts: 116
Joined: Sun May 11, 2008 2:55 pm
Location: UK

Post by Kovacs » Thu Dec 11, 2008 6:07 pm

Cool, thanks QN!

The -aftercore option should help with all those BM runs where killing the NS is just not going to happen the first few (dozen) times.

alacrity
AFH
Posts: 501
Joined: Sun May 27, 2007 6:55 am
Location: Santa Barbara, CA
Contact:

Post by alacrity » Thu Dec 11, 2008 6:16 pm

Kovacs wrote:Cool, thanks QN!

The -aftercore option should help with all those BM runs where killing the NS is just not going to happen the first few (dozen) times.

The only people who face the 3rd form and don't beat her then and there are people who forgot to make a *smurf* so... it seems like a fairly reasonable assumption that in the vast majority of cases seeing the 3rd form is equivalent to ending the run.

Other possible exceptions would be people who don't free the king immediately due to trophy acquisition (GMS?) or whatnot, though I personally would put off killing the NS, if I knew I was going to be gaining more stats before freeing the king anyway, to make the fight even more easier!

Kovacs
AFH
Posts: 116
Joined: Sun May 11, 2008 2:55 pm
Location: UK

Post by Kovacs » Sat Dec 13, 2008 10:13 am

If I download from the link in the first post, I still get version 4.8. Am I doing something wrong?

User avatar
QuantumNightmare
AFH
Posts: 1827
Joined: Mon Jan 29, 2007 10:28 pm
Location: Montreal

Post by QuantumNightmare » Sat Dec 13, 2008 12:59 pm

Oops! I fixed the link.

Kovacs
AFH
Posts: 116
Joined: Sun May 11, 2008 2:55 pm
Location: UK

Post by Kovacs » Fri Dec 19, 2008 7:04 am

QuantumNightmare wrote:A useful command (again thanks to deus) to to type into the gCLI once is:
alias nb => echo Note: %%


I'm going to set this up and try to remember to use it starting from my next run, but was thinking that it would be even more useful for me if the notes I made during the run came out in italics, to differentiate from comments I make when actually posting & analyzing the log. Would the following work, or will it break mafia?

Code: Select all

alias nb => echo Note: [i]%%[/i]


If that won't do it, could the italics code tags be included directly into the parser? (Assuming anybody else even sees this as relevant.)

Also, the parser still seems to be including aftercore turns in the summary, at least for me.

User avatar
QuantumNightmare
AFH
Posts: 1827
Joined: Mon Jan 29, 2007 10:28 pm
Location: Montreal

Post by QuantumNightmare » Fri Dec 19, 2008 9:37 am

That's a good idea.

Code: Select all

alias nb => echo Note: [i]%%[/i]
should work, since mafia will display the HTML tags like normal text, for the forum to use.

What problems are you having with your log? The easiest thing to do is zip the problem ascension and send it to quantum dot afh at gmail dot com. I'll take a look and see what has to be done.

User avatar
KujjieKujjieKoo
Spade Ninja
Posts: 2240
Joined: Mon Apr 30, 2007 8:50 pm
Location: !Chicago
Contact:

Post by KujjieKujjieKoo » Mon Jan 19, 2009 9:59 pm

Request: Within the "end of run" +stats summary, add S.S. Scarf, Ice Sickle, Fireworks, and Spooky Mitre.

Also, how difficult would it be to track stat gains from familiars?

Also Also, putty uses?
Image

User avatar
NardoLoopa
Market Manipulator
Posts: 1201
Joined: Tue Jun 19, 2007 10:37 pm

Post by NardoLoopa » Sun Jan 25, 2009 11:01 am

[ninja-ed by newer version not have same issues as V4.5]
Image

User avatar
QuantumNightmare
AFH
Posts: 1827
Joined: Mon Jan 29, 2007 10:28 pm
Location: Montreal

Post by QuantumNightmare » Mon Feb 02, 2009 10:01 am

NardoLoopa wrote:I would like to make a small request:
Sir cheddar sent me a copy of his mafia logs generated under linux, and I realized just how annoying it is to have the last letter cut off every time. While the method you described doesn't quite work, the script should now be compatible with both linux and windows linebreaks. Let me know if there are still problems!

Also, the bottleneck section has been revamped. Let me know if there is anything else you want to see there. The current printout includes: red snowcone summons, day 1 apple summons, sewering, louvre stats, navel ring escapes, turns to lab key. What are the things that can really screw over a run?

User avatar
QuantumNightmare
AFH
Posts: 1827
Joined: Mon Jan 29, 2007 10:28 pm
Location: Montreal

Post by QuantumNightmare » Thu Feb 05, 2009 4:05 pm

Is there any point in keeping this perl script updated, now that Flolle has decided to copy it?

User avatar
lostcalpolydude
Forum Maestro
Posts: 2940
Joined: Wed Apr 23, 2008 6:54 pm

Post by lostcalpolydude » Thu Feb 05, 2009 4:32 pm

QuantumNightmare wrote:Is there any point in keeping this perl script updated, now that Flolle has decided to copy it?

That depends whether Flolle plans on keeping what he has up to date, or if he likes you doing the work and integrating it.
bmaher: Softcore character
lostcalpolydude: Hardcore character (abandoned)

User avatar
Flolle
Fie the Pie
Posts: 848
Joined: Mon Apr 02, 2007 10:13 am

Post by Flolle » Fri Feb 06, 2009 8:46 am

lostcalpolydude wrote:That depends whether Flolle plans on keeping what he has up to date, or if he likes you doing the work and integrating it.

I have every intention of keeping the Ascension Log Visualizer up to date. Not that it is that hard to do really, log formats (be it mafia session log or pre-parsed) don't change that often.


QuantumNightmare wrote:Is there any point in keeping this perl script updated, now that Flolle has decided to copy it?

First to address the actual question:
If you feel that the program could be further developed, if there are people who would like to continue using it, if you think you personally can benefit from continued development and have the will, motivation and time to stay with the project, then you should not end it's development. Other programs that can do similar things as your program don't really go into the equation here.


Now, to address the wording, specifically the bolded part:
I'll simply ask you a couple of question that you can think about: Did Linux copy Windows which in turn copied Mac OS? Did OpenOffice Calc copy Microsoft Excel or did both just copy VisiCalc? Did all First-Person-Shooters copy Wolfenstein 3D or was it Maze War? To what extend exactly did KoL copy NetHack? Is KolMafia a copy of KoL CLI?

I think where I'm going here should be clear now. Just because a program can do the same stuff as another program doesn't mean it copied it. Influenced, yes. Attempted to give similiar results such as, say, a certain output format, yes. But outright coping it? I'd be careful with that statement.

Mara Jade
Just Plain Old Pie
Posts: 16
Joined: Fri Dec 26, 2008 10:38 pm
Location: USA
Contact:

Mac User Here

Post by Mara Jade » Mon Mar 23, 2009 11:23 pm

Hey,

I'm a Mac user...KolMafia works on my computer (yay!). I'd like to use the parser, but I'm not very technical and have no idea how to set it up and everything. I would really appreciate if someone could help me out :)

-MJ

User avatar
lostcalpolydude
Forum Maestro
Posts: 2940
Joined: Wed Apr 23, 2008 6:54 pm

Re: Mac User Here

Post by lostcalpolydude » Tue Mar 24, 2009 12:27 am

Mara Jade wrote:Hey,

I'm a Mac user...KolMafia works on my computer (yay!). I'd like to use the parser, but I'm not very technical and have no idea how to set it up and everything. I would really appreciate if someone could help me out :)

-MJ

Put the file into ~/Library/Application Support/KoLmafia/sessions/ . Open up Terminal (in /Applications/Utilities) and type

cd ~/Library/Application Support/KoLmafia/sessions/

then type

perl mafiaLogParser.pl -all Mara_Jade

you can even copy-paste those lines into Terminal if you want. The parser will create a bunch of parsed ascension log files in that folder.
bmaher: Softcore character
lostcalpolydude: Hardcore character (abandoned)

Mara Jade
Just Plain Old Pie
Posts: 16
Joined: Fri Dec 26, 2008 10:38 pm
Location: USA
Contact:

Post by Mara Jade » Tue Mar 24, 2009 8:06 pm

Thank you very much for your help :)

I will try it out when I ascend..and of course, I will post my horrible turncount ascension logs here where y'all can help me out.

Post Reply