Ascension Log Visualizer (Contd.)

Phazic · Post by **Phazic** » Mon Feb 16, 2015 7:01 pm

The original thread for ALV can be found here. -lost

Since the first post seems to be dying, I'll make a new one with updates.

Finished enough planned updates to roll out v3.7.3 during the holiday.

New features:
Report DNA hybridization via @> prefix.
Added detection of notes following consumable usage.
Added experimental detection of Standard path.

Updates:
Data files synced to v15436.
Compressing item drops if quantity greater than 2 on a single turn.
LARGE update to list of interesting items.

Interesting items updates:
Added IOTM drops up to Chateau
Added Xi Receiver drops
Updated quests up to NS13 revamp
Removed old quest items
Removed pastamancer thrall items

Bugfixes:
Fixed double printing of notes in certain instances.
Fixed printing of HTML encoded entities (ie. thermos of "whiskey")

Source is at github at: https://github.com/Phazic/kol-alv
Binary is at github at: https://github.com/Phazic/kol-alv/releases/

Here are examples of the changes.

DNA hybridization and Gene Tonic acquisition (I don't have a DNA lab so thanks to Hellno for the sample text!):

[1-6] The Sleazy Back Alley [6,5,18]
@> [1] Hybridizing yourself: Human-Weird Thing
+> [3] Got Gene Tonic: Insect, dirty hobo gloves
+> [4] Got bum cheek
&> [6] Smith 1 lump of Brituminous coal + 1 dirty hobo gloves
+> [6] Got Hand in Glove

Compressed item reporting (I don't really need to see 30 beach bucks listed one by one):

[16-36] Sloppy Seconds Diner [179,205,407]
+> [16] Got bottle of rum
+> [17] Got possessed sugar cube
+> [19] Got Beach Buck, Meat-inflating powder
+> [21] Got Meat-inflating powder
+> [29] Got Beach Buck
+> [30] Got Beach Buck
+> [33] Got Agitated Turkey
+> [35] Got Meat-inflating powder
+> [36] Got Beach Buck x 30, ten-leaf clover, ten-leaf clover

More detailed list of item updates:

For the next update I'll probably look into preliminary support for handling Ed. Some of the things I'd like to do are:

Record when Ed skills are chosen.
Track Ka acquisition (most likely over intervals, instead of turn by turn)
Track Ka spending.

I'd also like to extend the skill choosing support to the older Avatar paths and Picky so that it can record skill and familiar selection.

Automatic detection of Valhalla choices, ie consumables and astral item would be nice too.

I think there's enough information to handle Florist Friar tracking as ALV tracks every encounter including turn-free ones although I'd need to filter out encounters that don't influence last adventure.

Any other low hanging fruit people can think of?

bombar · Post by **bombar** » Tue Feb 24, 2015 5:23 pm

Waiting for the code to get pulled back into Phazic's repo, but until then if anyone is willing to test my changes please have at it and let me know if you find any issues, I've run through most of my past HR/Picky/Standard/Ed runs and can't find anything major

link: http://goo.gl/VYaa7h

Changes:
Add tons of support for Ed (banishing, tracking, disentrgrate, end of run detection)
Support for DNA Kit
Added support for tracking of combat items used, allows for such things as knowing how many flyer attempts you had to perform, or how many times using pirates book
Banishing via spells and combat items should now be tracked
"Important" combat item tracking implemented, right now limited to interesting things in Ed run
Track gaining of new skills
Fixes new terminology for "chew" instead of "use" when filling up your spleen
Changed how free rests at chateau work so you can get an accurate idea of exactly how much exp you got from the chateau, note: treats free rests and rests that cost a turn same way currently

Note: Combat zone reporting has changed a little bit with this one now free zone entrances (First time in highland quests for example, or free NC in crypt, first time in a game dungeon) are logged in the output, this is there because it will be needed for my next update for the florist, so you can accurately see what zone the plants are being planted in.

Final Note: Full support is only found in the File -> Parse Mafia logs option, will be adding it to the visualizer soon.

ungawa · Post by **ungawa** » Tue Feb 24, 2015 7:08 pm

Hey, I just ran 3.7.3 and 3.7.4 on my latest Ed run. The banish/sniff tracking is great, as is the skill tracking. Very nice!

I did notice that the change to track free adventures is throwing off the "ADVENTURES" summary after the turn rundown. For example, it includes "Video Game Level 1: 2" even though my two adventures there were free. It is nice having those show up in turn rundown to make it a bit more transparent, and it's not a big deal since the number of these in any given zone will likely be consistent from one run to the next.

Thanks for keeping this up to date, guys!

Phazic · Post by **Phazic** » Wed Feb 25, 2015 5:08 am

As people are posting more Ed logs, it seems good to get something out that has some support for Ed.

Ascension Log Visualizer v3.7.4 is released!

New features:
UNDYING! (Lots of support for Ed).
Track when important combat items used.
Track all combat items used.
Track banishing via skills and items.
Track learning of new skills.
Track Ka acquisition per interval and daily.

Changes:
Improved free turn tracking.
Improved handling for DNA kit.
Updated mp costs for all skills.

Fixes:
Updated parser to understand "chew" usage for spleen consumables.

Known issues:
Full support is currently only available via File->Parse Mafia Logs pending port of changes to XML logs.

Special thanks to bombar for joining me on the endeavor of keeping ALV up to date and for providing most of these changes!

Source on github: https://github.com/Phazic/kol-alv
Binary from github: https://github.com/Phazic/kol-alv/relea ... -3.7.4.jar

This should provide a lot of support for handling Ed. A few things still need to be added such as, tracking imbue spending.

bombar · Post by **bombar** » Wed Feb 25, 2015 2:40 pm

Some more known issues:
1) Disintegrates, banishes and some other item usage might not end up in the proper section if they are used on the first turn after free turn happened. Example:

Code: Select all

[489] Rest in your bed in the Chateau [0,442,0]
     +> [489] Got ten-leaf clover
     o> Drank 1 Ye Olde Meade (15 adventures gained) [29,22,27]
     *> [489] Started hunting dirty old lihc

Ka earned today: 182

===Day 3===

[489] Rest in your bed in the Chateau [0,552,0]
[489-500] The Defiled Niche [158,356,157]
     +> [493] Got talisman of Renenutet x 7

In this case the Started hunting dirty old lihc got tagged to the first turn 489 found, not the proper 489 in Defiled Niche

2) Free turns that happen on both sides of a day break could get rolled up into the day before or the day after, this could cause nightcaps not to show up (Fixed already but not in the initial release made this morning) (Fixed in 3.7.4.1)

3) Not handling the troll over the bridge in the chasm correctly, it's encounter block the mafia log doesn't conform to either a regular encounter, or even what the ALV knows as broken encounters so its stat gains get wrapped into whatever turn occurred before, even if it was a free turn

4) Also, I'm pretty sure the code pulled from my fork fixed the issues with the xml parser used by the log visualizer, so that known issue should be fixed, but a lot of the new tabs are not supported yet in the visulalizer (combat items, banishments, etc..)(The visualizer should now visualize properly, however you won't see new tabs for new sections, 3.7.4.1)

~~5) If you use a combat item after going to the underworld, that combat item is not being picked up as being used in combat~~ (Fixed in 3.7.4.1)

Example of new log:

Phazic · Post by **Phazic** » Thu Feb 26, 2015 10:55 am

Cut a point release to pick up bombar's bug fixes.

Point release to fix bugs.

Bug fixes:
Continue tracking Ed encounters through Underworld visits.
Correct turn tracking that spans day boundaries.

Binary: https://github.com/Phazic/kol-alv/relea ... .7.4.1.jar

Kalefornya · Post by **Kalefornya** » Thu Mar 05, 2015 7:18 pm

I'm using 3.7.4.1 and it seems to have missed one of my talisman uses--

mafia output:

Code: Select all

[179] The F'c'le
Encounter: curmudgeonly pirate
Round 0: kalefornya wins initiative!
Round 1: kalefornya casts SUMMON LOVE SCARABS!
Round 2: curmudgeonly pirate takes 11 damage.
You gain 4 Mana Points
Round 2: kalefornya casts SUMMON LOVE GNATS!
Round 3: kalefornya uses the talisman of Renenutet!
Round 4: kalefornya casts FIST OF THE MUMMY!
Round 5: curmudgeonly pirate takes 97 damage.
Round 5: Khedjedhotep waves around a censer of mystical incense.
Round 5: kalefornya casts FIST OF THE MUMMY!
Round 6: curmudgeonly pirate takes 97 damage.
Round 6: Khedjedhotep waves around a censer of mystical incense.
Round 6: kalefornya wins the fight!
You gain 58 Meat
You acquire an item: mizzenmast mop
You acquire an item: grumpy old man charrrm
You acquire an item: all-purpose cleaner
You acquire an item: handful of sawdust
You acquire an item: Ka coin (3)
You gain 8 Muscleboundness
You gain 31 Enchantedness
You gain 10 Roguishness

use 1 mizzenmast mop

upeffect.php?efid=1800&qty=1&pwd=eb7ba1c8ec481a0f5745429f20263929&ajax=1&_=1425583036306

cast 1 Purr of the Feline
You acquire an effect: Purr of the Feline (duration: 10 Adventures)

[180] The F'c'le
Encounter: creamy pirate
Round 0: kalefornya wins initiative!
Round 1: kalefornya casts SUMMON LOVE SCARABS!
Round 2: creamy pirate takes 12 damage.
You gain 4 Mana Points
Round 2: kalefornya casts SUMMON LOVE STINKBUG!
Round 3: creamy pirate takes 14 damage.
You lose 89 hit points
Encounter: Like a Bat Into Hell
You gain 41 hit points
Took choice 1023/1: unknown
choice.php?whichchoice=1023&option=1&pwd=eb7ba1c8ec481a0f5745429f20263929

place.php?whichplace=edunder&action=edunder_bodyshop
Encounter: Underworld Body Shop
Took choice 1052/1: unknown
choice.php?pwd&skillid=45&option=1&whichchoice=1052
Encounter: Underworld Body Shop
You learned a new skill: More Elemental Wards
Took choice 1052/2: unknown
choice.php?pwd&option=2&whichchoice=1052

place.php?whichplace=edunder&action=edunder_leave
Encounter: Like a Bat out of Hell
Took choice 1024/1: unknown
choice.php?pwd&whichchoice=1024&option=1
Encounter: creamy pirate
Round 0: kalefornya wins initiative!
Round 1: kalefornya casts SUMMON LOVE SCARABS!
Round 2: creamy pirate takes 9 damage.
You gain 3 Mana Points
Round 2: kalefornya casts SUMMON LOVE STINKBUG!
Round 3: creamy pirate takes 16 damage.
You lose 92 hit points
Encounter: Like a Bat Into Hell
You gain 41 hit points
Took choice 1023/2: unknown
choice.php?whichchoice=1023&option=2&pwd=eb7ba1c8ec481a0f5745429f20263929
Encounter: creamy pirate
Round 0: kalefornya wins initiative!
Round 1: kalefornya casts SUMMON LOVE SCARABS!
Round 2: creamy pirate takes 8 damage.
You gain 3 Mana Points
Round 2: kalefornya casts SUMMON LOVE GNATS!
Round 3: kalefornya uses the talisman of Renenutet!
Round 4: kalefornya casts FIST OF THE MUMMY!
Round 5: creamy pirate takes 97 damage.
Round 5: Khedjedhotep chants some stuff that you don't really understand, but presumably the gods do.
Round 5: kalefornya wins the fight!
You gain 58 Meat
You acquire an item: ball polish
You acquire an item: creamsicle
You acquire an item: cream stout
You acquire an item: Ka coin (3)
You gain 8 Strongness
You gain 30 Enchantedness
You gain 10 Sarcasm

ALV output:

Code: Select all

TRACKED COMBAT ITEMS
----------
149 : talisman of renenutet (P imp)
179 : talisman of renenutet (curmudgeonly pirate)

Code: Select all

COMBAT ITEMS
----------
Used 8 the big book of pirate insults
Used 2 talisman of renenutet
Used 1 cocktail napkin

Seems like it's probably undying-related.

Looks great otherwise though, thanks for this

bombar · Post by **bombar** » Fri Mar 06, 2015 2:42 am

Thanks, will look into it, could you by chance send me the mafia log file to bombarKol@gmail.com

bombar · Post by **bombar** » Fri Mar 06, 2015 9:48 am

And I figured it out, something screwy with the way mafia changed its logging on you caused the issue.

If you look at turn 166 you will see the way I expected the choice to go to the underworld to be logged:

Code: Select all

Round 0: kalefornya loses initiative!
You lose 99 hit points
Encounter: Like a Bat Into Hell
You gain 39 hit points
Took choice 1023/1: unknown
choice.php?pwd&whichchoice=1023&option=1

This is how its always showed up in my log, but for some reason on the turn in question, that last line got changed to:

Code: Select all

choice.php?whichchoice=1023&option=1&pwd=eb7ba1c8ec481a0f5745429f20263929

Unfortunately, the way I coded it up, was I was looking for the exact string 'choice.php?pwd&whichchoice=1023&option=1' to signify that you were going into hell and process the next section special. Even if I had coded up with a starts with, the pwd section got moved from the front to the end so would have still had issues, fixed the issue on my side, sending you a patched jar in an email.

Post by **lostcalpolydude** » Fri Mar 06, 2015 3:59 pm

Mafia shouldn't ever log your password hash, I'll look into that.

Post by **lostcalpolydude** » Fri Mar 20, 2015 11:49 am

Rather than linking to each specific binary, I changed the link in the first post to point to the releases page, which will always have the latest binary. I also stickied this thread and unstickied the old one.

Phazic · Post by **Phazic** » Sat Mar 21, 2015 2:34 pm

Thanks lost!

Just put out a point release for v3.7.4.2.

This one handles effect formatting change introduced in KoLMafia v15554.

Handling of the effects is backwards compatible.

Release notes:
Point release to handle KoLMafia's new effect formats.

Pulled in bombar's fixes for banish handling too.

KoLMafia v15554 use a new format for logging effects.

Old:
You acquire an effect: On the Trail (duration: 40 Adventures)

New (some have 2 spaces):
You acquire an effect: On the Trail [unemployed knob goblin]
You acquire an effect: Everything Looks Yellow
You acquire an effect: Wisdom of Thoth (10)

PsyMar · Post by **PsyMar** » Sun Dec 20, 2015 10:36 am

Cursedness is throwing it off, particularly getting it from the noncombat.
Log of turn causing error:

Code: Select all

[1412] The Hidden Apartment Building
Encounter: Action Elevator
Took choice 780/2: Get Thrice-Cursed
choice.php?pwd&whichchoice=780&option=2
You lose an effect: Twice-Cursed (-10)
You acquire an effect: Thrice-Cursed (0)

Also, it doesn't properly log the new Barrel full of Barrels.

Also, something is throwing off the familiar tracking, as it claims my latest bad moon run (on Koriandr (#1676253)) was 98.8% black cat (1488/1506 combat turns) when in fact it was 100% (as shown in ascension history) -- 14 turns could be attributable to the haunted bedroom if that is throwing things off, but I don't know where the other four were. Maybe barrel full of barrels mimics.

Hellno · Post by **Hellno** » Tue Dec 22, 2015 11:59 pm

There are a couple of new bugs with this program due to I assume changes in Mafia:

Stat-gains are now reported like this in the logs:

Code: Select all

After Battle: You gain 85 Strongness
After Battle: You gain 91 Mysteriousness
After Battle: You gain 176 Roguishness

As opposed to how ALV currently expects it:

Code: Select all

You gain 85 Strongness
You gain 91 Mysteriousness
You gain 176 Roguishness

Also you get a parsing error from lines that for whatever reason display (0) turns of an effect, for example:

Code: Select all

cast 1 Suspicious Gaze
You lose an effect: Wry Smile (-10)
You acquire an effect: Suspicious Gaze (0)

If you want to post a log, you can fix these things manually with find/replace.

Phazic · Post by **Phazic** » Wed Dec 23, 2015 2:20 am

Thanks for the reports!

Found some time during Crimbo break to work on these fixes.

I've put out a point release v3.4.7.3-gb905bce to address these things.

It's been a while since I've worked on ALV so let me know if there are any issues.

https://github.com/Phazic/kol-alv/relea ... 3-gb905bce

https://github.com/Phazic/kol-alv/relea ... 905bce.jar

PsyMar · Post by **PsyMar** » Wed Dec 23, 2015 6:36 am

I just noticed in my recent black cat run it says I spent 0 turns in the Gallery and found 0 Louvres, when in fact I was using the Gallery to powerlevel to 13.

Also, I downloaded the new version and it still doesn't report substat gains from combats.

Phazic · Post by **Phazic** » Wed Dec 23, 2015 3:09 pm

I identified the issue with the summary parser at the end. ALV was checking for area names without articles, ie Haunted Gallery instead of The Haunted Gallery.

I'm not seeing any issue with after-combat stat parsing as I can take a log from 3.7.4.2 and compare it to 3.7.4.3 and see significant differences in stat gains during combats. This fix also seems to affect mp gains and meat gains as well.

Can you tell me which section you're looking at that's not reporting the stat gains?

I've looked over the: stat gains overview, stats per area, stats per turn per level, stat development charts.

I've also looked at the summaries at the end and see something like this:

v3.7.4.2:

Code: Select all

STATS
----------
           	Muscle	Myst	Moxie
Totals:   	2139	5488	2229
Combats:	972	1050	1147
Noncombats:	529	3688	484
Others:   	0	0	0
Eating:   	26	168	26
Drinking:	462	417	422
Using:   	150	165	150

v3.7.4.3:

Code: Select all

STATS
----------
           	Muscle	Myst	Moxie
Totals:   	9392	22162	9471
Combats:	8202	17646	8357
Noncombats:	552	3766	516
Others:   	0	0	0
Eating:   	26	168	26
Drinking:	462	417	422
Using:   	150	165	150

Either that or post a log that I can look at to see the issue you're having?

PsyMar · Post by **PsyMar** » Wed Dec 23, 2015 5:30 pm

Ah, I tried refreshing the logs directory and now it can't find the appropriate ascension at all, so it was apparently using cached data from 3.7.4.2.

Trying to just parse the logs, it says there is an error on turn 557 on Dec. 11. The problem is, I had already started (and maybe completed?) turn 558 on Dec. 10, and I honestly can't find the issue. There are no lines of more than 500 characters, so I will attach the logs for both days.

Or I would, except I can't upload attachments.

EDIT: I have pastebinned the relevant portions of the logs.

Phazic · Post by **Phazic** » Wed Dec 23, 2015 10:09 pm

Thanks for the logs! I was able to trace the issue due to how Mafia handles the Barrel Full of Barrels.

You can see it bleeding into your lihc eye pie consumption on the 11th.

ALV was trying to handle the whole thing as a consumption block and didn't know how to parse the stat gains from the combat that occurred during the BfoB.

I'm not quite sure what the best way to handle this is. I can put up a workaround that has it stop parsing the block if it encounters a choice in the middle of a consumption block.

PsyMar · Post by **PsyMar** » Thu Dec 24, 2015 1:49 am

Phazic wrote:Thanks for the logs! I was able to trace the issue due to how Mafia handles the Barrel Full of Barrels.

You can see it bleeding into your lihc eye pie consumption on the 11th.

ALV was trying to handle the whole thing as a consumption block and didn't know how to parse the stat gains from the combat that occurred during the BfoB.

I'm not quite sure what the best way to handle this is. I can put up a workaround that has it stop parsing the block if it encounters a choice in the middle of a consumption block.

That'd be nice, so I can have correct stat gains on the log I posted of that run.

somersaulter · Post by **somersaulter** » Sat Jan 28, 2017 1:49 am

Necrobump!

An informal list of items that I think would be helpful to track that have been added since the last release in the spoiler, although some of these items could be redundant (some are consumables), and some items could be missing.

I'm... not sure how to go about adding these, sorry!

somersaulter · Post by **somersaulter** » Mon Aug 28, 2017 8:18 pm

And another necrobump!

~~Attached to this post~~ This pastebin is an updated itemsDefault.xml (good for things introduced between Dec 2015-Aug 2017). I'm having trouble making a working .jar that uses this new file (I've never done anything like this before), and I would appreciate some assistance (from anyone) so I can do this on my own later when new items are introduced.

Deatvert · Post by **Deatvert** » Tue Oct 17, 2017 10:55 pm

Were you able to make a working .jar?

Deatvert · Post by **Deatvert** » Thu Oct 26, 2017 12:54 pm

somersaulter wrote: ↑
Mon Aug 28, 2017 8:18 pm
And another necrobump!

~~Attached to this post~~ This pastebin is an updated itemsDefault.xml (good for things introduced between Dec 2015-Aug 2017). I'm having trouble making a working .jar that uses this new file (I've never done anything like this before), and I would appreciate some assistance (from anyone) so I can do this on my own later when new items are introduced.

I ended up being able to open the .jar with WinRAR and then edit the xml file directly inside that way. Seemed to work. My logs are still screwed up from other things, but it will at least mention when newer items were used.

Alliance From Hell

Ascension Log Visualizer (Contd.)

Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)

Re: Ascension Log Visualizer (Contd.)