Fix the direction of 5000 history messages limit and make a GUI option
Description
Environment
Attachments
Activity
Tim Jentz July 25, 2013 at 8:41 AM
-fixed bug where year could be calculated wrong in case the first day of first week of the year was a day of the last year.
-removed old code to calculate pagination with one week per page.
wroot July 24, 2013 at 5:57 PM
I have fixed the first one. Second one is tougher, but now i know the cause. I think the can mark this ticket as Fixed.
wroot July 24, 2013 at 5:50 PM
I think i've found the problem (not the first time actually). Transcript file is not contiguous in its format. My history files are very old and huge. Some start at 2011.01 and whole xml goes without text wrap and then at some point (new Spark version) format changes, identation is introduced and from that point history is not shown (only in the chat window, last 20 messages). In the file i will attach there is also a corruption of a tag:
<message><to>admin@wroot</to><from>john@wroot/Exodus</from><body>asdas</body><date>2011-05-23 21:55:01.171 EEST</date></message><message><to>admin@wroot</to><from>john@wroot/Exodus</from><body>sdfs</body><date>2011-05-31 23:22:02.453 EEST</date></messag <message>
<to>admin@wroot</to>
<from>john@wroot/Exodus</from>
<body>dffd</body>
<date>2011-06-08 21:30:33.765 EEST</date>
</message>
i see 2011-05-23 message, but i don't see all subsequent ones. I guess this "</messag" breaks the flow. Another file:
<message><to>admin@wroot</to><from>dainius\40is.lt@msn.wroot</from><body>su Naujais metais ir tave :)</body><date>2011-01-01 16:50:21.531 EET</date></message> </messages>root</to>
<from>dainius\40is.lt@msn.wroot</from>
<body>sveikas</body>
<date>2011-10-25 20:53:32.859 EEST</date>
I see 2011-01-01 and of course no next messages. This looks even weirder, some "root" element and no opening <message> tag.
Most of my transcripts are messed up. Probably the only thing for me is to try fix them manually..
Tim Jentz July 24, 2013 at 4:07 PM
Ok I got a little bug, caused by code I introduced before to display the history week by week.
I did some testing with large files, I've generated.
Last test I did was 10.7mb sized file with 46080 messages with characer length around 63 over 4 years.
Displaying 11520 messages with one year per page worked fine. Also did one month per page displaying 960 messages with 48 months.
Displaying all messages on one page took to long.
If this patch is still not working for you, can you send me any example transcripts i can working with?
wroot July 24, 2013 at 1:01 PMEdited
I have just tried to install 597 build on top of my 596 build at work. It shows me only 2007-2011 years with one of my contacts. It seems that this 5000 limit is still in place and you only can see 5000 first messages in history. Maybe you can remove it since we have month per page and loading shouldn't be long.
Also i have tried setting maximumHistory to 50000 in spark.properties, but that doesn't change anything
In a 5000 history messages limit was introduced to speed up History loading (all messages are still there, just not showing). But it shows not the last 5000, but the first ones. So you can't find last messages in your history. Should be reversed.
Currently one may control this by adding maximumHistory=xxx into spark.properties There should be an option in the Preferences to change that value. It can be 5000 by default. This option can be removed when history loading will be improved, but this doesn't look to happen very soon.