[SPARK-1363] decrease ChatTranscriptPlugin loading time and Ram Created: 09/Jun/11  Updated: 20/May/21

Status: Open
Project: Spark
Components: None
Affects versions: None
Fix versions: None

Type: Improvement Priority: Critical
Reporter: Wolf P. Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: Text File MM_SPARK-1363-2011-07-16.txt     Text File MM_SPARK-1363-2012-01-01.patch    

 Description   

need to speedup history loading
and remove the 5000message limitation



 Comments   
Comment by Walter Ebeling [ 20/Jun/11 ]

no fix available for 2.6.1

Comment by Mike McMahon [ 16/Jul/11 ]

I found a marked increase if we switch to an InputStream / BufferedInputStream vs. a File when reading in the transcripts. The read time went from 500ms average to about 100ms average, this is just on loading a large chat file, not displaying. Still need to find a way to speed up displaying of large chat logs

Comment by Mike McMahon [ 16/Jul/11 ]

Also found that by calling window.setPage vs. window.setText we save (on my system) roughly 5-6 seconds on average. The only caveat is that to use setPage we need to take the chat log that has been parsed to HTML and save it as an HTML file somewhere and reload it in as a URL and pass that url into the window.setPage method.

Here are my timings from two runs (the first being with setPage and the second with setText)
Getting transcript stream forwroot@igniterealtime.org
Start xml parsing - 1310793455403
Finish xml parsing - 1310793456032
Parsing Start - 1310793456109
Parsing finished - 1310793456452
Setting the text - 1310793456452
Running a replace on the builder... - 1310793456508
Ending - 1310793456509
1106

Getting transcript stream forwroot@igniterealtime.org
Start xml parsing - 1310793626502
Finish xml parsing - 1310793627004
Parsing Start - 1310793627121
Parsing finished - 1310793627427
Setting the text - 1310793627428
Running a replace on the builder... - 1310793634475
Ending - 1310793634475
7973

As you can see it is a lot of time saved! The major diff. being that setPage calls updates/draws once the entire file has been loaded whereas setText redraws the handle after each line (which is why very large files can take quite a bit of time to load).

You can definitely feel the setPage drawing though - it may take about 1 second to finish loading the large file (in my case it's roughly 3MB) but it takes an additional one or two seconds to finish the update/draw method on that much text. I'm certain there are more ways to speed this up though.

Comment by wroot [ 16/Jul/11 ]

Any patch to test that? Also, maybe this is related to SPARK-1404 Open ?

Comment by Mike McMahon [ 16/Jul/11 ]

sure, it's a very messy patch so let me get it together, definitely not what i would want to see in the source

Comment by Mike McMahon [ 16/Jul/11 ]

Pretty messy, not what i would put in the source.

Basically we can speed up the portion of loading/dumping into the JEditorPane to under a second (on certain systems) but it's the drawing on the JEditorPane that's killing us. You can (in the code) change the window.setText() / window.setPage() (line 473/474 in ChatTranscriptPlugin) to see how it affects the performances (on the console output).

It becomes painfully apparent where our slowness is when you use the setPage() method!

Comment by wroot [ 17/Jul/11 ]

I didn't notice much improvement on huge history loading. It takes same amount of time. Maybe a scrollbar appears faster, but it is still not possible to use it for some time.

Comment by Mike McMahon [ 17/Jul/11 ]

Yeah it's the drawing that's killing us unfortunately. the setPage loads faster but once it calls to draw it takes roughly the same amount of time.

Comment by Mike McMahon [ 02/Jan/12 ]

please test the newest patch, i switch to using invokeLater for loading the text vs. nothing at all.

This seems to cut down on the loading by about half on my system. Though i would love to get some more input from others.

Comment by wroot [ 20/Jan/16 ]

Looks like ChatTranscriptPlugin.java has evolved much since these patches (using TextArea), so can't apply and test them.

Generated at Thu Mar 28 17:31:37 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100248-rev:e207e3a88e19bebfd0fd5834088a20d22d89a0a2.