Hello. I have written a custom manager and parser and am attempting to log a large amount (around 4000 logs). This is taking more than a minute, and I would like it to be faster. While trying to pinpoint any bottlenecks in my code, I noticed from console logging certain key points in my manager (e.g. every time readLine() is called) and parser (e.g. parseLine()) as well as looking at the GUI's status bar (which will read +101 entries added) that LogMX will process 101 of my logs in the blink of an eye, wait a moment, process 101 more, wait again, ect. This led me to wonder if perhaps the speed issue is out of my control, and has to do with how LogMX handles logs internally? Do you have any insights on this?
Perhaps important as well: I am receiving logs over a socket, and all 4000 are received very quickly. It is the processing of the logs that is taking a while.
Thanks!
Improving Speed?
Moderator: admin
Re: Improving Speed?
For a little bit of context - we are using LogMX to view logs in realtime from our Facebook app. So timing is important. Our current logging solution (that we're trying to move away from) allows us to view the logs in realtime with no noticeable delay.
That solution is SOSMax and uses TCP:
http://www.sos.powerflasher.com/develop ... /download/
That solution is SOSMax and uses TCP:
http://www.sos.powerflasher.com/develop ... /download/
Re: Improving Speed?
Hello,
You are right, in LogMX v5.3.4, all Managers except LocalFileManager use a 2000 ms AutoRefresh delay. So it means that the embedded SocketManager or your user-defined Manager will be triggered every 2000 ms, not faster. Unfortunately, this delay cannot be configured in v5.3.4. The good news is that we have already fixed this in LogMX v5.4.0 while rewriting our SocketManager: There is now 3 configurable settings for LocalManager, SocketManager, and other Managers:
We are also thinking of allowing each user-defined Manager to force a specific delay for itself using a new method on class "LogFileManager". Also, the whole SocketManager has been re-written to improve its performances and to allow multiple TCP/UDP connections in parallel (as you already know ). A synchronized queue is now used to store data coming from the different sockets, and socket readers can freely put some data in the queue without any delay (except for when the queue reaches a -configurable- size, then the socket reader is blocked as long as the queue size is greater than the configured size). This queue is read each X ms, and X is configurable like described above.
LogMX v5.4.0 is still planned for July, there is currently no major delay for its release.
Xavier
You are right, in LogMX v5.3.4, all Managers except LocalFileManager use a 2000 ms AutoRefresh delay. So it means that the embedded SocketManager or your user-defined Manager will be triggered every 2000 ms, not faster. Unfortunately, this delay cannot be configured in v5.3.4. The good news is that we have already fixed this in LogMX v5.4.0 while rewriting our SocketManager: There is now 3 configurable settings for LocalManager, SocketManager, and other Managers:
We are also thinking of allowing each user-defined Manager to force a specific delay for itself using a new method on class "LogFileManager". Also, the whole SocketManager has been re-written to improve its performances and to allow multiple TCP/UDP connections in parallel (as you already know ). A synchronized queue is now used to store data coming from the different sockets, and socket readers can freely put some data in the queue without any delay (except for when the queue reaches a -configurable- size, then the socket reader is blocked as long as the queue size is greater than the configured size). This queue is read each X ms, and X is configurable like described above.
LogMX v5.4.0 is still planned for July, there is currently no major delay for its release.
Xavier
Re: Improving Speed?
Excellent; it's good to know it's not my code and that there will also be an added feature.
Thanks!
Thanks!
Re: Improving Speed?
Thanks Xavier. Two questions:
[*]Since you're exposing the delay as configurable, would it also make sense to expose the number of entries per parse pass to allow?
[*]July is a a big month Can you share when you expect this release to occur?
[*]Since you're exposing the delay as configurable, would it also make sense to expose the number of entries per parse pass to allow?
[*]July is a a big month Can you share when you expect this release to occur?
Re: Improving Speed?
Hello jobemakar,jobemakar wrote:Thanks Xavier. Two questions:
- Since you're exposing the delay as configurable, would it also make sense to expose the number of entries per parse pass to allow?
- July is a a big month Can you share when you expect this release to occur?
- OK, we will add this.
- It is currently planned for July 17th
Xavier
Re: Improving Speed?
Absolutely we'd love to evaluate the pre-release, thanks!
Re: Improving Speed?
All these new options are now implemented in v5.4.0, I've just sent you a Beta version by e-mail