[oops, this went out from the wrong email address the first time I tried to send. Let me try this again, with the address I'm subscribed under]

I'm using a snapshot version of Lazarus, so I have the latest updates to TAchart including tatools.
I only mention it since it may affect the answers to my questions.

There are several things I'd like to do with tachart that I have not yet been able to figure out how to do.

Easy one: how can I change the thickness and color of the reticule?

Is there any way to have two reticules, each one to be positioned by dragging, so I can display the horizontal distance between them?

I'd like to display multiple line series, one on top of the other. Is there a way to adjust their relative vertical positions? Maybe something like add a new vertical axis for each series and shift the axis origin up or down.

Thanks,
Rick

Rick wrote

There are several things I'd like to do with tachart that I have not yet been able to figure out how to do. Easy one: how can I change the thickness and color of the reticule?

Not so easy, unfortunately.
Currently, you can hack in change of thickness by adding ACanvas.Pen.Width assignment after line 876 in TAGraph.pas
It would certainly good to set it as a property of TReticuleTool. However, current reticule design suffers from a kind of schizophrenia, because ReticuleMode and ReticulePos properties still belong to TChart due to compatibility concerns.
It is unfortunately very hard to judge the impact of backwards-incompatible changes in such a relatively rarely used module as TAChart. So let's take you as a representative sample ;-)
Do you feel it is justified to remove these
properties to get multiple reticules and TReticule.Pen property instead?

As for color, there is another issue:
reticule is meant to be drawn on every cursor move and be visible across various complex charts. To facilitate that, reticule is drawn with pmXor mode. So you can not really get a consistent color across the reticule.

Is there any way to have two reticules,

Currently, if there are several TReticuleTools, the last one "wins" due to compatibility issues noted above.

each one to be positioned by dragging, so I can display the horizontal distance between them?

However, what you want is not actually a job for reticules, since they are designed to follow cursor and data points. It is better performed by dragging TConstantLine-s In fact, similar feature request has already come up in the past, so I implemented it now -- see TConstantLineDragTool and toolsdemo at r25751

I'd like to display multiple line series, one on top of the other. ?Is there a way to adjust their relative vertical positions? Maybe something like add a new vertical axis for each series and shift the axis origin up or down.

This can be done, see "Linear transform" page in axis demo. Try changing Offset value of catTCelToFahr transformation. The most serious limitation is that you must calculate the offset (and scale, if needed),
yourself -- TChart can not do this automatically based on series value range. This is the main point left for closing two long-standing tracker issues.

Alexander Klenin wrote

So let's take you as a representative sample ;-) Do you feel it is justified to remove these properties to get multiple reticules and TReticule.Pen property instead?

As I was browsing through the code last night, I was actually surprised that the reticule pen property
was not already made public in TAChart. I'll have the check out the LineDragTool--it sounds like it
will work better for what I want to do anyway. Rather than make so many heavy changes to the reticule,
I wonder if it would be easier and more compatible to make a new tool like LineDrag that can be configured
to snap to point and act like a reticule--or not. Perhaps at some point later the reticule, with all its
hard-coded properties and marriage to mouse movements, could be deprecated.

I don't mind the way the reticule snaps to data points for my current application. I'm working with
discrete samples and any interpretation between data points is meaningless anyway. Even if the
motion is made continuous, I still want to return discrete horizontal-axis values:
(sample1, sample2, sample3...) or (0ms, 10ms, 20ms, 30ms ...), for example. I haven't looked at the
new tool yet, but it would be neat if it had a couple properties to make the motion continuous or discrete
and to make the returned value continuous or discrete. Combined with the pen property, that should make
it able to replace the vertical reticule in functionality while still allowing multiple instances, dragging, pen properties, etc.

If it was possible to have two vertical drag tools and two horizontal drag tools, you could implement
measurement cursors on the chart that acted like oscilloscope cursors. Very useful to me.

I did check that out and it almost works for me. I'm displaying series of the same height, so
I can use a scale of 1 and manually calculated offsets. I just have to come up with a way to add
names for each series on the left of the chart. I think I can do it if I hide every axis but the first one
and then name the visible axis similar to the "customized marks" example. I want something like
CH1 -
CH2 -
CH3 -
where each series is vertically aligned with the label on the left.

I REALY like TAChart and TATools, by the way. Thanks for all the work you've done maintaining and extending it.

-Rick

Alexander Klenin wrote

so I implemented it now -- see TConstantLineDragTool and toolsdemo at r25751

I just checked it out and played with the demo.
WOW! I like :) The tools add-on was a brilliant idea, if these recent tools are any indication.

I tried vertical and horizontal lines, which worked great. Adjusted the zorder of the line series
so it would draw on top of the bar graph.

I am concerned about one thing, though: It looks like the tool will move ANY TConstantLine series
on the chart. I would suggest a property in the drag tool that is a list of movable lines. Perhaps
if empty, the default could be to move all TConstantLine series. What do you think?

Yes, I wanted to ask you the very same question ;-) The reason I did not implement it immediately is that I am not quite sure about the
format of said property and how should it refer to the series. My current thinking is that it should be a simple comma-separated list of series indexes, but that seems somewhat un-Pascalish, so maybe you have a better idea?

BTW, please do not write much code depending on current TConstantLineDragTool interface just yet -- I think I might change it substantially to a more general one.

The important feature of the current reticule implementation is performance. Users have reported charts with thousands of points, which require up to a second
to redraw. I am afraid that redrawing on every mousemove might be unaccebtable for such charts.

I'm too new to Pascal and Lazarus to be of much help here. If there's a way to
pop up a list of TConstantLine types and store the selected indexes into an array,
that would work. Maybe we can find another custom control that does something
similar and already solved this problem.

The other ugly-hack idea I had was to add a new property to TConstantLine. IsDragable or ControlledByDragTool. I'd much rather have the property in the
tool, though.

BTW, please do not write much code depending on current TConstantLineDragTool interface just yet -- I think I might change it substantially to a more general one.

Oops, I was just in the middle of adding some events to it. I was going to send you a patch
but perhaps I'll just make a request if you're going to go in there soon anyway.

I think the tool should have OnStart, OnChangePosition, and OnEnd events (or something similarly named).
OnStart and OnEnd to be called from the MouseDown and MouseUp events, and OnChangePosition to be
called from the MouseMove event.

I hooked into the TChart MouseMove event to calculate the distance between the two lines and write it to
a text box, but I'd rather link it to an event that only occurs when the lines are actually moved.

Ok, I did it -- now the tool is called TDataPointDragTool and is applicable to (almost) any editable series -- see dragdrop demo for an example. It also has gained AffectedSeries property as described above.

This is the way UseReticule property works now, and I agree that it is not the optimal design.

Thanks for your efforts. It is indeed easier for me at this stage to do it myself,
since it is a small, but tricky change.

I have added a group of mouse events to all chart tool descendants in r25799. See toolsdemo for an example usage.

I was able to use axis transformations/linear offset to stack several series vertically as we discussed.
I discovered an issue with area series and linear transformation, however.

I expected area series to shade the area between the horizontal axis and graph. Instead, it appears to
fill the area between the bottom of the chart and the graph. It doesn't respect the linear transformation.
Try making an area series and then shifting it up 10. It'll draw all the way down to -10 on its axis.

Also, I think somebody asked about this before but it would be nice if there was an area series that would
draw an outline but not the vertical lines at each data point. That looks like it'll be a pain in the butt to
implement, but if you ever find yourself needing to kill a little time ;) It would make for prettier stairstep
graphs that don't have the area shaded, IMHO.

-Rick

Well, that's kind-of as-designed, I think. At least it was that way before I started maintaining TAChart, and I suspect it was done intentionally. But you are right that drawing to zero level deserves at least an option. There are some tricky corner cases -- e.g. drawing series with both positive and negative values, but it is doable.

Yes, that is a long standing feature request.

On the whole, the area series needs a serious amount of work. It currently does not support two other important features -- 3d look and rotation.
So if I have time, I will just rewrite area drawing, taking into account all these changes.

Meanwhile, the patches are very welcome ;-).

BTW, did you test events for TDataPointDragTool?
I have implemented a different set of events than
you suggested -- are they enough for your use case?

Alexander Klenin wrote

BTW, did you test events for TDataPointDragTool? I have implemented a different set of events than you suggested -- are they enough for your use case?

The mailing list seems to be running really slow lately, almost as if it's only processing mail in nightly batches.

I did do a quick and basic experiment with the drag tool events and they work great for my purposes =] (and probably everybody's future use cases as well). Thanks!

I really needed a stairstep series, and I found it much easier to implement in the line series rather than the area series. (The bar graph series also does not respect linear offset transformation, by the way.) I'll paste a unified diff so you can see what I did to the line series. It's really quite basic and not particularly elegant, but it works great. I didn't add reverse stairs since I don't need them, but it's only another three lines of code.

I'm having a little bit of trouble with my axis labels. TAChart seems to only want to display even numbers on the horizontal axis, for example. Even when I use a custom source and set it to X values, it only shows the even numbers. What am I missing?

Another thing I noticed is that performance is pretty bad with 32 line series. I'm not sure what the bottleneck is yet. I'm guessing that it's floating-point math and thinking about trying a custom integer-based series. Any thoughts on that?

Well here's my version of a line-based stair series:

VC: In print_context_label
--- \LazarusSVN\components\tachart\taseries.pas Mon May 31 06:37:02 2010 VC: In print_context_label
+++ \lazarus2\components\tachart\taseries.pas Wed Jun 02 23:38:06 2010 @@ -171,6 +171,7 @@

     FOnDrawPointer: TSeriesPointerDrawEvent;
     FPointer: TSeriesPointer;
     FShowPoints: Boolean;

+ FStairs: Boolean;

     function GetShowLines: Boolean;
     procedure SetLinePen(AValue: TPen);
@@ -179,6 +180,7 @@
     procedure SetSeriesColor(AValue: TColor);
     procedure SetShowLines(Value: Boolean);
     procedure SetShowPoints(Value: Boolean);
+    procedure SetStairs(Value: Boolean);
   protected
     procedure AfterAdd; override;
     procedure GetLegendItems(AItems: TChartLegendItems); override;
@@ -208,6 +210,7 @@
     property ShowPoints: Boolean
       read FShowPoints write SetShowPoints default false;
     property Source;
+    property Stairs: Boolean read FStairs write SetStairs default false;
     property UseReticule default true;
   end;
@@ -379,6 +382,28 @@
     ai := ParentChart.GraphToImage(AA);
     bi := ParentChart.GraphToImage(AB);
     ACanvas.Pen.Assign(LinePen);
+    if Stairs then
+    begin
+      if bi.Y = ai.Y then
+      begin
+        ACanvas.Line(ai, bi);
+      end
+      else
+      begin
+        if ai.Y < bi.Y then  // Step down
+        begin
+          ACanvas.Line(bi, point(bi.X, ai.Y));
+          ACanvas.Line(point(bi.X, ai.Y), ai);
+        end
+        else                 // Step up
+        begin
+          ACanvas.Line(ai, point(bi.X, ai.Y));
+          ACanvas.Line(point(bi.X, ai.Y), bi);
+        end;
+      end;
+    end
+    else
+    begin
     if Depth = 0 then
       ACanvas.Line(ai, bi)
     else begin
@@ -387,6 +412,7 @@
       ACanvas.Pen.Color := clBlack;
       DrawLineDepth(ACanvas, ai, bi, Depth);
     end;
+   end;
   end;
 var
@@ -474,6 +500,12 @@
 procedure TLineSeries.SetShowPoints(Value: Boolean);
 begin
   FShowPoints := Value;
+  UpdateParentChart;
+end;
+
+procedure TLineSeries.SetStairs(Value: Boolean);
+begin
+  FStairs := Value;
   UpdateParentChart;
 end;

Rick wrote

The mailing list seems to be running really slow lately, almost as if it's only processing mail in nightly batches.

Are you sure it is not a problem on your side?
I do not seem to notice a large delays for mails in other threads.

I really needed a stairstep series, and I found it much easier to implement in the line series rather than the area series.

Of course, if you only need lines, then TLineSeries is the way to go.

(The bar graph series also does not respect linear offset transformation, by the way.)

Now that is an outright bug -- fixed in r25885

?I'll paste a unified diff so you can see what I did to the line series. ?It's really quite basic and not particularly elegant, but it works great. ?I didn't add reverse stairs since I don't need them, but it's only another three lines of code.

Thanks for the patch.
One problem with your code is that it does not take into account other properties -- for example, it will not work for 3D-look series. Also, TLineSeries already has a LineType property, which can be used to encode stepped in addition to existing values. Finally, there are two kinds of steps possible -- "X, then Y" and "Y, then X". Stepped series were on my TODO list anyway, so I implemented them in r25889. Also, I added a new linedemo to simplify testing of various property combinations.
Please test.

I'm having a little bit of trouble with my axis labels. ?TAChart seems to only want to display even numbers on the horizontal axis, for example. Even when I use a custom source and set it to X values, it only shows the even numbers. ?What am I missing?

I am not sure, seems to work for me.
Can you post a project demonstrating the problem?

Another thing I noticed is that performance is pretty bad with 32 line series. ?I'm not sure what the bottleneck is yet. ?I'm guessing that it's floating-point math and thinking about trying a custom integer-based series. ?Any thoughts on that?

How many points per serie do you have?

Ok, I have added performance testing to linedemo.
Press 'More series' button a few times, then try resizing the window. I have not noticed any degradation for 40 series of 1000 points each. So please, tell what are you doing more precisely or post a demo.

Alexander Klenin wrote

Are you sure it is not a problem on your side? I do not seem to notice a large delays for mails in other threads.

I just sifted through the mail headers of a message I received from the list.
It looks like the message queues on the list server in Austria at the appropriate time,
but it then takes 8 hours before my local mail server sees it. Maybe messages are
getting stuck in the queue because of a bad route or DNS record. All my other mail
is getting through OK.

Of course, but the real reason I sent you the patch was....

We both knew you'd get around to doing a better implementation with just a little bit of prodding
}:-)

I tried out the demo and it works great. It does what I needed and it will do what the next
guy needs too. Good stuff, as usual!

Well since we're talking about it, let's take the line demo, for example. The horizontal axis displays 0, 5, 10, ...
What would I have to do to make it display 0,1,2,3,4,5,6,7,8,9,10... ? I've tried everything I can think of including giving the axis its own source
and changing the values, but it seems that something else is deciding what numbers to actually print.

How many points per serie do you have?

Well this is what seems to make the difference. I have 10,240 points per series.
I tried it with the line demo, as well. Throw a vertical line and drag tool on the chart, then
drag the line around. Then click the button to add a bunch of series and try dragging
the line. It starts to slow even at 1000 points per series, but panning and dragging become
unacceptable at 10,000 points per series.
(I just got your other email and tried resizing the window. The slowdown is barely noticeable
at 1000pts per series, but quite obvious at 10,000 points)

I did implement a user-defined series that manually retrieves the data and draws the lines.
That works much better, but it's a lot wasted/duplicate effort.

It seems like data access might be at the heart of the issue, since the problem scales with the
number of data points in the series. I wonder if a sorted data source that does a binary search
would be the ticket.

With linedemo:
1) Add TListChartSource component
2) Edit its DataPoints property, enter 1,2,...,10 on a separate lines
3) Set Marks.Source for bottom axis to that source
4) Set Marks.Style to smsXValue
Of course, you can use UserDefinedSource instead for better efficiency.

On the one hand, that is ridiculously large dataset -- averaging more than five data points per x-axis pixel (!).
On the other hand, you are not the first user who does this, so I guess there is some reason for such datatsets ;-)

I have added simple time measurement to linedemo and performed some experiments. It turned out that the most expensive call is ACanvas.Line itself. Fortunately, I was able to transform drawing algorithm to split the series into a set of poly-lines, and then draw those. Unfortunately, it cost a lot of code complexity.
The drawing time is reduced by approximately 3x.
You can measure the speedup by comparing with 'Origin' line type, which does not benefit from that optimization.
_Now_ the data access/transformation is indeed
the most expensive operation remaining.
I did another minor optimization to bypass transformation calls in trivial case, which gives another 10-15% speedup.

Further optimization is possible by caching the poly-lines. However, the complexity will raise even further.
So before we go there, please test the current version -- maybe it is already good enough?

OK, that's working :)

Well of course I don't look at the entire zoom in to maybe 10 or 20 points at
a time and use a scroll bar the pan the chart. It's a run of data that I captured from a
logic analyzer. So when I'm zoomed-in, there's an appreciable distance between data points :)

I have added simple time measurement to linedemo and performed some experiments. It turned out that the most expensive call is ACanvas.Line itself.

This seems counter to what I observed. I can take a data set of 100 points and zoom
in so I'm displaying only 10 points and drag a vertical line around just fine. Expand the data
set to 10k points and look at the same 10-point window, but now the line lags and the draw
event seems to be much slower. The calls to ACanvas.Line should be the same in both cases,
shouldn't they? In other words, the time to draw should be unchanged. I mostly cut and pasted
the code from the draw routine for my user-defined series. What was different/what I think
was faster was the mapping between the chart's logical extent and my data set.

So before we go there, please test the current version -- maybe it is already good enough?

I tried the latest version but there's really no change as compared to before. The draw
routine still takes a significant amount of time with 32x10240 series, zoomed in to a window
of just 10 points. I attached a screenshot so you can see. Graphically, this should not be difficult
at all to draw. Most of the lines are even straight.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Clipboard01.png
Type: image/png
Size: 35160 bytes
Desc: not available
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20100604/a36b9d19/attachment-0001.png>

i don't see anything like that, either... i write a message, send it and a minute later (or faster) it appears in my inbox...

I had the problem for about the last three days, but it seems cleared up today.

So we seem to have established that other people have been receiving mail fine,
which rules out a problem with the list server itself. Meanwhile, my server is
receiving mail from elsewhere but no connection attempts from the list server
for 8+ hours at a stretch.

From the headers, it looks like messages destined for me get queued up on the list server but sit there for quite some time before successful delivery.

I think it was a temporary routing problem from the list server to my mail server.
There are quite a few hops along the way and plenty of opportunity for something
out of everyone's control to go awry.

I'll just leave it at that and be relieved that it has cleared up on its own.
Thanks for the feedback
-Rick

Rick wrote

Thanks for the feedback

np :)

2010/6/5 Rick :

Ah, I see now -- this completely changes things.

Of course. I optimized the slowest case -- that is the drawing of the full extent.
It did became much faster.
Zoomed-in case for large zoom factors stayed about the same, because I thought it is already (relatively) fast.

What was different/what I think was faster was the mapping between the chart's logical extent and my data set.

Yes, for the extremely-zoomed-in case your previous comment about using binary search definitely makes sense.
This is yet another long-standing todo item.
[I should publish my todo list somewhere... maybe as feature requests on issue tracker?]
It is hard to implement properly due to many possible combinations of data sources and axis transformations (consider inverted logarithmic axis case, for example).
Still, I think I did it in r25912:25925.
This is a complex code, so I please test *and* review it, if you can. The bugs like this one:
http://svn.freepascal.org/cgi-bin/viewvc.cgi/trunk/components/tachart/tacustomseries.pas?root=lazarus&r1=25923&r2=25922&pathrev=25923 are really easy to miss with just one set of eyeballs ;-)

To turn on the optimization, you must set Sorted property to true on your chart source.
It defaults to false for compatibility -- look at dragdrop demo to see the difference.

Alexander Klenin wrote

This is yet another long-standing todo item.

That was actually on the list? And here I thought I was the only one using TAChart
in such perverse ways...

[I should publish my todo list somewhere... maybe as feature requests on issue tracker?]

That's as good a place as any, although in my experience publishing todo lists never
seems to help with obtaining those coveted "round-tuits." :D

(consider inverted logarithmic axis case, for example).

You know, I found a way to generate a SIGFPE in the axis demo on the logarithm transform page...
First uncheck log scale, then zoom in on some empty space ((-0.5,200), (1.5,1.5)), for example,
and then check log scale again. Poof!

Still, I think I did it in r25912:25925. This is a complex code, so I please test *and* review it, if you can.

You're not kidding--this was non-trivial. I started reviewing the changes but there are many
areas of TAChart I'm not familiar with so it is slow-going. I'll have a more thorough look tomorrow
after I am rested.

I did test the line demo and then test the algorithm in my application with my data set. Huge
performance increase compared to the previous TLineSeries. The difference between my single
user-defined series and 32 line series with axis transformations is now negligible. I can move the vertical
lines and scroll the graph smoothly, so this is sound in principle.

You've really covered a lot of ground this last week. This is really amazing work! I know you're not
really doing this for _me_, but I have to extend my appreciation and thank you. You rock, man :)

I had to cull a lot of usage information from the mailing list archives, demo source, and asking you. Is
there a place on the wiki where I could perhaps contribute some documentation for posterity?

This is the last time it was brought up:
http://article.gmane.org/gmane.comp.ide.lazarus.general/36748 After your request, I have actually implemented the changes outlined in this message.

Also, this post made me include stepped line series in todo list: http://forum.lazarus.freepascal.org/index.php/topic,8568.0.html

> [I should publish my todo list somewhere... maybe as feature requests > on issue tracker?] That's as good a place as any, although in my experience publishing todo lists never seems to help with obtaining those coveted "round-tuits." ? :D

At least it might tell users something about future plans. For now, I published the list on the wiki:
http://wiki.lazarus.freepascal.org/TAChart#Roadmap

Good catch. This is unrelated to the recent changes, but one of the problems to overcome for the full multi-axis support.
Committed a workaround in r25941 -- TChart now zooms to full extent instead of failing.

I had to cull a lot of usage information from the mailing list archives, demo source, and asking you. ?Is there a place on the wiki where I could perhaps contribute some documentation for posterity?

There is actually a partial documentation for tachart in fpdoc directory. Unfortunately, current versions of fpdoc fails to parse TAChart source, so the documentation effort is stalled. Moreover, this is API-level documentation.
A higher-level, "Tutorial" or "HOWTO"-style is certainly welcome. I think wiki would be a good place for that -- you can create a page and link it from "Usage" section of the TAChart page.