xmlgraphics-batik-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Justin Couch <jus...@vlc.com.au>
Subject Re: GVTBuilder error with custom DOMs
Date Sun, 17 Mar 2002 10:08:33 GMT
Chris Lilley wrote:
> JC> the W3C folks they keep telling us "X3D is trying to solve the same 
> JC> problems as SVG - SVG does it, why can't you".
> 
> I would appreciate a reference to who said that, and when.

Persistent theme since at least mid last year. I was in one halway 
meeting at Siggraph last year that involved 7 or 8 people, I don't 
remember names now. Since then, the Web3d people that also go to the W3C 
meetings have consistently reported this. The main two that I know from 
the web3d side are Don Brutzman and Joe Williams. In addition, my 
business partner was at the Web3d conference in late feb and had a very 
long conversation with someone. IIRC it was Evan Ireland, but I'll have 
to check on that.

> Grin. Here is a 2D system, addition of a third dimension is left as an
> exercise to the reader..... its not really like that, is it?

Pretty much. If you boil it down, there really isn't much different 
between X3D and SVG. There is a scenegraph of transformations, of 
geometry objects and material information about that geometry and of 
user input sensors. In addition there is also a scripting system and a 
runtime system - don't get these confused they are not one and the same 
thing. SVG happens to use an external spec for defining runtime 
semantics (SMIL) where X3D includes it as part of the core 
specification. It still does the same sort of thing though - makes 
dynamic changes to the properties defined in the original text-based 
file format. X3D scripting is a bit more explicit than SVG, but in 
rough, hand-waving terms, they are the same.

> JC> and a bunch of css stuff. Admittedly
> JC> X3D doesn't have CSS, but a CSS document is no different to an image, 
> JC> video, audio file, or in our case - external prototype files.
> 
> That is not a very close analogy. Since CSS does tree decoration, an
> implementation has to either maintain two, synchronized trees - one
> the raw parsed xml, one a copy of that with some extra information -
> or, more reasonably, maintain a single data structure that can be used
> to satisfy API calls for both the XML DOM and the CSS OM 9and so forth
> for the SVG-specific methods).

In the way we do it with X3D, there is the external world view - the DOM 
, then there are the modifiers and externally defined content (we have a 
system for templating new "element" types called PROTO/EXTERNPROTO) and 
then there is the rendering engine. The rendering engine maintains its 
own state information. How much information is communicated to "the 
outside world" is dependent on the user of that information.

So we sort of do the second option - we have an DOM and an internal 
structure. To us, CSS is just another external input to the system, 
another structure that the core rendering engine uses during the 
rendering cycle. It looks for changes in the CSS and applies that during 
the next cycle, just like any other change that would come from the DOM. 
For example, we have both DOM and EAI access to the internal scenegraph. 
  EAI is a high-level Java API that has nothing to do with XML. If you 
count VRML97-defined APIs, we have four different APIs all wanting to 
access and modify the runtime structure - potentially simultaneously. 
That's why I say that to us CSS is just another API view into the core.

If we now sit down and compare our spec/system to the Batik core, you 
will notice that Batik is already 50% of the way there. GVT is the 
rendering core, SVG DOM is the external API and CSS is another external 
API, that may or may not be part of the DOM (when described as XML 
Processing Instructions rather than as a LINK). After parsing the CSS, 
it is just another, separate, input to the rendering engine. The core 
listens to events in the external APIs and makes the appropriate changes 
to the visual output. In this way, the DOM has its view, the CSS has its 
own, just as required in an earlier email. To boot, there's a hell of a 
lot less management code needed too.

For SVG, SMIL is just another part of the rendering engine too. It is 
identical to our event model, in that it is responsible for 
synchronizing multiple different content streams over a period of time. 
A SMIL instruction that says "start now" to the SVG is no different to 
an external user interaction telling the system the same thing through a 
UI component. The basic action is "start the system clock now/stop the 
system clock now". How that instruction is initiated is inconsequential 
and the rendering engine must treat it as such. At no time should the 
rendering engine need to know that the start command came from a SMIL 
document, from a mouse click on a menu item or a keyboard accelarator 
keystroke. Start, stop, that's all you need to care about.

> Is that a unidirectional transfer of data? Or are there events such as
> user interface gestures that have to be mapped back from the scene
> graph to the original DOM? In other words, are events collected on
> scenegraph elements and then captured and bubbled on the DOM tree?

It depends on how the events are created and how the original source was 
generated. If the DOM comes from some external source, we just do the 
conversions into our internal scenegraph. Next, we look other features. 
Does the DOM have Events capability? If yes, we set up mappings from the 
DOM to our scene graph - we attach DOM EventListeners to the fragment we 
are working from and pass the information through to the render core 
when we see them. Right now, for the externally generated DOM, we don't 
pass events back out that change within the scenegraph internals. 
There's a bunch of reasons why, mainly inconsequential, so I'll footnote 
those at the end of the email.

When the DOM is generated by us, we still take the separated stand, but 
the DOM is now delivered all of the events. We take a bunch of 
optimisations about when and how events are delivered. We mainly operate 
in a stand-off mode. Our DOM works out whether a listener has been 
registered on a particular DOM node. If there is no listener in the tree 
looking for that event, we never deliver the value to the DOM, hence 
avoiding the huge overheads of the DOM event cascade/bubble 
requirements. In addition, our notification is purely "this has 
changed". There is no value involved. The DOM maintains a dirty flag on 
the field and then only updates the value when someone explicitly asks 
for it. We do this because a lot of the time, our events consist of one 
of two things - TimeSensor events and coordinate changes. For example, 
animating a typical humanoid character involves a single set of 
coordinate changes, where there is 1000+ 3-space coordinates as an 
attribute of a single element (if you throw in texture information, 
you'd triple that number at least). Turning that into an attribute 
string every frame would be a killer for performance.


Now, on to user interface events - Again we act in a two-way mode. We 
look for MouseEvent on the DOM and use that to drive objects and 
changes. In X3D, there is really only one node type that would respond 
to that - Anchor. We have touch and planesensors, but the way the spec 
is worded, basically implies that the events must originate from 
"inside" the scene graph.

In addition to the DOM model, we also have to deal with events generated 
from within the rendering engine. We don't really work with user input 
events on a generic architecture. The reason is that the various 
rendering APIs all define how they send input events to user-land code. 
For example, Java3D would send us AWT events through its behaviour 
system. J3D captures those events by itself, from its rendering surface, 
and will deliver those (potentially) in parallel with whatever the 
external UI might deliver through the DOM interface. However there are 
many other ways of getting user input events into the system. Again, 
using J3D behaviours, input can come from custom input devices which 
come through the behaviours system. In this case, we process the event 
through the rendering engine, make whatever calculations we need to do 
and then deliver the results to the externals of the scene graph.

This process probably sounds a bit odd, so probably needs more 
explaining. Within X3D, we don't deal with user input directly. There is 
no such equivalent to "onClick" attributes. Instead, we have nodes that 
are termed sensors. These are effectively the post-processed output of 
the user input. So, to listen to, and then act on, a click we have a 
TouchSensor node. This does the translation and says "the click happened 
here in 3D space, on these texture coordinates, on this geometry". 
(there are also equivalents of mouse over and mouse up/down). You then 
connect the output of this sensor to a script or another node, using an 
explicit model called ROUTEs. So, for us, having a mouse event come down 
through the DOM would be a very rare event, as it is mostly driven from 
internally detected sources.

> Is that 100 fps with only software, or does that include hardware acceleration?

Hardware accel for the lowest-level rendering. Everything above that is 
userland code. Event model, timing, scripting. At least for us, the 
performance bottleneck is the rendering API. We take advantage of 
multi-cpu machines by keeping the event model and rendering loops in 
separate threads. Only on the most complex of worlds have we noticed the 
  rendering loop waiting for the event loop to finish. On average user 
content, the render is the slow item.

In comparison to the SVG world, there shouldn't be any difference 
really. Most of the lowest-level stuff is still in hardware, or at least 
  down in the operating system-specific APIs (at least from JDK 1.3 
onwards, 1.2 was doing pure-java software rasterisers). If you started 
using VolatileImage as the output source, even most of the high-level 
operations like image transformation and clipping would be done in 
hardware on the video card. Given equivalent complexity of content, I 
would expect that SVG would end up with more software renderer usage 
than Xj3D. (However, we could, nominally, implement SVG with an OpenGL 
API and so have H/w accel for almost all operations).

> JC> happens when someone wants to render SVG content that is a document 
> JC> fragment in a far larger, multi-spec XML file (like, say, we do)? Batik, 
> JC> as a rendering engine, will be stuffed.
> 
> No, but it will need to get sync control over part of the larger,
> multi-namespace DOM and make its local, enriched copy of the SVG
> subtree.

Why do you require sync control? Why can't you just be a good servent of 
the containing application and only update your output when it perceives 
it is a good time?


> JC> You really have to play ball 
> JC> with all the other specs out there. For example, here is a not 
> JC> unreasonable situation - A user creates a web page that contains a math 
> JC> formula, a bunch of text boxes so the user can change variables in that 
> JC> formula, and then a visualisation of that formula - both 2D and 3D.
> 
> This is hardly a new scenario.

Exactly, and currently Batik seems to be brushing this off as something 
that it is not willing to cater for. In doing so it is limiting its 
usefulness as a toolkit.

> JC> handles X3D namespace, here's another for SVG namespace, and here for 
> JC> MathML". Batik is useless in this environment. There is no valid reason 
> JC> why that should be the case.
> 
> There is no particular reason that this *is* the case.

Can you give a reason why? Surely you want your toolkit to be useful as 
widely as possible. If not, I'm wasting my time here and should go find 
another toolkit or write my own.

>>>   - SVG requires implementation of the SVG DOM which has a lot of
>>>     extensions compared to DOM Core. This is not supported by generic
>>>     implementations.
>>
> 
> JC> No it does. It requires a bunch o objects to work with that happen to 
> JC> come from an XML document. Again, see the scripting in X3D. We have the 
> JC> X3D DOM, otherwise known as the SAI, and all our scripting engine works 
> JC> just fine without needing it. We use Rhino.
> 
> That does not seem to address the question, at all.

Can you tell me why it doesn't address it? The statement is "SVG 
requires the core DOM + SVG extension". My response is that SVG requires 
it, but the implementation does not. The specification says "if you want 
to interact with SVG through a set of custom APIs that is specific to 
this content, use this set of extensions to DOM". I can quite happily, 
and perfectly spec compliantly, access and modify the SVG content 
through the DOM Core APIs.

If you read Appendix B, nowhere does it state that support for SVG DOM 
is required for an implementation of an SVG renderer. If you read the 
intro, it says " The SVG DOM is builds upon and is compatible with the 
Document Object Model (DOM) Level 2 Specification". B.6.2 quite 
concisely states that the DOM does not need to support the full DOM2 CSS 
implementation. Nowhere does it say "the only way to interact with SVG 
content is through the SVG DOM API".  The introduction in 1.1 say 
"Sophisticated applications of SVG are possible by use of a supplemental 
scripting language" Note the word "supplemental", which means "optional".

> I would love to hear how you plan to calculate the result of a
> bounding box request without a rendering engine.

Quite simple. The rendering engine does rendering. The bounding box 
calculation is not a function of rendering. It is a pre/post processing 
step.

Sometimes it may be more efficient to calculate the bounding box during 
the rendering process, but the rendering process is not *required* to 
calculate the bounding box. I could quite easily in userland code, 
traverse the DOM scenegraph and calculate the bounding box by hand. Or, 
I could create a null renderer, which does no output rendering, but 
responds to scripting and animations, and it could still calculate 
bounding boxes. There are thousands of webpages out there that can help 
you implement various bounding box calculation algorithms. None of them 
ever consider rendering.

> JC> You need a bunch of classes that implement the SVG
> JC> interfaces to provide a structure. You also need a rendering engine. The 
> JC> two do not need to be combined.
> 
> If they are not combined, then they clearly need a fairly intimate
> level of communication.

No. They are as intimate as the source materail allows. See the section 
above. If intimacy is permitted, make use of it. If given the cold 
shoulder, then live with it and get on with the rendering.

> That seem to me to be saying that in the case of inlined SVG
> fragments, the implementation would be non conformant. I guess that is
> not very interesting to the Batik developers.

Any application, at any one time will not be conformant. The transcoder 
API is not conformant using your rules because it generates a one-time 
image and doesn't take into account user input or scripting. Inlined 
content is no different. It is just part of a bigger structure. You can 
still be entirely conformant, but also working in unison with other 
players on the same page.

You also have to consider that conformance comes on many levels. You can 
be conformant to the static structure, and not support runtime 
information. You can also be conformant to various levels of runtime 
handling - see the profiling work that is going on with SVG 1.1. X3D is 
already a couple of steps ahead of you in that department. The minimal 
profile for a mobile phone is no different to the "no runtime" system 
too (in very general hand-waving terms).

Back to your statement about being non-conformant. If you read the 
Rendering Model of the spec (section 3), nowhere does it state that the 
implementation must control synchronisation of the user input events or 
anything. If you read G.6 or G.7 on Conforming Interpreters, nowhere 
does it say that the implementation must maintain control over its own 
rendering state. All it says is that the output must be correct. How you 
drive the internals of the animation engine to achieve that correctness 
is not specified.  In contrast, it does talk about the SVG being used as 
a document fragment within a larger document.

> That doesn't sound too hard. Just do a deep clone of the SVG subtree,

But that's what I explicitly trying to avoid! I don't want to clone the 
SVG tree. That's yet another piece of my memory you are consuming for 
your own greediness. I want you to use my DOM, and my DOM only. How rude 
of you to expect that we all feel like tossing all this extra memory in 
your directions when we already have a perfectly good copy floating 
around in memory already. My end users will be interacting through my 
DOM, so now I need to propogate not one, but two separte event 
cascades/bubbles everytime something changes. How badly with that stuff 
performance!

[1] Footnote on stuff about why we don't pass events out sometimes. The
Xj3D implementation of XML bindings is still lagging behind the core 
rendering API. This is because the XML DTD has been a somewhat moving 
target. There are some concepts in VRML that are close to impossible to 
express cleanly in XML and so there's been a lot of tossing and turning. 
Because some of these issues are core to the way the XML interacts with 
the rendering engine, we've let that part of the codebase lie dormant 
for a while. In the meantime, we've concentrated on getting the 
rendering engine and eventmodel handling correct. Once the XML side 
settles down, it shouldn't been too hard to then catch up. Effectively, 
anything XML has been put on the backburner, hence we haven't gotten 
around to work with sending internal rendering events back out to the 
DOM when the DOM is user supplied.

--
Justin Couch                         http://www.vlc.com.au/~justin/
Java Architect & Bit Twiddler              http://www.yumetech.com/
Author, Java 3D FAQ Maintainer                  http://www.j3d.org/
-------------------------------------------------------------------
"Humanism is dead. Animals think, feel; so do machines now.
Neither man nor woman is the measure of all things. Every organism
processes data according to its domain, its environment; you, with
all your brains, would be useless in a mouse's universe..."
                                               - Greg Bear, Slant
-------------------------------------------------------------------



---------------------------------------------------------------------
To unsubscribe, e-mail: batik-users-unsubscribe@xml.apache.org
For additional commands, e-mail: batik-users-help@xml.apache.org


Mime
View raw message