cocoon-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathaniel Alfred" <>
Subject RE: too large XSP logicsheet? aka How to cut XML in to pieces and generate from each one independly (using Cocoon not text editor ;) )
Date Tue, 26 Apr 2005 15:19:39 GMT
XSP is the wrong tool here.  Have a look at the SQL transformer.
That should allow you to make a pipeline:
1.) map:generate from XML input
2.) map:transform by stylesheet converting <ENTRY> to SQL transformer markup
3.) map:transform by SQL transformer
4.) map:serialize

Do you really want ot use Cocoon for initial bulk loads of your database?
Why not just send your XML file through a command line Xalan with a stylesheet to generate
SQL in text mode.
The output file you can then send through your database command line interface.

HTH, Alfred.

> -----Original Message-----
> From: news []On Behalf Of Cocoon User
> Sent: Dienstag, 26. April 2005 15:44
> To:
> Subject: too large XSP logicsheet? aka How to cut XML in to 
> pieces and generate from each one independly (using Cocoon 
> not text editor ;) )
> Hi,
> I've got a problem with too large XSP logicsheet (hypothesis).
> Problem origin:
>     I must INSERT INTO a Postgres database a large amount of data from
> XML source.
>     XML source is something about 1MB. Lots of records like:
> <ENTRY start="2005-03-05 05:45" next="2005-03-05 06:15" duration="26"
> language="PL" age_category="NA">
>   <TITLE>Pod słońcem Afryki</TITLE>
>   <INSTALMENT number="22/26" />
>   <DESCRIPTION short="">Serial o przyrodzie Afryki. Autorzy filmu
> koncentrują się na problemach związanych z ochroną        zagrożonych
> gatunków i ekosystemów.</DESCRIPTION>
>   <PRODUCTION country="RPA" />
>   <HIT>no</HIT>
>   <CATEGORY formal="SER" a="DOK">serial dokumentalny</CATEGORY>
>   <FEATURES x="no" k="no" z="no" p="no" s="no" b="no" m="no" d="no" />
> </ENTRY>
> I thought it would work fine with one big XSP generated by XSL
> transformer:
> for each <ENTRY> in source XML target XSP gets:
> <esql:connection>
>     <esql:pool>geozeta</esql:pool>
>       <xsp:logic>
>         cal.set(<xsl:value-of select="$y"/>,<xsl:value-of select="$m -
> 1"/>,<xsl:value-of select="$d"/>);
>         dayofweek = cal.get(Calendar.DAY_OF_WEEK);
>       </xsp:logic>
>       <result>
>       <esql:execute-query>
>         <esql:query>
>           INSERT INTO programs ( Channel, Title, Chapter, MaxChapters,
> Description, ProductionCountry, Category, Date, DayOfWeek, StartTime,
> EndTime)
>           VALUES ('<xsl:value-of select="parent::*/@name"/>'
>           ,<esql:parameter><xsl:value-of
> select="normalize-space(XSLTExtension:escapeDoubleQuotes(TITLE
> ))"/></esq
> l:parameter>
>           ,0<xsl:value-of
> select="substring-before(INSTALMENT/@number,'/')"/>
>           ,0<xsl:value-of
> select="substring-after(INSTALMENT/@number,'/')"/>
>           ,<esql:parameter><xsl:value-of
> select="normalize-space(XSLTExtension:escapeDoubleQuotes(DESCR
> IPTION))"/
> ></esql:parameter>
>           ,'<xsl:value-of select="PRODUCTION/@country"/>'
>           ,'<xsl:value-of select="CATEGORY"/>','<xsl:value-of
> select="substring-before(@start,' ')"/>'
>           ,<xsp:expr>dayofweek</xsp:expr>
>           ,'<xsl:value-of select="substring-after(@start,'
> ')"/>','<xsl:value-of select="substring-after(@next,' ')"/>');
>         </esql:query>
>         <esql:results>Dodano do bazy</esql:results>
>         <esql:no-results>Nie dodano do bazy</esql:no-results>
>         <esql:error-results>Error typu Błont</esql:error-results>
>       </esql:execute-query>
>       </result>
> </esql:connection>
> (Each INSERT has its own connection in case of possible connection
> timeout postgres feature)
> It works fine for several <ENTRY> tags but when I try to run pipeline
> for whole XML (really big number of <ENTRY>) it chokes for several
> minutes then browser returns err. 500. Nothing is inserted into DB.
> My guess is: too large XSP. (XML structure and content defects would
> return different errors probably)
> So I'm looking for some way to iteratively cut source XML into pieces,
> transform into smaller XSP's and generate from each indepedently and
> sequentially. But I can't find a way how to do it in Cocoon. 
> (maybe flow
> control could be of some use).
> Any suggetions, maybe some smarter way how to insert a lot of 
> data from
> XML to DB, or any hint about a possible trival mistake i might have
> done, would be appreciated.
> Thanx and sorry for my clumsiness in en.
> -- 
> Marcel Gołuński
>   Netventure,
>   ul. Kozłowskiej 1, 00-710 Warszawa
>   tel.: (0-22) 6519430, fax: (0-22) 6518604
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:
This message is for the named person's use only. It may contain confidential, proprietary
or legally privileged information. No confidentiality or privilege is waived or lost by any
mistransmission. If you receive this message in error, please notify the sender urgently and
then immediately delete the message and any copies of it from your system. Please also immediately
destroy any hardcopies of the message. You must not, directly or indirectly, use, disclose,
distribute, print, or copy any part of this message if you are not the intended recipient.
The sender's company reserves the right to monitor all e-mail communications through their
networks. Any views expressed in this message are those of the individual sender, except where
the message states otherwise and the sender is authorised to state them to be the views of
the sender's company.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message