freemarker-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christoph Rüger <c.rue...@synesty.com>
Subject Re: Lambda Expressions - filter list without <#list> directive
Date Tue, 02 Jul 2019 12:57:52 GMT
Thanks for the heads up. Very nice. We will run our test suite to see if
those test are still green.

Am Mo., 1. Juli 2019 um 09:30 Uhr schrieb Daniel Dekany <ddekany@freemail.hu
>:

> Since then I have also made a change that ensures that if the lambda
> argument is null (which in FTL is the same as if the variable isn't
> there at all), then it will not fall back to find an identically named
> variable in higher variable scopes. This is important when doing
> things like:
>
>   <#-- filters out null-s -->
>   myList?filter(it -> it??)
>
> because if some day someone adds a variable called "it" to the
> data-model, then suddenly the above won't filter out the null-s.
>
> The same thing was always an issue with #list loop variables as well,
> also with #nested arguments. So I have added a configuration setting
> called "fallbackOnNullLoopVariable", which is by default true
> (unfortunate historical baggage... but we can't break backward
> compatibility). If you set it to false, then this will print "N/A" at
> null list items, rather than "Clashing variable in higher scope":
>
> <#assign it = "Clashing variable in higher scope">
> <#list myList as it>
>   ${it!'N/A'}
> </#list>
>
> These changes are pushed and deployed to the Apache snapshot Maven
> repo in both branches.
>
>
> So, apart from documentation, the local lambda feature is about ready,
> or so I hope. I'm worried of rough edges though, so I think I will add
> lambda support to some more builtins (?seq_contains, ?sort_by), and
> explore some more use cases... If you have your own that you actually
> keep running into, or want to be in the 2.3.29, tell it.
>
>
> Monday, June 24, 2019, 1:59:21 AM, Daniel Dekany wrote:
>
> > Well, I'm not exactly fast nowadays either... Anyway, I have pushed
> > and deployed to the snapshot repo the changes I was talking about
> > recently. That is, ?map or ?filter won't make a sequence out of an
> > enumerable non-sequence (typically an Iterator) anymore. Because, it
> > was the concern that if hugeResultSet is an Iterator because it's
> > huge, then someone might writes:
> >
> >   <#assign transformed = hugeResultSet?map(it -> something(it))>
> >   <#list transformed as it>
> >
> > instead of just
> >
> >   <#list hugeResultSet?map(it -> something(it)) as it>
> >
> > and thus consuming a lot of memory without realizing it. So now if
> > hugeResultSet wasn't already a sequence (List-like), the assignment
> > will be an error, since we can't safely store a lazily transformed
> > collection (lambdas will break), and we can't condense it down to a
> > sequence (List-like thing) automatically either, as that might
> > consumes too much memory. If hugeResultSet was a sequence, then it's
> > not an error, as we assume that keeping all of it in memory is fine,
> > as the original was stored there as well (in practice, most of the
> > times... in principle we can't know).
> >
> > Now if the user feels confident about it, they can still write:
> >
> >   <#assign transformed = hugeResultSet?map(it -> something(it))?sequence>
> >
> > Similarly, hugeResultSet?map(it -> something(it))[index] will be an
> > error, as [index] is for sequences only, and ?map will not change a
> > non-sequence to a sequence anymore. Similarly, if the user feels
> > confident about it, they can write hugeResultSet?map(it ->
> > something(it))?sequence[index].
> >
> > An interesting consequence of these is that ?sequence is now a bit
> > smarter than before. Like if you write myIterator?sequnce[n], it will
> > not fetch the elements into an in-memory sequence, it just skips n
> > elements from myIterators, and returns the nth one. Similarly,
> > myIterator?sequence?size won't store the elements in memory, it just
> > counts them.
> >
> > As an interesting note, these two are also identically efficient:
> >
> >   <#assign seq = hugeResultSet?filter(it -> something(it))?sequence>
> >   <#assign seq = hugeResultSet?sequence?filter(it -> something(it))>
> >
> > In both cases the actual conversion to a sequence (in-memory list)
> > happens only just before assigning the value to seq. Once again,
> > ?sequence now just means "it's OK to treat this as a sequence, however
> > inefficient it is", and not "convert it to sequence right now".
> >
> >
> > Friday, June 7, 2019, 10:38:50 AM, Christoph Rüger wrote:
> >
> >> These optimisations sound great. I will try to run some tests within the
> >> next weeks. A bit busy lately.
> >> Thanks
> >> Christoph
> >>
> >> Am Mi., 29. Mai 2019 um 23:55 Uhr schrieb Daniel Dekany <
> ddekany@apache.org
> >>>:
> >>
> >>> Tuesday, April 2, 2019, 12:10:16 PM, Christoph Rüger wrote:
> >>>
> >>> [snip]
> >>> >> Well, if you fear users jumping on ?filter/?map outside #list for
no
> >>> >> good enough reason, there can be some option to handle that. But
I
> >>> >> don't think restricting the usage to #list is a good compromise
as
> the
> >>> >> default.
> >>> >
> >>> > I agree. Just keep as it is.
> >>> >
> >>> >> >> I'm not sure how efficiently could a configuration setting
catch
> >>> these
> >>> >> >> cases, or if it should be addressed on that level.
> >>> >> >
> >>> >> > Maybe let's postpone configurability discussion a bit until
the
> above
> >>> is
> >>> >> > more clear.
> >>> >>
> >>> >> In the light of the above, I think we can start thinking about
that
> >>> >> now.
> >>> >
> >>> > On that note on configurability: Would it be possible to
> programmatically
> >>> > influence the Collection (Sequence) which is created under the hood?
> >>> > E.g. by specifying a Factory? I ask because we are using something
> like
> >>> > this (
> >>> >
> >>>
> https://dzone.com/articles/a-filebasedcollection-in-java-for-big-collections
> >>> )
> >>> > in other places for large collections. I know it is very specific,
> but
> >>> just
> >>> > wanted to bring it up.
> >>> [snip]
> >>>
> >>> I think a good approach would be to ban the *implicit* collection of
> >>> the result, when the filtered/mapped source is an Iterator, or other
> >>> similar stream-like object that's often used for enumerating a huge
> >>> number of elements. So for example, let's say you have this:
> >>>
> >>>   <#assign xs2 = xs?filter(f)>
> >>>
> >>> If xs is List-like, then this will work. Since the xs List fits into
> >>> the memory (although a List can be backed by disk, that's rather
> >>> rare), hopefully it's not the kind of data amount that can't fit into
> >>> the memory again (as xs2). On the other hand, if xs is an
> >>> Iterator-like object, then the above statement fails, with the hint
> >>> that xs?filter(f)?sequence would work, but might consumes a lot of
> >>> memory.
> >>>
> >>> This is also consistent with how xs[i] works in the existing
> >>> FreeMarker versions. That only works if xs is List-like (an FTL
> >>> sequence). While xs[i] would be trivial to implement even if xs is
> >>> Iterator-like, we don't do that as it's not efficient for a high i,
> >>> and so the template author is probably not meant to do that. If he
> >>> knows what's he doing though, he can write xs?sequence[i]. Yes, that's
> >>> very inefficient if you only use [] once on that sequence, but you see
> >>> the logic. map/filter breaks it, as xs?filter(f)[i] works even if xs
> >>> is an Iterator, because filter/map currently always returns a
> >>> sequence. If xs is Iteartor-like, then I want filter/map to return an
> >>> Iterator-like as well, so then [] will fail on it.
> >>>
> >>> As a side note, I will make ?sequence smarter too, so that
> >>> xs?sequence[i] won't actually build a sequence if xs is Iterator-like.
> >>> It just have to skip the first i elements after all. (The ?sequence is
> >>> still required there. It basically says: "I know what I'm doing, treat
> >>> this as a sequence.")
> >>>
> >>> --
> >>> Thanks,
> >>>  Daniel Dekany
> >>>
> >>>
> >>
> >
>
> --
> Thanks,
>  Daniel Dekany
>
>

-- 
Christoph Rüger, Geschäftsführer
Synesty <https://synesty.com/> - Anbinden und Automatisieren ohne
Programmieren - Automatisierung, Schnittstellen, Datenfeeds

Xing: https://www.xing.com/profile/Christoph_Rueger2
LinkedIn: http://www.linkedin.com/pub/christoph-rueger/a/685/198

-- 
Synesty GmbH
Moritz-von-Rohr-Str. 1a
07745 Jena
Tel.: +49 3641 
5596493Internet: https://synesty.com <https://synesty.com>
Informationen 
zum Datenschutz: https://synesty.com/datenschutz 
<https://synesty.com/datenschutz>

Geschäftsführer: Christoph Rüger

Unternehmenssitz: Jena
Handelsregister B beim Amtsgericht: Jena

Handelsregister-Nummer: HRB 508766
Ust-IdNr.: DE287564982

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message