freemarker-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Woonsan Ko <>
Subject Re: [FM3] Basic treatment of null VS missing
Date Wed, 07 Mar 2018 23:07:16 GMT
On Tue, Mar 6, 2018 at 5:08 AM, Daniel Dekany <> wrote:
> Tuesday, March 6, 2018, 3:59:22 AM, Woonsan Ko wrote:
>> On Sat, Mar 3, 2018 at 11:22 AM, Daniel Dekany <> wrote:
>>> Below I propose a solution for the basic treatment of null VS
>>> "missing" in FM3. Please tell me what you think!
>>> The way FM2 deals with null-s has quite severe limitations (and
>>> glitches). The root of the problems is that null and missing is the
>>> same for the template language. It's not aware of the concept of null;
>>> for example a variable has either non-null value, or the variable is
>>> considered to be nonexistent. So <#assign x = obj.m()> fails when
>>> obj.m() returns null, for the same reason <#assign x = noSuchVariable>
>>> does. (Note that even if we store null into x on the implementation
>>> level, reading x will fall back to higher scopes to find an x there,
>>> instead of returning null.) Another problem is with
>>> macro/function/method arguments; f(noSuchVariable) will not fail if f
>>> provides a default for the parameter (as in <#function f(x=0)>), on
>>> the basis that noSuchVariable is missing, so the parameter was
>>> practically omitted. Not good, as thus we have suppressed an error
>>> that tells us that there's no noSuchVariable. Also, if in f you want
>>> to call a obj.javaMethod(x), where x is the parameter of f, and you
>>> want to allow x to be null, because obj.javaMethod(x) allows that, you
>>> can't. Because, if you don't provide a default, f(m.willReturnNull())
>>> will be an error for using a null/missing value for a required
>>> parameter, and if you do provide a default, then since null is not a
>>> thing in the template language, you can't specify null as the default
>>> value of the parameter.
>>> So, here's my plan for FM3.
>>> The template language will know null, and it's different that from
>>> "missing". I think so far everyone who has used FM2 heavily will
>>> agree.
>>> On the API and implementation level we will have a singleton,
>>> TemplateNullModel.INSTANCE, that stands for things that do exist but
>>> has null value, while a plain null value indicates that the thing is
>>> missing. For example, Environment.getVariable(name),
>>> TempateHashModel.get(key), and TemplateSequenceModel.get(index)
>>> returns null if the subvariable is missing, and
>>> TemplateNullModel.INSTANCE if it's present but stores a null value.
>>> Also in TemplateDirectiveModel and TemplateFunctionModel the
>>> `TemplateModel[] args` argument of execute(...) will contain null if
>>> the argument was omitted on the caller side (like in f() the 1st
>>> parameter is omitted), but TemplateNullModel.INSTANCE if it the
>>> argument was present, but has null value on the template language
>>> level (like in f(null) or f(obj.willReturnNull())).
>>> The above can be confusing, as null on the template language level is
>>> TemplateNullModel.INSTANCE on the API level, and null on the API level
>>> is "missing" on the template language level... So when I write null,
>>> pay attention to if it's null on the template language level, or null
>>> on the API/implementation level.
>> Could it be an option to use either TemplateModel.NULL_VALUE or
>> TemplateModel.MISSING_VALUE, whether in Environment.getVariable(name)
>> or in TemplateCallableModel#...)?
>> It might be more readable if doable.
>> I guess there should be quite a change even with
>> TemplateNullModel.INSTANCE or null in (e.g,
>> #getOptionalStringArgument(...)) as it should be determined how to
>> handle null or missing value there.
> While using TemplateModel.MISSING_VALUE in place of null indeed makes
> the source code and some API-s easier to understand, it has some
> performance implications, so I'm not sure if we want to pay that,
> given that 99% of the users never call or implement API-s that are
> affected by this decision. If you use null to indicate missing value,
> then you can utilize that Java initializes arrays of reference types
> with null-s. Like consider an argument list array for a callables. If
> you use MISSING_VALUE, then for each invocation, you have to fill that
> array with MISSING_VALUE in a loop (or need a logic that sets the
> omitted parameters to MISSING_VALUE at least). That's after Java has
> already, and unavoidably, has initialized the array with null-s for
> you. (You had some directives with lot of optional parameters, so I
> guess you feel the pain.) Also, Map.get naturally returns null for
> missing items, so you don't have to add `if (v == null) v =
> TemplateModel.MISSING_VALUE);` (or worse, actually add MISSING_VALUE-s
> to a Map), which can come handy at some places. Also, `v == null` is,
> maybe, some times, a tiny bit faster than `v ==
> TemplateModel.MISSING_VALUE`, though surely it depends on JIT magic.
> Another thing with TemplateModel.MISSING_VALUE is that it makes a
> TemplateModel out of "missing value", but "missing value" is not a
> value at all. Or at least I don't want it to be a value, like
> `undefined` in JavaScript is. It's an out-of-band signal that you need
> to be able to pass through the API-s where they declare to return or
> consume a TemplateModel. If we use null for it, then it's not a
> TemplateModel, yet the API-s are compatible with it.

I see. Indeed, it seems more efficient to be cautious on null's than
filling in MISSING_VALUE's.

> As of TemplateNullModel.INSTANCE VS TemplateModel.NULL (and not
> TemplateModel.NULL_VALUE I think), depends on if you want null to have
> a type in the template language. Like, in Scala null has class Null.
> In Java null has no type, it's just the common special value for any
> reference type. I think it's nice if you can query the type of a value
> even if the value happens to be null (in which case you can return
> Null). But it can be either way. To prevent misunderstanding, if we
> will have TemplateNullModel, it a final class and only has a single
> instance.

Makes sense. Thanks for the explanation!



>> Regards,
>> Woonsan
>>> On the template language level we should have this:
>>> - null is a keyword. "missing" (aka "undefined") is not a value on the
>>>   template language level, and there's no keyword for it.
>>> - <#assign x = null> is legal. <#assign x = noSuchVariable> is an
>>>   error, as noSuchVariable is "missing", and not null.
>>> - Reading x doesn't fall back to higher variable scopes if x is null,
>>>   only if x is missing.
>>> - javaObject.someMethod() evaluates to null when the Java someMethod
>>>   method returns null in Java. But if someMethod() has void return
>>>   type, javaObject.someMethod() evaluates to "missing" in the
>>>   template. (These two cases were indistinguishable in FM2 templates.)
>>> - bean.noSuchProperty (where you don't have a getNoSuchProperty()
>>>   method in the bean class) is "missing", while bean.nullProperty
>>>   (where you do have a getNullProperty() method in the bean class, but
>>>   it returns null) is null.
>>> - javaMap.noSuchKey is null, similarly as Map.get("noSuchKey") returns
>>>   null in Java. So we don't differentiate the case when the Map
>>>   doesn't contain the key from the case when the Map contains the key
>>>   and the associated value is null. While we could do that using
>>>   Map.contains(key), I believe that would be highly impractical, as
>>>   you rarely put null values into to Map-s to ensure that all possible
>>>   keys are present. Compare that with beans, where the "keys" are
>>>   defined by the bean class statically.
>>> - importNamespace.noSuchVar is "missing". This will make more sense
>>>   after #var/#set was introduced, and so variables always will be
>>>   declared on parse time (not just on runtime), so the set of
>>>   variables in a namespace is fixed.
>>> - [#var x] (future feature) initializes x to null.
>>> It's hard to evaluate the above if you don't known how the
>>> null/missing handler operators
>>> (
>>> will react to "missing". That was touched in
>>> but going into details would derail discussion. But the basic idea is
>>> that exp!default only handles null, not "missing". Why? Let's say you
>>> write ${user.realNaem!'Not provided'}. As you see you expect the
>>> realName to be null, and also you managed to write realNaem instead of
>>> realName (a typo). If exp!default handles "missing", then this mistake
>>> is hidden by it, while if it only handles null then you get an error
>>> (as there's no getRealNaem() method in that bean) as desirable. Now,
>>> there are some rough edges in this idea, like what if you write
>>> user.phoneNumber, and while getPhoneNumber() dies exists in class
>>> EmployeeUser, it doesn't exist in CustomerUser... so in case user in
>>> the data-model is a customer, ${user.phoneNumber!'Not provided'} will
>>> not save you. If that can't be addressed on the data-model level (like
>>> abstract class User is annotated with @ValidPropertyNamesFrom({
>>> EmployeeUser.class, CustomerUser.class })), there's the possibility
>>> that there will be something like exp!!default that handless both null
>>> and "missing", or that the right compromise will be if exp!default
>>> handle both after all. But, I can proceed without deciding this part
>>> right now.
>>> Another interesting aspect is parameter defaults in macro/function.
>>> Clearly, an argument that's "missing" is immediately an error (unlike
>>> in FM2), so defaults are irrelevant in that case. But what will happen
>>> if you have <#function f(x=0)>, and then you do f(null)? The parameter
>>> was not omitted, so will x be 0 or null? I think parameters should
>>> have an attribute that tells if it should accept null-s or not (kind
>>> of like "not null" constraint in databases). If we accept that idea,
>>> then the answer to the question is that if x has a "not null"
>>> constraint then x will be 0, but if x is nullable then x will be null
>>> in the example case (because the default value is not applied). BTW, I
>>> believe the best would be if macro/function parameters by default has
>>> "not null" constraint, and you explicitly indicate if want to allow
>>> null-s (e.g. `<#function f(x{nullable}=0)>`).
>>> --
>>> Thanks,
>>>  Daniel Dekany
> --
> Thanks,
>  Daniel Dekany

View raw message