freemarker-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siegfried Goeschl <siegfried.goes...@gmail.com>
Subject Re: FreeMarker generator release preparations
Date Mon, 10 Aug 2020 20:13:31 GMT
Also got some interesting test errors

https://travis-ci.org/github/apache/freemarker-generator <https://travis-ci.org/github/apache/freemarker-generator>

Currently looking into it

Siegfried

> On 10.08.2020, at 20:53, Siegfried Goeschl <siegfried.goeschl@gmail.com> wrote:
> 
> Hi Daniel,
> 
> Please see my comments below
> 
> Thanks in advance, 
> 
> Siegfried Goeschl
> 
>> On 09.08.2020, at 22:00, Daniel Dekany <daniel.dekany@gmail.com <mailto:daniel.dekany@gmail.com>>
wrote:
>> 
>> See my answers inline...
>> 
>> On Sun, Aug 9, 2020 at 7:16 PM Siegfried Goeschl <
>> siegfried.goeschl@gmail.com <mailto:siegfried.goeschl@gmail.com>> wrote:
>> 
>>> Hi Daniel,
>>> 
>>> I think there is a way to define a custom FreeMarker template path in
>>> IntelliJ - see
>>> https://intellij-support.jetbrains.com/hc/en-us/community/posts/115000686590-How-to-configure-the-freemarker-template-path-or-what-s-the-default-path-
<https://intellij-support.jetbrains.com/hc/en-us/community/posts/115000686590-How-to-configure-the-freemarker-template-path-or-what-s-the-default-path->
>>> 
>>> Having said that we should take care that the IntelliJ plugin is happy
>>> 
>>> * Some users do create web applications based on Apache FreeMarker
>>> * Having FTL support built-in is a huge bonus for beginners
>>> 
>> 
>> What do you mean, realistically what can we do, in case that plugin is
>> indeed that broken? I mean, for FreeMarker Generator alone it doesn't
>> really matter for users. For all the other projects, FreeMarker doesn't
>> prevent them from doing dirty things to accomodate to the plugin (as you
>> saw yourself ;) ), and I'm not sure what else could FreeMarker itself do
>> about this. Fixing/extending the plugin itself is also not likely, as it's
>> closed source. (Of course, it would be great to have an open source LSP
>> language server, and then adding open source plugins based on that... but
>> things go very, very slowly around here.)
> 
> Played around with the IntelliJ plugin but I don't get it perfectly working ...
>> 
>> Regarding the intended role of "/templates"
>>> 
>>> * The implemented template loading tries to resolve the given template
>>> name as file and if there is no file the actual template loading is
>>> delegated to a "MultiTemplateLoader"
>>> * In https://issues.apache.org/jira/browse/FREEMARKER-146 <https://issues.apache.org/jira/browse/FREEMARKER-146>
I split the
>>> existing templates into a reusable part (living now in "templates") and
>>> examples (living now in "examples")
>>> * IMHO the "templates" directory should contain commonly useful templates
>>> - so they could be considered as guaranteed functionality
>>>        * Transforming CSVs
>>>        * Transforming Excel files to CSV, HTML & Markdown (and maybe
>>> Confluence Markup)
>>>        * Transform JSON to YAML and the other way around
>>> * I'm happy to have those templates as files in the installation
>>> directories - better visibility and user can add there own templates (if
>>> the want to play nasty)
>>> 
>> 
>> The fundamental problem with that is this. Currently, if you pull in
>> freemarker-generator-cli as Maven dependency, the templates will not be
>> available. Surely, because it's the CLI, you could say that it's not
>> supposed to be used without the FreeMarker Generator Home Directory created
>> somewhere, which contains the launch script and templates/ and all. But, if
>> these templates are guaranteed functionality in FreeMarker Generator, then
>> they don't strictly belong to the CLI. When we will have a proper Maven
>> plugin for example, they should be still accessible. In that  case you only
>> have your Maven dependencies, so the templates must come from there.
>> 
>> Regarding visibility, it's a bit like with Java. Java classes are not too
>> readable without looking at the source code either. That's not an advantage
>> when it comes to "visibility", sure. But luckily this is open source, and
>> it's very easy to get to the source code, if someone really cares (like
>> from the Maven source artifact). That applies to core stuff implemented in
>> FTL as well. So, the previously mentioned advantage (that they are
>> available from a plain dependency) certainly overweights this disadvantage
>> (less visibility).
>> 
>> I don't think we want users to add their own templates to /templates, as
>> they will be potentially lost when someone updates FreeMarker Generator,
>> and is confusing for colleagues, etc. If we can prevent bad practices,
>> let's do it. Especially as we also provide a cleaner
>> solution, ~/.freemarker-generator/ added to the template root (which will
>> change to ~/.freemarker-generator/templates now, as you say below). If
>> people need a system-wide one, we can also add /etc/freemarker-generator,
>> or something like that.
>> 
> 
> I currently disagree here - I like the visibility aspect and it is pretty difficult to
get rid of templates loaded from the classpath.
> 
>> What I want to change
>>> 
>>> * Use "templates" as FreeMarker template root for the CLI installation and
>>> "~/.freemarker-generator" directory
>>> * Adopt the examples and documentation, e.g.  "freemarker-generator -t
>>> info.ftl" instead of "freemarker-generator -t templates/info.ftl"
>>> 
>> 
>> Except, we should avoid adding info.ftl and such to the template directory
>> root. Not only because of potential name clashes, but because it can be
>> confusing if someone looks at that (where's info.ftl coming from). How
>> about reserving the "freemarker-generator" directory on the top level, and
>> then it's  "freemarker-generator -t freemarker-generator/info.ftl".
>> (Hopefully that helps realizing that it's something built into FreeMarker
>> Generator.) Actually, maybe it should be even
>> "freemarker-generator:/info.ftl ".
> 
> Mhmm, already pushed the changes :-(
> 
> Need to think about it
> 
>> 
>> Thanks!
>> Daniel Dekany
>> 
>> 
>>> What do you think?
>>> 
>>> Siegfried Goeschl
>>> 
>>> 
>>>> On 05.08.2020, at 13:28, Daniel Dekany <daniel.dekany@gmail.com <mailto:daniel.dekany@gmail.com>>
wrote:
>>>> 
>>>> Ah, you have the FreeMarker plugin in IntelliJ, and that assumes
>>> something
>>>> crazy like that? The Community Edition doesn't have that plugin, and I
>>>> never saw it. But if that plugin indeed assumes that the template root is
>>>> the project root... that wouldn't make sense, in any real world project.
>>>> Even if some have a "templates" directory laying around in the directory
>>> as
>>>> the POM, then that would be the template root (means, a template path is
>>>> like "/foo.ftl", not "/templates/foo.ftl"), not the whole project. So, I
>>>> certainly don't think we should align with a broken plugin, especially as
>>>> most users won't be affected (they just use the binary).
>>>> 
>>>> Also, I'm a bit confused about the intended role of "/templates". Is it
>>>> core (guaranteed) functionality in Generator CLI? As opposed to a core
>>>> functionality of Generator itself? (Sure, for now Generator can be called
>>>> via CLI only, but you see what I mean.) Or is it just some examples
>>> laying
>>>> around, whose presence users shouldn't count on in their projects? If
>>> it's
>>>> core functionality, then it certainly should be a Java resource, packed
>>>> into the jar.
>>>> 
>>>> Also let's not skip the question from the earlier mail, regarding what
>>> path
>>>> should this be available for the TemplateLoader (and for -t)?
>>>> 
>>>> On Wed, Aug 5, 2020 at 11:47 AM Siegfried Goeschl <
>>>> siegfried.goeschl@gmail.com <mailto:siegfried.goeschl@gmail.com>>
wrote:
>>>> 
>>>>> Hi Daniel,
>>>>> 
>>>>> When moving the "templates" to a subdirectory IntelliJ complains about
>>> not
>>>>> finding the FTL include files - technically it is working but without
>>>>> tweaking IntelliJ it spits out errors. And the Travis tests failed
>>> while my
>>>>> local tests succeeded - see
>>>>> 
>>> https://travis-ci.org/github/apache/freemarker-generator/builds/714600403 <https://travis-ci.org/github/apache/freemarker-generator/builds/714600403>.
>>>>> 
>>>>> Regarding template root directory - I guess there is some room for
>>>>> improvements
>>>>> 
>>>>> * I'm passing a list of template root directories (current directory,
>>> CLI
>>>>> installation directory, ~/.freemarker-cli/"
>>>>> * The "templates" directory is just the way I organised the overall file
>>>>> structure
>>>>> * Additional template root directory can be passed on the command line
>>>>> * The implementation can pick up any FTL file using absolute or relative
>>>>> file paths (and URLs)
>>>>> 
>>>>> Thanks in advance,
>>>>> 
>>>>> Siegfried Goeschl
>>>>> 
>>>>> 
>>>>>> On 04.08.2020, at 22:12, Daniel Dekany <daniel.dekany@gmail.com
<mailto:daniel.dekany@gmail.com>>
>>> wrote:
>>>>>> 
>>>>>> Can you clarify, what part of IntelliJ is happy about that, and why?
>>>>>> 
>>>>>> Also it reminds me of another thing I wrote about earlier. Generally,
>>> we
>>>>>> say that the template root directory shouldn't not contain anything
but
>>>>>> templates, so if the project root is the template root directory,
or
>>> the
>>>>>> freemarker-generator home directory, that's a bit fishy. Yes, the
>>> reason
>>>>>> for this purist approach is partially security concerns with web
>>>>>> applications, and this is not one. But whatever the original reasons
>>> are,
>>>>>> FreeMarker has this omnipresent concept of the template root directory,
>>>>>> which is kind of a virtual root directory for a template (i.e., they
>>> can
>>>>>> leave it, and it's all templates, or rarely files referred from
>>>>> templates,
>>>>>> like images).
>>>>>> 
>>>>>> Also, it's strange/confusing if the template root directory has a
>>>>>> "templates" subdirectory. I mean, the template root directory is
what
>>> one
>>>>>> would normally call the templates directory. As a template author,
I
>>>>> would
>>>>>> expect some path like "/freemarker-generator/whatever.ftl" (also
note
>>>>> that
>>>>>> it's an absolute path). There,  "/freemarker-generator/" tells me
that
>>>>> this
>>>>>> is something included in freemarker-generator, and not templates
of my
>>>>>> project.
>>>>>> 
>>>>>> On Tue, Aug 4, 2020 at 8:51 PM Siegfried Goeschl <
>>>>>> siegfried.goeschl@gmail.com <mailto:siegfried.goeschl@gmail.com>>
wrote:
>>>>>> 
>>>>>>> I'm not happy about it :-)
>>>>>>> 
>>>>>>> * A top-level "templates" directory makes IntelliJ happy when
includes
>>>>> are
>>>>>>> used
>>>>>>> * I have an ExamplesTest which tests each documented command
line
>>> usage
>>>>>>> and I prefer to have it in sync with the documentation
>>>>>>> * I actually tried it but reverted the changes since the Travis
build
>>>>> fails
>>>>>>> 
>>>>>>> Thanks in advance,
>>>>>>> 
>>>>>>> Siegfried Goeschl
>>>>>>> 
>>>>>>> 
>>>>>>>> On 28.07.2020, at 22:34, Daniel Dekany <daniel.dekany@gmail.com
<mailto:daniel.dekany@gmail.com>>
>>>>> wrote:
>>>>>>>> 
>>>>>>>> It's just that "templates" belongs to the "main" source code
of the
>>>>>>>> freemarker-generator-cli, similarly to
>>>>> config/freemarker-cli.properties,
>>>>>>>> which is under src/main. (I think many of us expect a Maven
project
>>>>> root
>>>>>>>> directory to only contain the "noise" needed for build and
legal, and
>>>>>>> "src"
>>>>>>>> that contains *all* the essential source code.) Also, as
the source
>>>>> tree
>>>>>>>> doesn't mirror the binary distribution directory tree anyway,
so I'm
>>>>> not
>>>>>>>> sure if there's any fundamental reason to make an exception
with
>>>>>>>> "templates", and put it under the root.
>>>>>>>> 
>>>>>>>> On Tue, Jul 28, 2020 at 4:09 PM Siegfried Goeschl <
>>>>>>>> siegfried.goeschl@gmail.com <mailto:siegfried.goeschl@gmail.com>>
wrote:
>>>>>>>> 
>>>>>>>>> Hi folks,
>>>>>>>>> 
>>>>>>>>> Back from the mountains and in front of the keyboard
again :-O
>>>>>>>>> 
>>>>>>>>> * I created a JIRA ticket and will push a feature branch
soon (bad
>>>>>>> habits
>>>>>>>>> die hard)
>>>>>>>>> * I will go through the licences (review and collect
in "licenses"
>>>>>>>>> directory)
>>>>>>>>> * Good point about plugin versions being defined in apache
POM -
>>> will
>>>>>>>>> rework them
>>>>>>>>> * I will delete the existing configuration of the "release"
profile
>>>>>>>>> 
>>>>>>>>> Some things to discuss
>>>>>>>>> 
>>>>>>>>> * What are the benefits to move "templates" to "src/main/templates"?
>>>>>>>>> Currently they are in sync with "freemarker-cli" which
is quite nice
>>>>> for
>>>>>>>>> tests & documentation ...
>>>>>>>>> 
>>>>>>>>> Thanks in advance,
>>>>>>>>> 
>>>>>>>>> Siegfried Goeschl
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> On 26.07.2020, at 01:27, Daniel Dekany <daniel.dekany@gmail.com
<mailto:daniel.dekany@gmail.com>>
>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> I said I will help in the Apache release process,
so only focusing
>>> on
>>>>>>>>> that,
>>>>>>>>>> so some points:
>>>>>>>>>> 
>>>>>>>>>> - We are required to have a so-called source release
(every other
>>>>>>>>>> artifact is optional in the policy). As we are using
the
>>>>>>>>> org.apache:apache
>>>>>>>>>> parent, that should generate that automatically,
with .asc and
>>> sha512
>>>>>>>>> and
>>>>>>>>>> all. But currently it doesn't, because maven-release-plugin
>>>>>>>>> config/argument
>>>>>>>>>> is overwritten with this:
>>>>>>>>> <arguments>-Dmaven.javadoc.skip=true</arguments>.
>>>>>>>>>> We should keep configuring release at minimum, to
avoid such
>>>>>>> accidents.
>>>>>>>>>> Maybe as in
>>>>>>>>>> 
>>> https://github.com/apache/freemarker-docgen/blob/master/pom.xml#L70 <https://github.com/apache/freemarker-docgen/blob/master/pom.xml#L70>.
>>>>>>>>>> - I assume we also want a binary release, for the
CLI only, and
>>>>>>>>>> freemarker-generator-cli-x.y.z-*app*.zip (note the
"-app") will be
>>>>> our
>>>>>>>>>> binary release artifact. Then:
>>>>>>>>>> - It bundles some dependency binaries that are not
under ASL2
>>>>> license.
>>>>>>>>>>  Unfortunately, the licenses of those must be included
in the
>>>>>>>>>> distribution.
>>>>>>>>>>  See the LICENSE at
>>>>>>>>>>  https://github.com/apache/freemarker-docgen/blob/master/LICENSE
<https://github.com/apache/freemarker-docgen/blob/master/LICENSE>.
>>>>>>> At
>>>>>>>>>>  the bottom, it lists the licenses, then it refers
to the actual
>>>>>>>>> license
>>>>>>>>>>  files. As we will have many licenses, let's create
a "licenses"
>>>>>>>>> directory
>>>>>>>>>>  for them. (In the future, the dependencies have
to be checked
>>>>>>>>>> for changes.
>>>>>>>>>>  Even version upgrades my pull in sneaky transient
dependencies.
>>>>>>> Some
>>>>>>>>>>  licenses are not even allowed, so anything but ASL2,
MIT,
>>>>>>>>>>  BSD-without-advertisement-clause, will need closer
attention.)
>>>>>>>>>>  - I noticed that the documentation is not included
in the binary
>>>>>>>>>>  distribution. But because of the extra legal burden
including it
>>>>>>>>> would
>>>>>>>>>>  bring (we have fonts and icons under CC-SA and SIL
OFL in the
>>>>>>> Docgen
>>>>>>>>>>  output), I actually prefer that to stay like that.
>>>>>>>>>>  - .sha512 file is not yet generated
>>>>>>>>>> - freemarker-generator-cli/src/site: If you agree,
instead of this
>>> I
>>>>>>>>>> will create freemarker-generator*-site*/src/docgen,
and convert the
>>>>>>>>>> Markdown to XDocBook. For now this will be only the
CLI
>>>>> documentation,
>>>>>>>>> and
>>>>>>>>>> the JavaDoc, as the freemarker-generator-maven-plugin
is not ready.
>>>>>>> One
>>>>>>>>>> annoyance I realized is that we should have Docgen
in Maven Central
>>>>>>>>> for the
>>>>>>>>>> builds to work reliably in the future, which means
that Docgen has
>>> to
>>>>>>>>> be
>>>>>>>>>> officially released (it never was, it's an internal
tool). That
>>> would
>>>>>>>>> be a
>>>>>>>>>> minimalistic release, means, no announcement, no
web site, just the
>>>>>>>>> bare
>>>>>>>>>> minimum (i.e., source release, and deployment to
Maven Central). I
>>>>>>> have
>>>>>>>>>> some backlog there (Google keeps nagging me about
mobile issues),
>>> but
>>>>>>> I
>>>>>>>>>> hope I can fix that in the coming days, then go through
the
>>> official
>>>>>>>>>> release process (takes 1-2 weeks).
>>>>>>>>>> - Some smaller things:
>>>>>>>>>>  -
>>>>>>>>>>  - Having a "release" profile is also hopefully unnecessary,
>>>>> because
>>>>>>>>>>  org.apache:apache takes care of signing.
>>>>>>>>>>  - We should also remove most plugin version management,
as many
>>> of
>>>>>>>>>>  those versions are set in org.apache:apache.
>>>>>>>>>>  - freemarker-generator-cli/templates should be inside
>>>>>>>>>>  freemarker-generator-cli/src/main/templates, I guess.
>>>>>>>>>> 
>>>>>>>>>> P.s.: Siegfired asked our opinions in another thread.
I did my
>>> part,
>>>>>>> even
>>>>>>>>>> too much (;, so, would be good if others participate
in that as
>>> well.
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> Best regards,
>>>>>>>>>> Daniel Dekany
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Best regards,
>>>>>>>> Daniel Dekany
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Best regards,
>>>>>> Daniel Dekany
>>>>> 
>>>>> 
>>>> 
>>>> --
>>>> Best regards,
>>>> Daniel Dekany
>>> 
>>> 
>> 
>> -- 
>> Best regards,
>> Daniel Dekany


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message