quetz-mod_python-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick <n...@dd.revealed.net>
Subject Re: Solving the import problem
Date Tue, 07 Jun 2005 21:16:45 GMT
I'm all for fixing the importing of handler modules, because that can bite 
you if they are being loaded and initialized under heavy load in the same 
process in different threads.

As for user imports in general, I would prefer that everything work in a 
standard Python way out of the box.  mod_python should be what it is -- an 
interface to apache.  That's most useful for application and framework 
developers I think.  Otherwise you could end up breaking other people's 
applications and frameworks who have resolved the problem in a different way.

I'm not opposed to providing some sort of module or function for psp or 
publisher or whatever "example" handlers that come with mod_python, just so 
long as it's not in the core of mod_python itself.  To me this is an 
application/framework issue, not a mod_python issue.  That said, the 
behaviour should probably be documented well so people can avoid the pitfalls.


Nicolas Lehuen wrote:
> One last thing that we should prepare is a clear and definite answer
> to the zillion users who need to import a custom utility module.
> Today, we have 4 ways of importing code :
> a) the standard "import" keyword. Today, it works unchanged
> (mod_python doesn't install any import hook). The consequence is that
> the only modules that can be imported this way are those found on the
> PYTHONPATH. Importing custom code is easy if you can manipulate this
> variable (either directly or through the PythonPath configuration
> directive), but not everybody has this luxury (think shared hosting,
> although not being able to change the PythonPath through an .htaccess
> file seems pretty restrictive to me.).
> b) the PythonImport directive, which ensure that a module is imported
> (hence its initialization code is ran), but doesn't really import it
> into the handler's or published module's namespace.
> c) the apache.import_module() function, which is frankly a strange
> beast. It knows how to approximately reload some modules, but has many
> tricks that makes it frankly dangerous, namely the one that causes
> modules with the same name but residing in different directories to
> collide. I really think that mixing dynamic (re)loading of code with
> the usual import mechanisms is a recipe for disaster. Anyway, today,
> it's the only way our users can import some shared code, using a
> strange idiom like
> apache.import_module('mymodule',[dirname(__file__)]).
> d) the new publisher.get_page(req,path), which is not really an answer
> since it is designed to allow a published object to call another
> published object from another page (not to call some shared code).
> This mess should be sorted out. As a baseline, I'd say that we have 4
> kinds of code in mod_python :
> 1) the standard Python code that should be imported using the "import" keyword
> 2) handlers, which are dynamically loaded through apache.import_module
> (so they are declared in sys.module, with all the problem that can
> cause when sharing a single setup with multiple handlers that have the
> same name, "publisher" for example) - this should be fixed.
> 3) published modules, which are dynamically loaded by the
> mod_python.publisher handler (so now they don't have any problems that
> were previously caused by apache.import_module). An important thing to
> notice is that published module are usually stored in a directory
> which is visible by Apache (handlers don't need to reside in a public
> directory), amongst .html and image files. Hence, people can
> legitimately be reluctant to put their core application code
> (including DB passwords etc.) in published modules, for security and
> code/presentation separation issues.
> 4) custom library code, AKA core application code. This code should
> reside somewhere, preferably in a private directory (at least direct
> access to this code from the web should be denied) and be easily
> imported and reloaded into published modules, without having to tinker
> too much with the PYTHONPATH variable or the PythonPath directive.
> What would be nice is a clear and definite way to handle those 4 kinds
> of code. To me, layers 2, 3 and 4 could be handled by the same dynamic
> code cache, except that a careful directory structure or naming scheme
> would prevent the layer 4 to be visible from the web.
> I know Vampire solves a lot of these problems, so we have two alternatives : 
> A) We decide that we won't solve the whole problem into mod_python. We
> take apache.import_module out and shoot it. Handlers are loaded in a
> real dynamic code cache maybe the same as the one now used by
> mod_python.publisher), which solves a lot of problems.
> Custom library code is not handled : if you want to import some code,
> you put it wherever you like and make sure PYTHONPATH or the
> PythonPath directive point to it, so you can import it like a standard
> module. You'll never use apache.import_module anymore, it will
> blissfully dissolve into oblivion (and be removed from the module,
> anyway).
> If you need to reload your core application code without restarting
> Apache, then too bad, mod_python doesn't know how to do this. Check
> out Vampire.
> B) We decide to solve the whole problem into mod_python.
> apache.import_module is not much luckier this time, it is still taken
> out and shot in the head. We solve the handlers loading problem. But
> now, with a little help from Graham, custom application code can be
> dynamically loaded and reloaded from any place without having to
> tinker with the PYTHONPATH variable and/or the PythonPath directive.
> Everything can be done from the source code with a little help from an
> .htaccess file.
> So, sorry for this long mail, but I had to get this out. The current
> situation is pretty bad, zillions of people need to do this simple
> thing, and when they notice it's not that simple (or it's buggy), they
> decide to build the nth application framework on mod_python. So,
> either we reckon it's None of our business, that users should turn to
> higher level frameworks like Vampire, and we remove
> apache.import_module, or we decide to tackle the issue, and we remove
> apache.import_module. Either way, it must leave :).
> What do you think ?
> Regards,
> Nicolas

View raw message