commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thorsten Kiefer <>
Subject Re: [Math] Would like to know, if extensions are needed
Date Thu, 18 Dec 2014 22:13:00 GMT

Am 03.12.2014 um 12:53 schrieb Gilles:
> Hello.
> On Wed, 03 Dec 2014 10:13:08 +0100, Thorsten Kiefer wrote:
>> Hi,
>> the port is now finished.
>> The code may still contain bugs.
>> Maybe alpha-state.
>> But if you like it, you can add it.
> Thank you for offering code to enhance the scope of Commons Math.
> Yet I must emphasize that one of the goals of the project is to
> provide an "integrated" set of functionalities (a.o. consistent
> style and API throughout the codebase, full documentation and test
> coverage).
> Another on-going task is to include contributors who will stay and
> maintain the codebase.
> We do not usually have the resources neither to integrate a new
> functionality, nor to maintain it afterwards.
> You are very welcome to explain the purpose and state of your
> contribution; then we can start a discussion on how to move toward
> including the code in Commons Math.

The purpose of my project is a learning algorithm, which uses
function approximation and reinforcement learning.
With reinforcement learning a so-called agent can learn
optimal behaviour from only receiving rewards
for actions done in perceived states.

At the moment I am having problems to stabilize the algorithms.
The topology of the approximation functions and the initial
values for the parameters are essential for success.

At the moment only the fourier basis is practical as
approximation function.

Naively one would create a lookup table of scores for state-action pairs.
As the impractical huge size of these lookup tables is the problem,
the lookup table is being interpolated by an approximation function.

The lookup table and respectively the approximation function
map state-action pairs to scorings.

The RL algorithm adujsts the scorings during simulation.
And then it can decide, which action to take, by aiming for the highest 

But still all unstable.

Thorsten Kiefer

> Thanks for your interest in the project,
> Gilles
>> Best Regards
>> Thorsten Kiefer
>> Am 17.11.2014 um 23:57 schrieb Thorsten Kiefer:
>>> Hi,
>>> I decided to port it to java, and if you should like it, you can extend
>>> [Math] with RL algorithms.
>>> It will take a while, until I ported it all to java.
>>> Regards
>>> Thorsten
>>> Am 15.11.2014 um 15:01 schrieb Gilles:
>>>> On Sat, 15 Nov 2014 14:32:07 +0100, Thorsten Kiefer wrote:
>>>>> Hello,
>>>>> Am 15.11.2014 um 13:39 schrieb Gilles:
>>>>>> Hello.
>>>>>> On Sat, 15 Nov 2014 06:15:32 +0100, Thorsten Kiefer wrote:
>>>>>>> Hi,
>>>>>>> I coded this in JavaScript :
>>>>>>> I am wondering, if parts of this might be useful for the math

>>>>>>> module.
>>>>>>> If so, I could contribute it.
>>>>>> There are some machine-learning algorithms in Commons Math, more
>>>>>> would probably be welcome.
>>>>>> Could you perhaps describe with a little more details what you are
>>>>>> proposing?
>>>>> I would like to implement a (non-turing-complete) meta languge
>>>> I don't think that CM would be the right place for a special-purpose
>>>> language. [Some time ago, there was a proposal for a language to
>>>> create arithmetical expressions.]
>>>> If you think otherwise, could you provide some rationale, so that we
>>>> can comment on concrete arguments?
>>>>> for
>>>>> the approximation function and the RL-update rule.
>>>>> Then I would provide Q(lambda) and SARSA(lambda) using the fourier 
>>>>> basis.
>>>>> Might be useful for somebody.
>>>> Perhaps. I don't know what are RL, Q, SARSA.
>>>>> The Meta-compiler's result could implement the interfaces used by the
>>>>> FunctionFitters.
>>>>> As far as I can remember : MultivariateFunction or so...
>>>> This looks like your project would depend on Commons Math, but need 
>>>> not
>>>> be part of it.
>>>>> For more understanding you can also look up the code on my git 
>>>>> profile.
>>>> I tried to read "": it just says "My website"...
>>>> Care to give a link to a description?  Thanks.
>>>> Regards,
>>>> Gilles
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message