jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: New DataSet implementation
Date Sat, 30 Nov 2013 09:52:42 GMT
On 27 November 2013 21:21, Philippe Mouawad <philippe.mouawad@gmail.com> wrote:
> On Tue, Nov 26, 2013 at 1:05 AM, sebb <sebbaz@gmail.com> wrote:
>
>> On 23 November 2013 13:48, Philippe Mouawad <philippe.mouawad@gmail.com>
>> wrote:
>> > Hello sebb,
>> > What's your decision on this point ?
>>
>> I don't make the decisions - they are ideally arrived at by consensus;
>> failing that, majority.
>>
>> > Still having objection to code commit ?
>>
>> Yes, because it seems wrong to include code for a specific database
>> implementation.
>> We should be implementing generic solutions.
>>
>> So would you be ok to go for Spring-data based implementation ?
> It will impact number of embedded libraries.

There are other possibilities for NoSQL as well, for example
http://appscale.cs.ucsb.edu/datastores.html - AppScale
http://thrift.apache.org/ - Apache Thrift

See also
http://www.oracle.com/technetwork/database/nosqldb/overview/nosql-api-497225.html
http://stackoverflow.com/questions/9252034/is-there-an-emerging-standard-api-for-nosql-databases
http://stackoverflow.com/questions/3439878/are-there-any-nosql-standards-emerging

Given that there is no standard like JDBC available (and there may
never be), what I think needs to happen is to decide where in JMeter
such databases may be useful (probably not only CSV Dataset) and then
define a suitable API for this.
Users can then implement the API in terms of whatever database they
wish to use (which might include a JDBC implementation).
This is not something to be rushed into.

In any case, whatever is decided on belongs in a later version of JMeter.
We first need to get out the version with various fixes to keytool etc.

>
>> > Thanks
>> > Regards
>> > Philippe
>> >
>> >
>> > On Fri, Nov 15, 2013 at 10:37 PM, Philippe Mouawad <
>> > philippe.mouawad@gmail.com> wrote:
>> >
>> >> We could sue Spring-Data but it would introduce a lot of dependencies.
>> >>
>> >>
>> >> On Fri, Nov 15, 2013 at 12:54 AM, sebb <sebbaz@gmail.com> wrote:
>> >>
>> >>> On 13 November 2013 21:59, Philippe Mouawad <
>> philippe.mouawad@gmail.com>
>> >>> wrote:
>> >>> > Hello sebb,
>> >>> > Shall I commit this development or do we wait for 2.10.1 to be
>> released
>> >>> ?
>> >>>
>> >>> I am still concerned that this addition is specific to the Redis
>> server.
>> >>> I would much prefer to see a generic solution that can use various
>> >>> different kinds of servers.
>> >>>
>> >>> > When do you plan to commit your devs on keytool ?
>> >>>
>> >>
>> >> If I have time I will do it , I plan the following, if you want more let
>> >> me know:
>> >> - Add keytool path property
>> >> - Add popup to be clear about error
>> >> - Add info about how to fix it
>> >>
>> >>>
>> >>> Sorry, have not been able to spend any time on this recently.
>> >>>
>> >>> > Thank you
>> >>> > Regards
>> >>> > Philippe
>> >>> >
>> >>> >
>> >>> > On Sat, Nov 2, 2013 at 3:05 PM, Philippe Mouawad <
>> >>> philippe.mouawad@gmail.com
>> >>> >> wrote:
>> >>> >
>> >>> >> Hello Sebb,
>> >>> >> My answers below.
>> >>> >>
>> >>> >> On Sat, Nov 2, 2013 at 10:35 AM, sebb <sebbaz@gmail.com>
wrote:
>> >>> >>
>> >>> >>> On 2 November 2013 08:28, Philippe Mouawad <
>> >>> philippe.mouawad@gmail.com>
>> >>> >>> wrote:
>> >>> >>> > Hello,
>> >>> >>> > Can I proceed with commit these up coming days ?
>> >>> >>>
>> >>> >>> I'm not sure that the discussion was completed.
>> >>> >>>
>> >>> >>> As far as I can tell, the proposal only suits some types
of
>> >>> >>> cloud-based test, and relies on 3rd party servers to hold
the data.
>> >>> >>>
>> >>> >>
>> >>> >> No in my opinion if fits many scenarios:
>> >>> >> - Cloud based tests, this one seems to me an important one,
as Cloud
>> >>> based
>> >>> >> usage are increasing
>> >>> >> - Distributed tests, even if it is possible to do it with CSV,
>> having
>> >>> the
>> >>> >> data in a remote server is much easier to manage than having
to
>> >>> >> split/distribute on servers. Even it is true it requires some
>> skills to
>> >>> >> manage correctly the Redis server
>> >>> >> - Continuous integration tests where also having the data in
a
>> >>> centralised
>> >>> >> , remote servers is easier than managing CSVs. In this case
Redis
>> >>> server
>> >>> >> plays the same role as a JDBC repository
>> >>> >> - Compared to a database it should perform better for high
load
>> tests
>> >>> >> since it's an in-memory repo (although this can be done in
SQL
>> >>> databases),
>> >>> >> see http://redis.io/topics/benchmarks
>> >>> >>
>> >>> >>
>> >>> >>> I'm not yet convinced that this is how we should be extending
>> JMeter.
>> >>> >>>
>> >>> >>
>> >>> >> I don't understand this argument, it would be another string
for
>> >>> JMeter,
>> >>> >> my implementation is only 2 classes +  properties for I18N
?
>> >>> >>
>> >>> >> What do you propose in this case ?
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>> > Thanks
>> >>> >>> > Regards
>> >>> >>> > Philippe
>> >>> >>> >
>> >>> >>> > On Monday, October 28, 2013, Philippe Mouawad wrote:
>> >>> >>> >
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> On Monday, October 28, 2013, sebb wrote:
>> >>> >>> >>
>> >>> >>> >>> On 28 October 2013 01:26, sebb <sebbaz@gmail.com>
wrote:
>> >>> >>> >>> > On 26 October 2013 20:23, Philippe Mouawad
<
>> >>> >>> philippe.mouawad@gmail.com>
>> >>> >>> >>> wrote:
>> >>> >>> >>> >> Hello,
>> >>> >>> >>> >> These days Cloud based testing is
becoming popular and
>> having
>> >>> to
>> >>> >>> >>> distribute
>> >>> >>> >>> >> test data accross many servers through
CSV can become
>> painful
>> >>> if
>> >>> >>> not
>> >>> >>> >>> >> impossible.
>> >>> >>> >>> >>
>> >>> >>> >>> >> Even without Cloud, when using distributed
testing you
>> always
>> >>> have
>> >>> >>> to
>> >>> >>> >>> >> replicate your data on all servers,
which is a painful
>> manual
>> >>> step.
>> >>> >>> >>> >>
>> >>> >>> >>> >> Shouldn't we introduce a new DataSet
more suitable for these
>> >>> use
>> >>> >>> cases
>> >>> >>> >>> ?
>> >>> >>> >>> >>
>> >>> >>> >>> >> We could do it in many different
ways:
>> >>> >>> >>> >> - Integrate an automatic CSV replicator,
this would remain
>> >>> simple
>> >>> >>> and
>> >>> >>> >>> would
>> >>> >>> >>> >> not introduce any tier, but with
cloud based it would not
>> >>> >>> horizontally
>> >>> >>> >>> >> scale easily
>> >>> >>> >>> >
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>> > Not sure I follow the scaling argument.
The file would only
>> >>> have to
>> >>> >>> be
>> >>> >>> >>> > copied once before the test proper starts.
>> >>> >>> >>> > From then on, the data is accessed locally.
>> >>> >>> >>> >
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> The scale word was not good, In my thinking the
issue is more
>> about
>> >>> >>> >> deploying/copying/splitting the file among nodes
or cloud
>> members.
>> >>> A
>> >>> >>> >> centralized access makes it far easier.
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>> > However, with a database, each node will
need at least one
>> >>> >>> connection
>> >>> >>> >>> > to the database, and every time more
data is needed there
>> will
>> >>> be
>> >>> >>> >>> > network traffic.
>> >>> >>> >>> > Or the database has to be running on
the server node.
>> >>> >>> >>> >
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> Agree, I was not saying anything different. But
as I said this
>> can
>> >>> be
>> >>> >>> >> useful for middle or low volume
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>> >> - Use a JDBC DataSet, but we would
need to ensure it
>> performs
>> >>> >>> fine, and
>> >>> >>> >>> >> jdbc protocol is not well suited
for cloud based deployment
>> >>> (But it
>> >>> >>> >>> could
>> >>> >>> >>> >> also be an interesting feature for
Continuous Integration)
>> >>> >>> >>> >> - Use a NOSQL repository  (Redis
seems to me the best
>> choice)
>> >>> , see
>> >>> >>> >>> this
>> >>> >>> >>> >> high level summary which I find interesting
>> >>> >>> >>> >>
>> >>> >>> >>>
>> >>> >>>
>> >>>
>> http://www.journaldunet.com/developpeur/outils/comparatif-des-bases-nosql/comparatif-des-bases-nosql-tableau-de-synthese.shtml
>> >>> >>> >>> >>
>> >>> >>> >>> >> I have implemented a new Redis (based
on Java library for
>> >>> Redis)
>> >>> >>> >>> DataSet
>> >>> >>> >>> >> which I plan to commit if no objection.
>> >>> >>> >>> >>
>> >>> >>> >>> >> It will introduce 2 new dependencies
with Apache License:
>> >>> >>> >>> >> - Jedis (http://code.google.com/p/jedis/)
>> >>> >>> >>> >> - commons-pool
>> >>> >>> >>> >
>> >>> >>> >>> > There is also a dependency on Redis,
but I guess that would
>> not
>> >>> be
>> >>> >>> >>> bundled.
>> >>> >>> >>>
>> >>> >>> >>>
>> >>> >>> >> no need to anything else than jedis + commons-pool
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>> I've just noticed that Redis is provided as
source which needs
>> to
>> >>> be
>> >>> >>> >>> built before use.
>> >>> >>> >>> If it's difficult to copy CSV files to cloud
nodes, it seems
>> to me
>> >>> >>> >>> it's going to be much harder to install Redis.
>> >>> >>> >>> In which case additional network traffic will
be needed to
>> access
>> >>> the
>> >>> >>> >>> database.
>> >>> >>> >>>
>> >>> >>> >>> Also there is no official Windows release.
>> >>> >>> >>>
>> >>> >>> >>> >> Thoughts ?
>> >>> >>> >>> >
>> >>> >>> >>> > Is MongoDB not suitable?
>> >>> >>> >>> > We already include a jar for it.
>> >>> >>> >>> >
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>  Mongodb is more document oriented and has another
type of use
>> >>> cases.
>> >>> >>> One
>> >>> >>> >> interesting feature of redis is lists where you
can pop a line
>> it
>> >>> will
>> >>> >>> not
>> >>> >>> >> be available to next calls, it is suitable for
tests that need
>> >>> >>> uniqueness
>> >>> >>> >> accross all nodes.
>> >>> >>> >>
>> >>> >>> >>> >> --
>> >>> >>> >>> >> Regards.
>> >>> >>> >>> >> Philippe M.
>> >>> >>> >>> >> @philmdot <https://twitter.com/philmdot>
>> >>> >>> >>>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> --
>> >>> >>> >> Cordialement.
>> >>> >>> >> Philippe Mouawad.
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >
>> >>> >>> > --
>> >>> >>> > Cordialement.
>> >>> >>> > Philippe Mouawad.
>> >>> >>>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >> Cordialement.
>> >>> >> Philippe Mouawad.
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Cordialement.
>> >>> > Philippe Mouawad.
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Cordialement.
>> >> Philippe Mouawad.
>> >>
>> >>
>> >>
>> >
>> >
>> > --
>> > Cordialement.
>> > Philippe Mouawad.
>>
>
>
>
> --
> Cordialement.
> Philippe Mouawad.

Mime
View raw message