stratos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Imesh Gunaratne <im...@apache.org>
Subject Re: [Dev] [GSoC-2015] Showing health statistics in GUI
Date Sun, 02 Aug 2015 12:29:43 GMT
Hi Ashan,

Great work! May be we could arrange a public Google Hangout when you are
available to present the current status of the project.

Thanks

On Thu, Jul 30, 2015 at 11:00 PM, Ashan Dhananjaya <dhananjaya92@gmail.com>
wrote:

> Hi All,
>
> I have implemented formatters and did some changes in the cep event flow
> to change the RDBMS 3 tables in to one table as we discussed last week. And
> committed it. And id did few changes to the UI according to the feedback i
> had last week. I finished writing the REST API documentation and sent it to
> everyone.
>
> I have started to write a blog for the marketing aspect for stratos how
> Health Stat UI works and a documentation for the health statistics UI.
>
> [1]=
> https://github.com/apache/stratos/commit/1490310cae3ba085fbb77d84346dd66fd2b12132
>
> Thank You!
> Best Regards,
> Ashan
>
> On Fri, Jul 24, 2015 at 11:17 PM, Imesh Gunaratne <imesh@apache.org>
> wrote:
>
>> Great work Ashan!
>>
>> Actually you don't need to replace MySQL database with H2 rather both
>> options might be needed. The reason for having H2 is to allow users to see
>> the statistics when running on Single-JVM mode with embedded H2 database.
>>
>> On Thu, Jul 23, 2015 at 12:35 AM, Ashan Dhananjaya <
>> dhananjaya92@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> I have implemented the UI for the Health Statistics UI. The git hub
>>> commits can be find from here [1].
>>>
>>> And im moving forward to replace the MySql database from the embedded H2
>>> RDBMS as we discussed offline on 22nd of  July.
>>>
>>> Thank You!
>>> Best Regards,
>>> Ashan
>>>
>>>
>>> [1]=
>>> https://github.com/ashandk/stratos/commit/310ea343d80d009778f572da12a4098c7d2177ec
>>>
>>>
>>> On Thu, Jul 16, 2015 at 7:29 PM, Ashan Dhananjaya <
>>> dhananjaya92@gmail.com> wrote:
>>>
>>>> Hi Imesh,
>>>>
>>>> Noted. I will create a documentation for the REST API.
>>>>
>>>> Thank You!
>>>> Best Regards,
>>>> Ashan
>>>>
>>>> On Thu, Jul 16, 2015 at 10:51 AM, Imesh Gunaratne <imesh@apache.org>
>>>> wrote:
>>>>
>>>>>
>>>>> On Tue, Jul 14, 2015 at 9:44 PM, Reka Thirunavukkarasu <reka@wso2.com>
>>>>>  wrote:
>>>>>>
>>>>>>
>>>>>>> 1.getAverageClusterMemoryByClusterId
>>>>>>>
>>>>>>
>>>>>> /cluster/averageMemory/{Id}/{Interval}}
>>>>>>
>>>>>>
>>>>>>> 2.getAverageClusterLoadByClusterId
>>>>>>>
>>>>>>
>>>>>> /cluster/averageLoad/{Id}/{Interval}}
>>>>>>
>>>>>>
>>>>>>> 3.getAverageMemberMemoryByMemberId
>>>>>>>
>>>>>>
>>>>>> /member/averageMemory/{Id}/{Interval}}
>>>>>>
>>>>>> 4.getAverageMemberLoadByMemberId
>>>>>>>
>>>>>>
>>>>>> /member/averageLoad/{Id}/{Interval}}
>>>>>>
>>>>>>
>>>>>>> 5.getAverageClusterFlightRequestCountByClusterId
>>>>>>>
>>>>>>
>>>>>>  /cluster/flightRequestCount/{Id}/{Interval}}
>>>>>>
>>>>>
>>>>> +1 A great thought! @Ashan: It would be better if you can document the
>>>>> API methods (may be in a Google doc).
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Tue, Jul 14, 2015 at 9:44 PM, Reka Thirunavukkarasu <reka@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Ashan,
>>>>>>
>>>>>> Wouldn't it be better to stick with the REST API paths currently
used
>>>>>> by stratos? You can inherit and add more stuffs as you want. Please
see
>>>>>> below for some sample paths:
>>>>>>
>>>>>> On Tue, Jul 14, 2015 at 1:55 AM, Ashan Dhananjaya <
>>>>>> dhananjaya92@gmail.com> wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> I have implemented the REST api methods for the health statistics
UI
>>>>>>> in the stratos existing REST API.
>>>>>>>
>>>>>>> 1.getAverageClusterMemoryByClusterId
>>>>>>>
>>>>>>
>>>>>> /cluster/averageMemory/{Id}/{Interval}}
>>>>>>
>>>>>>
>>>>>>> 2.getAverageClusterLoadByClusterId
>>>>>>>
>>>>>>
>>>>>> /cluster/averageLoad/{Id}/{Interval}}
>>>>>>
>>>>>>
>>>>>>> 3.getAverageMemberMemoryByMemberId
>>>>>>>
>>>>>>
>>>>>> /member/averageMemory/{Id}/{Interval}}
>>>>>>
>>>>>> 4.getAverageMemberLoadByMemberId
>>>>>>>
>>>>>>
>>>>>> /member/averageLoad/{Id}/{Interval}}
>>>>>>
>>>>>>
>>>>>>> 5.getAverageClusterFlightRequestCountByClusterId
>>>>>>>
>>>>>>
>>>>>>  /cluster/flightRequestCount/{Id}/{Interval}}
>>>>>>
>>>>>> Thanks,
>>>>>> Reka
>>>>>>
>>>>>>>
>>>>>>> Please find the Github commit here[1].
>>>>>>>
>>>>>>> [1]=
>>>>>>> https://github.com/apache/stratos/commit/5ab9159744f136e649174f7293adabe5c77fb4bd
>>>>>>>
>>>>>>>
>>>>>>> Thank You!
>>>>>>> Best Regards,
>>>>>>> Ashan
>>>>>>>
>>>>>>> On Thu, Jul 9, 2015 at 8:30 PM, Ashan Dhananjaya <
>>>>>>> dhananjaya92@gmail.com> wrote:
>>>>>>>
>>>>>>>> Noted Lahiru.
>>>>>>>>
>>>>>>>> On Thu, Jul 9, 2015 at 7:48 PM, Lahiru Sandaruwan <lahirus@wso2.com
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> Hi Ashan,
>>>>>>>>>
>>>>>>>>> Good stuff. It is fine to set the time stamp from CEP
for now. But
>>>>>>>>> ultimately we need the timestamp to be sent from Python
cartridge agent.
>>>>>>>>> Let's fix that part we proceed.
>>>>>>>>>
>>>>>>>>> Thanks.
>>>>>>>>>
>>>>>>>>> On Thu, Jul 9, 2015 at 7:38 PM, Ashan Dhananjaya <
>>>>>>>>> dhananjaya92@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi All,
>>>>>>>>>>
>>>>>>>>>> I was able to get the TimeaStamp to the RDBMS without
any
>>>>>>>>>> conflict AFAIK. I am testing the flow more. Please
find the commits [1],
>>>>>>>>>> [2] for the Formatters.
>>>>>>>>>>
>>>>>>>>>> [1]=
>>>>>>>>>> https://github.com/ashandk/stratos/commit/efa25eb255cadbf2843e5f0d0e13b34c09535f91
>>>>>>>>>> [2]=
>>>>>>>>>> https://github.com/ashandk/stratos/commit/9c0fce9081c299c34031d602c94ea712602c7a6c
>>>>>>>>>>
>>>>>>>>>> Thank You!
>>>>>>>>>> Best Regards,
>>>>>>>>>> Ashan
>>>>>>>>>>
>>>>>>>>>> On Thu, Jul 9, 2015 at 2:41 PM, Lahiru Sandaruwan
<
>>>>>>>>>> lahirus@wso2.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Great work Ashan.
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jul 9, 2015 at 2:39 PM, Ashan Dhananjaya
<
>>>>>>>>>>> dhananjaya92@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi All,
>>>>>>>>>>>>
>>>>>>>>>>>> I was able to take the timestamp from the
cartridge agent
>>>>>>>>>>>> (WSO2CEPHealthStatisticsPublisher and WSO2CEPInFlightRequestPublisher)
to
>>>>>>>>>>>> CEP PayLoad definition.Git Hub commits can
be found here[1], [2]. Currently
>>>>>>>>>>>> working on sending the timestamp to the RDBMS
without conflicting execution
>>>>>>>>>>>> plans.
>>>>>>>>>>>>
>>>>>>>>>>>> I have included the read-me [2] file to the
cep extensions how
>>>>>>>>>>>> to configure should happen for the Health
Stat publishing to the RDBMS.
>>>>>>>>>>>>
>>>>>>>>>>>> [1]=
>>>>>>>>>>>> https://github.com/apache/stratos/commit/4c65e705be822fd420b4e48ebafef789c8686b76
>>>>>>>>>>>> [2]=
>>>>>>>>>>>> https://github.com/apache/stratos/commit/e09334cd98f245a4633c992f02db2878ac24d3d0
>>>>>>>>>>>> [3]=
>>>>>>>>>>>> https://github.com/apache/stratos/commit/5ca895ef030f6aaeba56183d982aa5ae95db9337
>>>>>>>>>>>>
>>>>>>>>>>>> Thank You!
>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>> Ashan
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Jul 7, 2015 at 1:13 PM, Lasantha
Fernando <
>>>>>>>>>>>> lasantha.fdo@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Ashan,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Currently, you would not be able to get
the timestamp out of
>>>>>>>>>>>>> the event
>>>>>>>>>>>>> unless you use a databridge agent callback.
In that case, you
>>>>>>>>>>>>> can use
>>>>>>>>>>>>> Java code to simply get by calling event.getTimestamp().
>>>>>>>>>>>>>
>>>>>>>>>>>>> However, if you want to use the timestamp
as an attribute
>>>>>>>>>>>>> while doing
>>>>>>>>>>>>> processing within CEP, you would have
to pass the timestamp as
>>>>>>>>>>>>> an
>>>>>>>>>>>>> attribute of the stream. i.e. add it
as an attribute of either
>>>>>>>>>>>>> meta,
>>>>>>>>>>>>> correlation or payload data. Then you
can use that timestamp
>>>>>>>>>>>>> and
>>>>>>>>>>>>> format the output event as necessary.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Lasantha
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Jul 7, 2015 at 1:04 PM, Ashan
Dhananjaya <
>>>>>>>>>>>>> dhananjaya92@gmail.com> wrote:
>>>>>>>>>>>>> > Hi All,
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > I applied the above patch and get
the timestamp in the Input
>>>>>>>>>>>>> Adapter. The
>>>>>>>>>>>>> > formatter i used was
>>>>>>>>>>>>> "member_average_memory_consumption_stats:1.0.0".
I
>>>>>>>>>>>>> > tried several time to get the timeStamp
attribute but
>>>>>>>>>>>>> failed. Is there any
>>>>>>>>>>>>> > way to get a data which is outside
in the payload? The Input
>>>>>>>>>>>>> stream is as
>>>>>>>>>>>>> > below,
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > 10:40:27,757 [-] [DataBridge-Core-pool-2-thread-1]
 INFO
>>>>>>>>>>>>> TenantId=-1234 :
>>>>>>>>>>>>> > Input Event Adaptor : DefaultWSO2EventInputAdaptor,
received
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Event{
>>>>>>>>>>>>> >   streamId='cartridge_agent_health_stats:1.0.0',
>>>>>>>>>>>>> >   timeStamp=1436245827731,
>>>>>>>>>>>>> >   metaData=null,
>>>>>>>>>>>>> >   correlationData=null,
>>>>>>>>>>>>> >   payloadData=[single-cartridge-app.my-php.php.domain,
>>>>>>>>>>>>> > single-cartridge-app-1, network-partition-1,
>>>>>>>>>>>>> >
>>>>>>>>>>>>> single-cartridge-app.my-php.php.domaina5598e2b-995f-4620-b652-e66341e24c99,
>>>>>>>>>>>>> > partition-1, memory_consumption,
20.0],
>>>>>>>>>>>>> >   arbitraryDataMap={},
>>>>>>>>>>>>> > }
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Any help will be really appreciated.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Thank You!
>>>>>>>>>>>>> > Best Regards,
>>>>>>>>>>>>> > Ashan
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > On Tue, Jul 7, 2015 at 9:32 AM,
Chamila De Alwis <
>>>>>>>>>>>>> chamilad@wso2.com> wrote:
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> Hi Ashan,
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> As I've seen from the PCA testing
recently, the timestamp
>>>>>>>>>>>>> is added to
>>>>>>>>>>>>> >> every ThriftEvent before publishing.
Are you working on PCA
>>>>>>>>>>>>> or the Mock
>>>>>>>>>>>>> >> IaaS? I took a look at the Mock
IaaS and the
>>>>>>>>>>>>> WSO2CEPStatisticsPublisher as
>>>>>>>>>>>>> >> well, and found out that the
latter indeed doesn't add the
>>>>>>>>>>>>> timestamp to the
>>>>>>>>>>>>> >> ThriftEvents. This means that
the JCA also publishes Thrift
>>>>>>>>>>>>> Events with
>>>>>>>>>>>>> >> empty timestamp fields.
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> Following is the simple fix
for that. I've attached the
>>>>>>>>>>>>> patch herewith.
>>>>>>>>>>>>> >> Could you quickly verify with
the patch applied build also?
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> chamilad@chamilad-ThinkPad-T530:~/dev/stratos[master
*]$
>>>>>>>>>>>>> git diff
>>>>>>>>>>>>> >> diff --git
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> a/components/org.apache.stratos.common/src/main/java/org/apache/stratos/common/statistics/publisher/wso2/cep/WSO2CEPStatisticsPublisher.java
>>>>>>>>>>>>> >> b/c
>>>>>>>>>>>>> >> index 653288d..f1fe426 100644
>>>>>>>>>>>>> >> ---
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> a/components/org.apache.stratos.common/src/main/java/org/apache/stratos/common/statistics/publisher/wso2/cep/WSO2CEPStatisticsPublisher.java
>>>>>>>>>>>>> >> +++
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> b/components/org.apache.stratos.common/src/main/java/org/apache/stratos/common/statistics/publisher/wso2/cep/WSO2CEPStatisticsPublisher.java
>>>>>>>>>>>>> >> @@ -29,6 +29,7 @@ import
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> org.wso2.carbon.databridge.agent.thrift.exception.AgentException;
>>>>>>>>>>>>> >>  import org.wso2.carbon.databridge.commons.Event;
>>>>>>>>>>>>> >>  import org.wso2.carbon.databridge.commons.StreamDefinition;
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> +import java.util.Date;
>>>>>>>>>>>>> >>  import java.util.HashMap;
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >>  /**
>>>>>>>>>>>>> >> @@ -98,6 +99,7 @@ public class
WSO2CEPStatisticsPublisher
>>>>>>>>>>>>> implements
>>>>>>>>>>>>> >> StatisticsPublisher {
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >>          Event event = new Event();
>>>>>>>>>>>>> >>          event.setPayloadData(payload);
>>>>>>>>>>>>> >> +        event.setTimeStamp(new
Date().getTime());
>>>>>>>>>>>>> >>          event.setArbitraryDataMap(new
HashMap<String,
>>>>>>>>>>>>> String>());
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >>          try {
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> Regards,
>>>>>>>>>>>>> >> Chamila de Alwis
>>>>>>>>>>>>> >> Committer and PMC Member - Apache
Stratos
>>>>>>>>>>>>> >> Software Engineer | WSO2 | +94772207163
>>>>>>>>>>>>> >> Blog: code.chamiladealwis.com
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> --
>>>>>>>>>>> Lahiru Sandaruwan
>>>>>>>>>>> Committer and PMC member, Apache Stratos,
>>>>>>>>>>> Senior Software Engineer,
>>>>>>>>>>> WSO2 Inc., http://wso2.com
>>>>>>>>>>> lean.enterprise.middleware
>>>>>>>>>>>
>>>>>>>>>>> phone: +94773325954
>>>>>>>>>>> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
>>>>>>>>>>> linked-in:
>>>>>>>>>>> http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Dev mailing list
>>>>>>>>>>> Dev@wso2.org
>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> --
>>>>>>>>> Lahiru Sandaruwan
>>>>>>>>> Committer and PMC member, Apache Stratos,
>>>>>>>>> Senior Software Engineer,
>>>>>>>>> WSO2 Inc., http://wso2.com
>>>>>>>>> lean.enterprise.middleware
>>>>>>>>>
>>>>>>>>> phone: +94773325954
>>>>>>>>> email: lahirus@wso2.com blog: http://lahiruwrites.blogspot.com/
>>>>>>>>> linked-in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Reka Thirunavukkarasu
>>>>>> Senior Software Engineer,
>>>>>> WSO2, Inc.:http://wso2.com,
>>>>>> Mobile: +94776442007
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Senior Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Senior Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>


-- 
Imesh Gunaratne

Senior Technical Lead, WSO2
Committer & PMC Member, Apache Stratos

Mime
View raw message