lens-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tao Yan <t...@linkedin.com>
Subject Re: Lens Scalibility
Date Fri, 02 Sep 2016 17:51:52 GMT
Thanks!

On Thu, Sep 1, 2016 at 12:54 AM, Puneet Gupta <puneet.gupta@inmobi.com>
wrote:

> As of now lens is self serve and we expect users /secondary developers  to
> onboard their schemas on to lens service.  So we spend minimal effort in
> supporting users as such.
> There are some internal tools that are made available to our falcon
> <https://falcon.apache.org/> run pipelines to interact with lens service
> for partition registration.
> We use the metrics emitted by lens service (http://lensserver:8080/admin/
> metrics/?pretty=true) to monitor the service health.
>
> Most developers are involved in building new features/improvements/bug
> fixes.
>
>
>
> Thanks,
> Puneet Gupta
>
> On Tue, Aug 30, 2016 at 10:07 PM, Tao Yan <tyan@linkedin.com> wrote:
>
>> Hi Puneet,
>>
>> Thanks very much for the information. Do you mind sharing some
>> information about the number of engineers developing / maintaining Lens?
>> the number of users and the percentage time developers used to support them?
>>
>> Thanks,
>>
>> On Tue, Aug 30, 2016 at 1:18 AM, Puneet Gupta <puneet.gupta@inmobi.com>
>> wrote:
>>
>>> Hi Tao
>>>
>>> You can find some discussion on this aspect in LENS-658
>>> <https://issues.apache.org/jira/browse/LENS-658>.
>>>
>>> As of now Lens server does not support clustered setup. From what we
>>> have seen in production, lens server consumes very little system resources
>>> as all the heavy lifting is done by target systems (Hive, Mysql, Vertica).
>>> The only operation on lens server that can consume some resources is query
>>> result formatting. We are able to easily support ~750 unique users and ~100
>>> concurrent users. We are running about 8000 queries per week.
>>>
>>> What is the kind or load your are expecting for your deployment ?
>>>
>>> Thanks,
>>> Puneet Gupta
>>>
>>> On Tue, Aug 30, 2016 at 6:18 AM, Tao Yan <tyan@linkedin.com> wrote:
>>>
>>>> Hi Lens Developers,
>>>>
>>>> As the number of users grow, one Lens server may not be able to support
>>>>  all users. Is it possible to setup a multi-server Lens cluster?
>>>>
>>>> Thanks,
>>>> --
>>>>
>>>> *Tao Yan*
>>>> Software Engineer
>>>> Data Analytics Infrastructure Tools and Services
>>>>
>>>>
>>>>
>>>> 206.250.5345
>>>> tyan@linkedin.com
>>>> https://www.linkedin.com/in/taousc
>>>>
>>>
>>>
>>> _____________________________________________________________
>>> The information contained in this communication is intended solely for
>>> the use of the individual or entity to whom it is addressed and others
>>> authorized to receive it. It may contain confidential or legally privileged
>>> information. If you are not the intended recipient you are hereby notified
>>> that any disclosure, copying, distribution or taking any action in reliance
>>> on the contents of this information is strictly prohibited and may be
>>> unlawful. If you have received this communication in error, please notify
>>> us immediately by responding to this email and then delete it from your
>>> system. The firm is neither liable for the proper and complete transmission
>>> of the information contained in this communication nor for any delay in its
>>> receipt.
>>
>>
>>
>>
>> --
>>
>> *Tao Yan*
>> Software Engineer
>> Data Analytics Infrastructure Tools and Services
>>
>>
>>
>> 206.250.5345
>> tyan@linkedin.com
>> https://www.linkedin.com/in/taousc
>>
>
>
> _____________________________________________________________
> The information contained in this communication is intended solely for the
> use of the individual or entity to whom it is addressed and others
> authorized to receive it. It may contain confidential or legally privileged
> information. If you are not the intended recipient you are hereby notified
> that any disclosure, copying, distribution or taking any action in reliance
> on the contents of this information is strictly prohibited and may be
> unlawful. If you have received this communication in error, please notify
> us immediately by responding to this email and then delete it from your
> system. The firm is neither liable for the proper and complete transmission
> of the information contained in this communication nor for any delay in its
> receipt.
>



-- 

*Tao Yan*
Software Engineer
Data Analytics Infrastructure Tools and Services



206.250.5345
tyan@linkedin.com
https://www.linkedin.com/in/taousc

Mime
View raw message