trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Zeng <billzeng2...@gmail.com>
Subject Re: generating hash from packet content
Date Thu, 28 Aug 2014 17:35:51 GMT
Just to throw another idea your way. We can insert another level of
indirection between URL's and objects. Every object has a unique hash.
URL's point to the hashes instead of objects. The hashes are used to look
up objects. Even if multiple URL's are duplicated and hence their hashes,
they always point to the same object. It seems a non-easy project though.
It requires major changes to ATS.

Bin



On Thu, Aug 28, 2014 at 12:50 AM, Niki Gorchilov <niki@gorchilov.com> wrote:

> Hi, Rasim,
>
> AFAICT metalink plugin has a code to calculate checksum of the object
> contents. Still I don't understand how this is going to resolve the problem
> you're trying to address.
>
> In order to have the hash, you need to download the whole object from
> origin server, thus you learn if you have it already cached post factum.
> Then how do you save bandwidth? In such mode, you can only save storage
> space by de-duplication of objects (storing them once instead of multiple
> times).
>
> Second issue I can foresee is RAM exhaust as you need to buffer the whole
> object in memory before making store decision. Especially in terms of
> youtube videos you're planing to process.
>
> HTH,
> Niki
>
>
>
>
> 2014-08-27 19:17 GMT+03:00 Rasim Saltuk Alakuş <ralakus@turksat.com.tr>:
>
>
>>
>> Hi All,
>>
>>
>>
>> ATS uses URL hash for cache storage. And CacheUrl plugin adds some more
>> flexibility in URL hashing strategy.
>>
>>
>>
>> We think of creating hash based on packet content and use it as the hash
>> while storing and retrieving from cache This looks a better solution, so
>> that URI changes won't hurt caching system. One immediate benefit for
>> example if you cache YouTube , each request for same video can have
>> different URL and CacheUrl plugin does not always provide a good solution.
>> Also maintaining site based hash filters looks not an elegant solution.
>>
>>
>>
>> Is there any previous or active work for implementing content based
>> hashing? What kind of problems and constrains you may guess. Is there any
>> volunteer to implement this feature together with us?
>>
>>
>>
>> Kind regards
>>
>> Saltuk Alakuş
>>
>>
>> *Rasim Saltuk Alakuş *
>> Kıdemli Uzman
>>  Senior Specialist
>>  Bilişim Ar-Ge ve Teknoloji Direktörlüğü
>>  IT R & D and Technology
>>
>>
>> *www.turksat.com.tr <http://www.turksat.com.tr> *ralakus@turksat.com.tr
>>
>> TUNA MAH. İSMAİL ÖZKUNT 1709 SOK. NO:3 KAT:2 KARŞIYAKA – İZMİR
>> T : +90 232 323 43 00
>> F : +90 232 323 43 44
>>
>>
>>
>>
>>  "Bu mesaj ve ekleri mesajda gönderildiği belirtilen kişi ya da kişilere
>> özel olup gizli bilgiler içeriyor olabilir. Mesajın muhatabı ilgilisi ya da
>> gönderileni değilseniz lütfen mesajı herhangi bir şekilde kullanmayınız
>> çoğaltmayınız ve başkalarına ifşa etmeyiniz. Eğer mesaj yanlışlıkla size
>> ulaşmışsa anılan mesaj ve ekinde yer alan bilgileri gizli tutunuz ve mesajı
>> gönderen kişiyi bilgilendirerek söz konusu mesaj ile eklerini derhal imha
>> ediniz. Bu mesaj ve ekindeki belgelerin bilinen virüslere karşı kontrolü
>> yapılmıştır. Ancak e-posta sistemlerinin taşıdığı risklerden dolayı
>> şirketimiz bu mesajın ve içerdiği bilgilerin size değişikliğe uğrayarak
>> veya geç ulaşmasından bütünlüğünün ve gizliliğinin korunamamasından virüs
>> içermesinden ve herhangi bir sebeple bilgisayarınıza ve sisteminize
>> verebileceği zararlardan sorumlu tutulamaz.”<<<<<
>>
>> “This message together with its attachments is intended solely for the
>> address(es) and may contain confidential or privileged information. If you
>> are not the intended recipient please do not use copy or disclose the
>> message for any purpose. Should you receive this message by mistake please
>> keep all information contained in the message or its attachments strictly
>> confidential and advise the sender and delete it immediately without
>> retaining a copy. This message and its attachments have been swept by
>> anti-virus systems for the presence of known viruses. However due to the
>> risks of e-mail systems our company cannot accept liability for any changes
>> or delay in receiving loss of integrity and confidentiality containing
>> viruses and any damages caused in any way to your computer and system
>> recipient, you are notified that disclosing, distributing, or copying this
>> e-mail is strictly prohibited. “
>>
>>
>>
>

Mime
View raw message