I have a question and was wondering if there is a way to configure
Apache and/or some combination of mods to help in the following:
I have a Data Warehouse vendor that has many, many terabytes of
information. The data is organized in zip files with each zip containing
many different individual data files. When a request is received for
the system determines the correct zip file and downloads it. The zip
is then unzipped onto local disk and cached for use by other users.
This part of the system is working as expected.
Many times the user requests are for different data files that just
to be in the same zip file. So, I end up with many requests to my
service for the same file. This is putting a lot of pressure on their
and wasting a lot of bandwidth. Now I know that I can write a server
will proxy and cache all the requests to the vendor and block on
for the same zip file. Unfortunately, I don't have time in the short
write the server, get it through QA and deployed in time.
I can get another server out in the system to act as this proxy/cache
Is there a way to configure Apache (maybe using mod-cache and/or
to not only cache completed requests but also force it block on
ones for the same zip file? Is there a better way that is pretty
achieve this kind of concurrent caching with Apache?
Thanks for taking the time to read this and respond.
Email me with any questions.