spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 小野圭二 <>
Subject Re: How to get the data url
Date Sat, 04 Nov 2017 01:04:52 GMT
Thank you for your reply, jgp.
The URL in my question was just a sample. Whatever it is.

I mean, let's image multi user environment spark.
This is just a case model.
 -I am a watcher on a spark system.
 -Some users run their applications on my spark and ,I need to know what
URL running on my spark.
 -Maybe You say I can see the URL in a log file, but it is troublesome.
 -If there is a API that get a access url string, I could make a program to
gather all running URL.

I believe there is the best API in it, but just i do not know yet.



2017-11-03 19:48 GMT+09:00 Jean Georges Perrin <>:

> I am a little confused by your question… Are you trying to ingest a file
> from S3?
> If so… look for net.jgp.labs.spark on GitHub and look for
> net.jgp.labs.spark.l000_ingestion.l001_csv_in_progress.S3CsvToDataset
> You can modify the file as the keys are yours…
> If you want to download first: look at net.jgp.labs.spark.l900_analytics.
> ListNCSchoolDistricts
> jgp
> On Oct 29, 2017, at 22:20, onoke <> wrote:
> Hi,
> I am searching a useful API for getting a data URL that is accessed by a
> application on Spark.
> For example, when this URL is in a application
>       new
> URL("")
> How to get this url from using Spark API?
> I looked in and org.apache.spark.status.api.v1, but
> they
> do not provide any URL info.
> Any advice are welcome.
> -Keiji
> --
> Sent from:
> ---------------------------------------------------------------------
> To unsubscribe e-mail:

View raw message