mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Angelo Immediata <>
Subject Write SequenceFile from custom data
Date Mon, 02 Dec 2013 09:31:15 GMT

I'm pretty newbie regarding learning achine and above all Apache Mahout, so
pardon me my low level questions

I need to do some cluster analysis by using some data. At the beginning
this data can be not too much huge, but after some time they can be really
huge (I did some calculation and after 1 year this data cann be around 37
billion of records) Since I have this huge data, I decided to do the
cluster analysis by using Mahout on the top of Apache Hadoop and its HDFS.
Regarding where to store this big amount of data I decided to use Apache
HBase always on the top of Apache Hadoop HDFS

Now I need to do this cluster analysi by considering some environment
variables. These variable may be the following:

   - *recordId* = id of the record
   - *arcId *= id of the arc between 2 points of my "street graph"
   - *mediumVelocity *= medium velocity of the considered arc in the
   - *vehiclesNumber* = number of the monitored vehicles in order to get
   that velocity
   - *meteo *= weather condition (a numeric representing if there is sun,
   rain etc...)
   - *manifestation *= a numeric representing if there is any kind of
   manifestation (sport manifestation or other)
   - *day of the week*
   - *month of the year*
   - *hour of the day*
   - *vacation *= a numeric representing if it's a vacation day or a
   working day

So my data are so formatted (raw representation):

*recordId arcId mediumVelocity vehiclesNumber meteo manifestation
weekDay yearMonth dayHour vacation*
1         1      34.5            20            1      3            4
   2011       10      3
2         156    66.5            3             2      5            1
   2008        6      2

As far as I know, in order to do the cluster analysis in Mahout I need to
format my data in Mahout format (that is in a SequenceFile) The question
is: how can I format my data represented as the previously written table in
a SequenceFile? I tried to find something but I was not able in finding any
good sample Any suggestion would be really appreciated

Thank you Angelo

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message