cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Liu (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-6091) Better Vnode support in hadoop/pig
Date Mon, 14 Oct 2013 21:05:42 GMT


Alex Liu commented on CASSANDRA-6091:

The following code
   List<TokenRange> masterRangeNodes = getRangeMap(conf);
returns all the token ranges. We need find a way to merge the token ranges into bigger token
ranges and keep the replica locations no change.

Merging token ranges helps reduce the number of splits. The reduction rate depends on how
random the token ranges are shuffled around the ring. It helps a lot if we could find a better
shuffle algorithm to maximum the merging.

> Better Vnode support in hadoop/pig
> ----------------------------------
>                 Key: CASSANDRA-6091
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Hadoop
>            Reporter: Alex Liu
>            Assignee: Alex Liu
> CASSANDRA-6084 shows there are some issues during running hadoop/pig job if vnodes are
enable. Also the hadoop performance of vnode enabled nodes  are bad for there are so many
> The idea is to combine vnode splits into a big sudo splits so it work like vnode is disable
for hadoop/pig job

This message was sent by Atlassian JIRA

View raw message