phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amarnath Ramamoorthi (JIRA)" <j...@apache.org>
Subject [jira] [Created] (PHOENIX-5097) Index Scrutiny Tool changes schema name to UPPERCASE
Date Fri, 11 Jan 2019 16:50:00 GMT
Amarnath Ramamoorthi created PHOENIX-5097:
---------------------------------------------

             Summary: Index Scrutiny Tool changes schema name to UPPERCASE
                 Key: PHOENIX-5097
                 URL: https://issues.apache.org/jira/browse/PHOENIX-5097
             Project: Phoenix
          Issue Type: Bug
            Reporter: Amarnath Ramamoorthi


Creating index table and runningĀ *Index Scrutiny Tool.*
{code:java}
CREATE INDEX IF NOT EXISTS "IDX_CAP_DEMO_TABLE" ON "CAP".DEMO_TABLE ("id_x") INCLUDE ("id_y");

hbase org.apache.phoenix.mapreduce.index.IndexScrutinyTool -s CAP -dt DEMO_TABLE -it IDX_FOO_DEMO_TABLE
-o
{code}
Works fine without error, since the actual schema name is in uppercase (CAP).

However on using lowercase schema name (nocap) it converts it to UPPERCASE.
{code:java}
CREATE INDEX IF NOT EXISTS "IDX_NOCAP_DEMO_TABLE" ON "nocap".DEMO_TABLE ("id_x") INCLUDE ("id_x");

[amar@locahost ~]$ hbase org.apache.phoenix.mapreduce.index.IndexScrutinyTool -s nocap -dt
DEMO_TABLE -it IDX_NOCAP_DEMO_TABLE -o
...........
...........
...........
19/01/11 13:45:09 INFO Configuration.deprecation: hadoop.native.lib is deprecated. Instead,
use io.native.lib.available
19/01/11 13:45:11 ERROR index.IndexScrutinyTool: An exception occurred while performing the
indexing job: IllegalArgumentException:  IDX_NOCAP_DEMO_TABLE is not an index table for NOCAP.DEMO_TABLE
 at:
java.lang.IllegalArgumentException:  IDX_NOCAP_DEMO_TABLE is not an index table for NOCAP.DEMO_TABLE

	at org.apache.phoenix.mapreduce.index.IndexScrutinyTool.run(IndexScrutinyTool.java:394)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
	at org.apache.phoenix.mapreduce.index.IndexScrutinyTool.main(IndexScrutinyTool.java:518)
{code}
It changes the schema name to uppercase, so input the schema name with quotes to ignore the
case like
{code:java}
hbase org.apache.phoenix.mapreduce.index.IndexScrutinyTool -s "nocap" -dt DEMO_TABLE -it IDX_NOCAP_DEMO_TABLE
-o
{code}
The same error follows when you use the above command with just quotes. On using the following
command
{code:java}
[amar@locahost ~]$ hbase org.apache.phoenix.mapreduce.index.IndexScrutinyTool -s \"\"nocap\"\"
-dt DEMO_TABLE -it IDX_NOCAP_DEMO_TABLE -o
...........
...........
...........
19/01/11 10:34:18 INFO Configuration.deprecation: hadoop.native.lib is deprecated. Instead,
use io.native.lib.available
19/01/11 10:34:20 INFO index.IndexScrutinyTool: Running scrutiny [schemaName="nocap", dataTable=DEMO_TABLE,
indexTable=IDX_NOCAP_DEMO_TABLE, useSnapshot=false, timestamp=1547202800130, batchSize=1000,
outputBasePath=null, outputFormat=TABLE, outputMaxRows=1000000]
19/01/11 10:34:20 INFO index.IndexScrutinyTool: Query used on source table to feed the mapper:
SELECT /*+ NO_INDEX */ "id_x","id_y","id_z" FROM NOCAP.DEMO_TABLE
19/01/11 10:34:20 INFO index.IndexScrutinyTool: Upsert statement used for output table: UPSERT
 INTO PHOENIX_INDEX_SCRUTINY ("SOURCE_TABLE", "TARGET_TABLE", "SCRUTINY_EXECUTE_TIME", "SOURCE_ROW_PK_HASH",
"SOURCE_TS", "TARGET_TS", "HAS_TARGET_ROW", "id_x","id_y","id_z" .... ) VALUES (?, ?, ?, ?,
?, ?)
19/01/11 10:34:20 INFO index.IndexScrutinyTool: Query used on source table to feed the mapper:
SELECT /*+ NO_INDEX */ "id_x","id_y","id_z" FROM NOCAP.IDX_NOCAP_DEMO_TABLE
19/01/11 10:34:20 INFO index.IndexScrutinyTool: Upsert statement used for output table: UPSERT
 INTO PHOENIX_INDEX_SCRUTINY ("SOURCE_TABLE", "TARGET_TABLE", "SCRUTINY_EXECUTE_TIME", "SOURCE_ROW_PK_HASH",
"SOURCE_TS", "TARGET_TS", "HAS_TARGET_ROW", "id_x","id_y","id_z" ..... ) VALUES (?, ?, ?,
?, ?, ?)
19/01/11 10:34:21 INFO index.IndexScrutinyTool: Running Index Scrutiny in Background - Submit
async and exit
19/01/11 10:34:23 ERROR mapreduce.PhoenixInputFormat: Failed to get the query plan with error
[ERROR 1012 (42M03): Table undefined. tableName=NOCAP.DEMO_TABLE]
19/01/11 10:34:23 INFO mapreduce.JobSubmitter: Cleaning up the staging area /user/amar/.staging/job_1540390314309_0119
19/01/11 10:34:23 ERROR index.IndexScrutinyTool: An exception occurred while performing the
indexing job: RuntimeException: org.apache.phoenix.schema.TableNotFoundException: ERROR 1012
(42M03): Table undefined. tableName=NOCAP.DEMO_TABLE at:
java.lang.RuntimeException: org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03):
Table undefined. tableName=NOCAP.DEMO_TABLE
	at org.apache.phoenix.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:139)
	at org.apache.phoenix.mapreduce.PhoenixInputFormat.getSplits(PhoenixInputFormat.java:81)
	at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:305)
	at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:322)
	at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200)
	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304)
	at org.apache.phoenix.mapreduce.index.IndexScrutinyTool.run(IndexScrutinyTool.java:440)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
	at org.apache.phoenix.mapreduce.index.IndexScrutinyTool.main(IndexScrutinyTool.java:518)
Caused by: org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03): Table undefined.
tableName=NOCAP.DEMO_TABLE
	at org.apache.phoenix.compile.FromCompiler$BaseColumnResolver.createTableRef(FromCompiler.java:577)
	at org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.<init>(FromCompiler.java:391)
	at org.apache.phoenix.compile.FromCompiler.getResolverForQuery(FromCompiler.java:228)
	at org.apache.phoenix.compile.FromCompiler.getResolverForQuery(FromCompiler.java:206)
	at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:468)
	at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:442)
	at org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1679)
	at org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1672)
	at org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1666)
	at org.apache.phoenix.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:120)
	... 14 more
{code}
As you can see it reads as lowercase, if you look at the second line form the above output,
which isĀ 
{code:java}
19/01/11 10:34:20 INFO index.IndexScrutinyTool: Running scrutiny [schemaName="nocap", dataTable=DEMO_TABLE,
indexTable=IDX_NOCAP_DEMO_TABLE, useSnapshot=false, timestamp=1547202800130, batchSize=1000,
outputBasePath=null, outputFormat=TABLE, outputMaxRows=1000000]
{code}
and the very next line
{code:java}
19/01/11 10:34:20 INFO index.IndexScrutinyTool: Query used on source table to feed the mapper:
SELECT /*+ NO_INDEX */ "id_x","id_y","id_z" FROM NOCAP.DEMO_TABLE
{code}
it changes the schema name to uppercase.

Tried other phoenix suggested option to run this "Index Scrutiny Tool" from Hadoop using phoenix-server.jar
{code:java}
HADOOP_CLASSPATH=$(hbase mapredcp) hadoop jar phoenix4.13-cdh5.9.2-marin-1.5.1-server.jar
org.apache.phoenix.mapreduce.index.IndexScrutinyTool -s \"\"nocap\"\" -dt DEMO_TABLE -it IDX_NOCAP_DEMO_TABLE
-o
{code}
However, I believe this ends up with the same error, since using the same module "org.apache.phoenix.mapreduce.index.IndexScrutinyTool"

Hope there is enough information here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message