flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [flink] bowenli86 commented on a change in pull request #8449: [FLINK-12235][hive] Support partition related operations in HiveCatalog
Date Fri, 17 May 2019 17:20:43 GMT
bowenli86 commented on a change in pull request #8449: [FLINK-12235][hive] Support partition
related operations in HiveCatalog
URL: https://github.com/apache/flink/pull/8449#discussion_r285215791
 
 

 ##########
 File path: flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/table/catalog/hive/HiveCatalog.java
 ##########
 @@ -200,44 +200,56 @@ protected Table createHiveTable(ObjectPath tablePath, CatalogBaseTable
table) {
 	// ------ partitions ------
 
 	@Override
-	public void createPartition(ObjectPath tablePath, CatalogPartitionSpec partitionSpec, CatalogPartition
partition, boolean ignoreIfExists)
-			throws TableNotExistException, TableNotPartitionedException, PartitionSpecInvalidException,
PartitionAlreadyExistsException, CatalogException {
-		throw new UnsupportedOperationException();
-	}
-
-	@Override
-	public void dropPartition(ObjectPath tablePath, CatalogPartitionSpec partitionSpec, boolean
ignoreIfNotExists)
-			throws PartitionNotExistException, CatalogException {
-		throw new UnsupportedOperationException();
-	}
-
-	@Override
-	public void alterPartition(ObjectPath tablePath, CatalogPartitionSpec partitionSpec, CatalogPartition
newPartition, boolean ignoreIfNotExists)
-			throws PartitionNotExistException, CatalogException {
-		throw new UnsupportedOperationException();
-	}
-
-	@Override
-	public List<CatalogPartitionSpec> listPartitions(ObjectPath tablePath)
-			throws TableNotExistException, TableNotPartitionedException, CatalogException {
-		throw new UnsupportedOperationException();
+	protected void validateCatalogPartition(CatalogPartition catalogPartition) throws CatalogException
{
+		if (!(catalogPartition instanceof HiveCatalogPartition)) {
+			throw new CatalogException(String.format("%s can only handle %s but got %s", getClass().getSimpleName(),
+				HiveCatalogPartition.class.getSimpleName(), catalogPartition.getClass().getSimpleName()));
+		}
 	}
 
 	@Override
-	public List<CatalogPartitionSpec> listPartitions(ObjectPath tablePath, CatalogPartitionSpec
partitionSpec)
-			throws TableNotExistException, TableNotPartitionedException, CatalogException {
-		throw new UnsupportedOperationException();
-	}
+	protected Partition createHivePartition(Table hiveTable, CatalogPartitionSpec partitionSpec,
CatalogPartition catalogPartition)
+		throws PartitionSpecInvalidException {
+		Partition partition = new Partition();
+		List<String> partCols = getFieldNames(hiveTable.getPartitionKeys());
+		List<String> partValues = getFullPartitionValues(new ObjectPath(hiveTable.getDbName(),
hiveTable.getTableName()),
+			partitionSpec, partCols);
+		// validate partition values
+		for (int i = 0; i < partCols.size(); i++) {
+			if (StringUtils.isNullOrWhitespaceOnly(partValues.get(i))) {
+				throw new PartitionSpecInvalidException(catalogName, partCols,
+					new ObjectPath(hiveTable.getDbName(), hiveTable.getTableName()), partitionSpec);
+			}
+		}
+		HiveCatalogPartition hiveCatalogPartition = (HiveCatalogPartition) catalogPartition;
+		partition.setValues(partValues);
+		partition.setDbName(hiveTable.getDbName());
+		partition.setTableName(hiveTable.getTableName());
+		partition.setCreateTime((int) (System.currentTimeMillis() / 1000));
+		partition.setParameters(hiveCatalogPartition.getProperties());
+		partition.setSd(hiveTable.getSd().deepCopy());
+
+		String location = hiveCatalogPartition.getLocation();
+		if (null == location) {
 
 Review comment:
   I see. I think we can address it by using location of old partition if the new partition
doesn't have it.  We already retrieved old partition from HMS in `partitionExist()` (as a
result we may need to refactor how alterPartition() checks if a partition exists or not).
It's similar to how `alterTable()` works.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message