ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Asger Askov Blekinge (JIRA)" <j...@apache.org>
Subject [jira] [Created] (RANGER-1243) One failed embedded ServiceDef creating prevent any service from being created in Ranger
Date Tue, 06 Dec 2016 11:11:29 GMT
Asger Askov Blekinge created RANGER-1243:
--------------------------------------------

             Summary: One failed embedded ServiceDef creating prevent any service from being
created in Ranger
                 Key: RANGER-1243
                 URL: https://issues.apache.org/jira/browse/RANGER-1243
             Project: Ranger
          Issue Type: Bug
          Components: admin, plugins, Ranger
    Affects Versions: 0.6.2, 0.6.1, 0.6.0, 0.5.2, 0.7.0, 0.6.3
         Environment: IBM IOP 4.2. Only reproduced on Ranger 0.5.2 but the relevant code seems
unchanged in newer versions. 
            Reporter: Asger Askov Blekinge


EmbeddedServiceDefsUtil have this method (I removed some log statements to make the code more
clear)

{code}
public class EmbeddedServiceDefsUtil {
    ...
	private RangerServiceDef getOrCreateServiceDef(ServiceStore store, String serviceDefName)
{
		....
		RangerServiceDef ret = null;
		boolean createServiceDef = (CollectionUtils.isEmpty(supportedServiceDefs) || supportedServiceDefs.contains(serviceDefName));
		try {
			ret = store.getServiceDefByName(serviceDefName);
			if(ret == null && createEmbeddedServiceDefs && createServiceDef) {
				ret = loadEmbeddedServiceDef(serviceDefName);
				if (ret.getId() != null) {
					store.setPopulateExistingBaseFields(true);
					ret = store.createServiceDef(ret);
					store.setPopulateExistingBaseFields(false);
				} else {
					ret = store.createServiceDef(ret);
				}
				LOG.info("created embedded service-def " + serviceDefName);
			}
		} catch(Exception excp) {
			LOG.fatal("EmbeddedServiceDefsUtil.getOrCreateServiceDef(): failed to load/create serviceType
" + serviceDefName, excp);
		}
		...
		return ret;
	}
}
{code}

When creating a service def, the flag PopulateExistingBaseFields must be set to true, but
it should be set to false afterwards.

If the statement ``` ret = store.createServiceDef(ret); ``` throws an exception, the flag
is never reset. But since the catch statement do not rethrow the exception, null will be returned
and Ranger will start normally.

Now, the flag PopulateExistingBaseFields are used by a lot of methods in ServiceDBStore

* public RangerServiceDef **createServiceDef**(RangerServiceDef serviceDef) throws Exception
* public RangerService **createService**(RangerService service) throws Exception
* public RangerService **updateService**(RangerService service) throws Exception
* public RangerPolicy **createPolicy**(RangerPolicy policy) throws Exception {

For createService and updateService, it will cause the method to select between svcServiceWithAssignedId
and svcService. svcServiceWithAssignedId require that the incoming service have an ID. Services
created via the web gui and via the ambari integration do NOT have a pre-selected ID. They
cannot be created, and thus you cannot actually use Ranger. 


Now, how can ```ret = store.createServiceDef(ret);``` fail? I do not know, but it happened
on my system with this exception log entry

    2016-12-06 09:34:25,850 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil
(EmbeddedServiceDefsUtil.java:136) - ==> EmbeddedServiceDefsUtil.getOrCreateServiceDef(kms)
    
    2016-12-06 09:34:25,850 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.biz.ServiceDBStore
(ServiceDBStore.java:934) - ==> ServiceDefDBStore.getServiceDefByName(kms)
    
    2016-12-06 09:34:25,855 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.biz.ServiceDBStore
(ServiceDBStore.java:946) - == ServiceDefDBStore.getServiceDefByName(kms): null
    
    2016-12-06 09:34:25,855 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil
(EmbeddedServiceDefsUtil.java:168) - ==> EmbeddedServiceDefsUtil.loadEmbeddedServiceDef(kms)
    
    2016-12-06 09:34:25,867 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil
(EmbeddedServiceDefsUtil.java:182) - ==> EmbeddedServiceDefsUtil.loadEmbeddedServiceDef(kms)
    
    2016-12-06 09:34:25,867 [kact-man-001.kact.sblokalnet-startStop-1] INFO  org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil
(EmbeddedServiceDefsUtil.java:146) - creating embedded service-def kms
    
    2016-12-06 09:34:25,868 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.biz.ServiceDBStore
(ServiceDBStore.java:242) - ==> ServiceDefDBStore.createServiceDef(RangerServiceDef={id={7}
guid={null} isEnabled={true} createdBy={null} updatedBy={null} createTime={null} updateTime={null}
version={null} name={kms} implClass={org.apache.ranger.services.kms.RangerServiceKMS} label={KMS}
description={KMS} rbKeyLabel={null} rbKeyDescription={null} c
    onfigs={RangerServiceConfigDef={itemId={provider} name={provider} type={string} subType={null}
mandatory={true} defaultValue={null} validationRegEx={null} validationMessage={null} uiHint={null}
label={KMS URL} description={null} rbKeyLabel={null} rbKeyDescription={null} rbKeyValidationMessage={null}
}RangerServiceConfigDef={itemId={username} name={username} type={string} subType={null} mandatory={true}
defaultValue={null} validationRegEx={null} validationM
    essage={null} uiHint={null} label={Username} description={null} rbKeyLabel={null} rbKeyDescription={null}
rbKeyValidationMessage={null} }RangerServiceConfigDef={itemId={password} name={password} type={password}
subType={null} mandatory={true} defaultValue={null} validationRegEx={null} validationMessage={null}
uiHint={null} label={Password} description={null} rbKeyLabel={null} rbKeyDescription={null}
rbKeyValidationMessage={null} }} resources={RangerResourc
    eDef={itemId={1} name={keyname} type={string} level={10} parent={} mandatory={true} lookupSupported={true}
recursiveSupported={false} excludesSupported={false} matcher={org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher}
matcherOptions={{wildCard=true, ignoreCase=false}} validationRegEx={} validationMessage={}
uiHint={} label={Key Name} description={Key Name} rbKeyLabel={null} rbKeyDescription={null}
rbKeyValidationMessage={null} }} acce
    ssTypes={RangerAccessTypeDef={itemId={1} name={create} label={Create} rbKeyLabel={null}
impliedGrants={} }RangerAccessTypeDef={itemId={2} name={delete} label={Delete} rbKeyLabel={null}
impliedGrants={} }RangerAccessTypeDef={itemId={3} name={rollover} label={Rollover} rbKeyLabel={null}
impliedGrants={} }RangerAccessTypeDef={itemId={4} name={setkeymaterial} label={Set Key Material}
rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={5} name={get
    } label={Get} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={6} name={getkeys}
label={Get Keys} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={7} name={getmetadata}
label={Get Metadata} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={8} name={generateeek}
label={Generate EEK} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={9} name={decrypteek}
label={Decrypt EEK} rbKeyLabel={null} implie
    dGrants={} }} policyConditions={} contextEnrichers={} enums={} })
    
    2016-12-06 09:34:25,870 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.common.db.BaseDao
(BaseDao.java:239) - Ignoring BaseDao.setIdentityInsert(). This should be executed if DB flavor
is sqlserver.
    
    2016-12-06 09:34:25,872 [kact-man-001.kact.sblokalnet-startStop-1] FATAL org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil
(EmbeddedServiceDefsUtil.java:156) - EmbeddedServiceDefsUtil.getOrCreateServiceDef(): failed
to load/create serviceType kms
    java.lang.NullPointerException
            at org.apache.ranger.service.RangerServiceDefServiceBase.mapViewToEntityBean(RangerServiceDefServiceBase.java:158)
            at org.apache.ranger.service.RangerServiceDefWithAssignedIdService.mapViewToEntityBean(RangerServiceDefWithAssignedIdService.java:31)
            at org.apache.ranger.service.RangerServiceDefWithAssignedIdService.mapViewToEntityBean(RangerServiceDefWithAssignedIdService.java:25)
            at org.apache.ranger.service.RangerBaseModelService.populateEntityBeanForCreate(RangerBaseModelService.java:180)
            at org.apache.ranger.service.RangerBaseModelService.preCreate(RangerBaseModelService.java:217)
            at org.apache.ranger.service.RangerBaseModelService.create(RangerBaseModelService.java:225)
            at org.apache.ranger.biz.ServiceDBStore.createServiceDef(ServiceDBStore.java:268)
            at org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil.getOrCreateServiceDef(EmbeddedServiceDefsUtil.java:149)
            at org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil.init(EmbeddedServiceDefsUtil.java:96)
            at org.apache.ranger.biz.ServiceDBStore$1.doInTransaction(ServiceDBStore.java:223)
            at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130)
            at org.apache.ranger.biz.ServiceDBStore.initStore(ServiceDBStore.java:220)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:498)
            at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:346)
            at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:299)
            at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:132)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:394)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1448)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
            at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
            at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
            at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
            at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
            at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:872)
            at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:814)
            at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:731)
            at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:485)
            at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:92)
            at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:284)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1106)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
            at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
            at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
            at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
            at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
            at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:872)
            at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:814)
            at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:731)
            at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:485)
            at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:92)
            at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:284)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1106)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
            at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
            at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
            at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
            at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
            at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:872)
            at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:814)
            at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:731)
            at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:485)
            at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:92)
            at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:284)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1106)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
            at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
            at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
            at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
            at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
            at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:872)
            at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:814)
            at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:731)
            at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:485)
            at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:92)
            at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:284)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1106)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)
            at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
            at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
            at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
            at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
            at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
            at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:605)
            at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:925)
            at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:472)
            at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:383)
            at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:283)
            at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:111)
            at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4992)
            at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5490)
            at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
            at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1575)
            at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1565)
            at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
            at java.lang.Thread.run(Thread.java:745)
    
    2016-12-06 09:34:25,875 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil
(EmbeddedServiceDefsUtil.java:160) - <== EmbeddedServiceDefsUtil.getOrCreateServiceDef(kms):
RangerServiceDef={id={7} guid={null} isEnabled={true} createdBy={null} updatedBy={null} createTime={null}
updateTime={null} version={1} name={kms} implClass={org.apache.ranger.services.kms.RangerServiceKMS}
label={KMS} description={KMS} rbKeyLabel={null} rbKeyDescription={null} configs={RangerServiceConfigDef={itemId={provider}
name={provider} type={string} subType={null} mandatory={true} defaultValue={null} validationRegEx={null}
validationMessage={null} uiHint={null} label={KMS URL} description={null} rbKeyLabel={null}
rbKeyDescription={null} rbKeyValidationMessage={null} }RangerServiceConfigDef={itemId={username}
name={username} type={string} subType={null} mandatory={true} defaultValue={null} validationRegEx={null}
validationMessage={null} uiHint={null} label={Username} description={null} rbKeyLabel={null}
rbKeyDescription={null} rbKeyValidationMessage={null} }RangerServiceConfigDef={itemId={password}
name={password} type={password} subType={null} mandatory={true} defaultValue={null} validationRegEx={null}
validationMessage={null} uiHint={null} label={Password} description={null} rbKeyLabel={null}
rbKeyDescription={null} rbKeyValidationMessage={null} }} resources={RangerResourceDef={itemId={1}
name={keyname} type={string} level={10} parent={} mandatory={true} lookupSupported={true}
recursiveSupported={false} excludesSupported={false} matcher={org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher}
matcherOptions={{wildCard=true, ignoreCase=false}} validationRegEx={} validationMessage={}
uiHint={} label={Key Name} description={Key Name} rbKeyLabel={null} rbKeyDescription={null}
rbKeyValidationMessage={null} }} accessTypes={RangerAccessTypeDef={itemId={1} name={create}
label={Create} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={2} name={delete}
label={Delete} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={3} name={rollover}
label={Rollover} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={4} name={setkeymaterial}
label={Set Key Material} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={5}
name={get} label={Get} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={6}
name={getkeys} label={Get Keys} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={7}
name={getmetadata} label={Get Metadata} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={8}
name={generateeek} label={Generate EEK} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={9}
name={decrypteek} label={Decrypt EEK} rbKeyLabel={null} impliedGrants={} }} policyConditions={}
contextEnrichers={} enums={} }





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message