nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From PuspenduBanerjee <...@git.apache.org>
Subject [GitHub] nifi pull request: NIFI-924:Nifi-Camel Integration
Date Mon, 01 Feb 2016 01:17:22 GMT
Github user PuspenduBanerjee commented on the pull request:

    https://github.com/apache/nifi/pull/197#issuecomment-177686181
  
    Hi @joewitt 
    For now the issue is happening for org.apache.nifi.processors.standard.TestExecuteStreamCommand.testExecuteIngestAndUpdate().
    Where we are running a Pattern on 100MB data. So, I have modified the testcase with a
sliding window matching mechanism. I hope it will work.:
    
        /**
         * Generates a String representation by taking a slice of backing byte[] from MockFile.
         * It's a low memory foot-print alternative to {@code new String(mockFile.toByteArray)}
         * @param mockFlowFile 
         * @param patternToMatch
         * @return
         * @throws NoSuchFieldException
         * @throws SecurityException
         * @throws IllegalArgumentException
         * @throws IllegalAccessException
         */
        private boolean matchPattern(MockFlowFile mockFlowFile, final String  patternToMatch)
throws NoSuchFieldException, SecurityException, 
                                                                IllegalArgumentException,
IllegalAccessException {
            final Pattern pattern= Pattern.compile(patternToMatch);
            Field dataField=MockFlowFile.class.getDeclaredField("data");
            dataField.setAccessible(true);
            byte[] byteArray = (byte[])dataField.get(mockFlowFile);
            final int sliceLength=patternToMatch.length()>(10240)?patternToMatch.length():(10240);
            final int windowLength=sliceLength-patternToMatch.length();
            int offset=0;
            StringBuilder stringBuilder=new StringBuilder();
            while(offset<=byteArray.length){
                if(pattern.matcher(stringBuilder.append(new String(Arrays.copyOfRange(byteArray,
offset, (offset+sliceLength))))).find()){
                    return true;
                }
                stringBuilder.delete(0, windowLength);
                offset+=sliceLength;
            };
            return false;
        }


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message