From dev-return-5680-apmail-samza-dev-archive=samza.apache.org@samza.apache.org Wed Mar 23 22:29:44 2016 Return-Path: X-Original-To: apmail-samza-dev-archive@minotaur.apache.org Delivered-To: apmail-samza-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 155AB18C0E for ; Wed, 23 Mar 2016 22:29:44 +0000 (UTC) Received: (qmail 94203 invoked by uid 500); 23 Mar 2016 22:29:43 -0000 Delivered-To: apmail-samza-dev-archive@samza.apache.org Received: (qmail 94141 invoked by uid 500); 23 Mar 2016 22:29:43 -0000 Mailing-List: contact dev-help@samza.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@samza.apache.org Delivered-To: mailing list dev@samza.apache.org Delivered-To: moderator for dev@samza.apache.org Received: (qmail 56852 invoked by uid 99); 23 Mar 2016 22:13:35 -0000 X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.879 X-Spam-Level: *** X-Spam-Status: No, score=3.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_BL_SPAMCOP_NET=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=uber.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uber.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=TDJDTlU4g7X5RVhNr7tCXykakKauPFmv7+6tTNoAlZs=; b=0UnrvFLQnQNq+nWnXJDHUogJ1Y5NBXhrqVe7ms7kSNHnBl2qcEU9xUzPOBo/qvfLA/ aypReMinu8EbcauI0tXgr28q83ml/bstMoRdB0T5Wg+E3dFGyS4lWH9xiOyL4Kfp3OGM cXjLOoEJaN0eepagwSR5UuS87tsjGALFcOCrg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=TDJDTlU4g7X5RVhNr7tCXykakKauPFmv7+6tTNoAlZs=; b=k6q5W9laxbWZmKkoKkpusrd3zKtSoAZi4XKuwsNITGN8V5WkuLGJe+96D9RcSoMBBL upAneYbYEWCf0+jNAFnc3sbKM9PycV9v2fAwk+xRCNNnG03oRHkc6QFfaAiMVrsMg03e GywOYWDZrXztosYe/JGLxhThOWFF/ltLtjYA/tUEUmSOyEwIbu+bZ0DaIF+n2mnQaOSy nfXqTAIMarvcvFwu1kvuZfFkAPGfZ7yXiFWeXMURjAs48dZi1hiEfFzOVkR8EBjoQR/y jcheRjH9xWZKcWRWT4zBZ6WVioA0OenNPogrmMzWSwvM1/Q6o0SaHYOR5PCkxGXAL0/v tsig== X-Gm-Message-State: AD7BkJKoL2SPGcAi23kD5FLpkPbO95vok8FUP+QmGy2m8PJ4o4Eo3LXjYmflSU1OUAKb+8Y8XgqUvarAf9xL6xru X-Received: by 10.129.131.82 with SMTP id t79mr2633775ywf.187.1458771209763; Wed, 23 Mar 2016 15:13:29 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: References: From: Yuanchi Ning Date: Wed, 23 Mar 2016 15:13:10 -0700 Message-ID: Subject: Re: Jackson null pointer when upgrading to Samza 0.10.0 To: Yi Pan Cc: dev@samza.apache.org, Athena Developers Content-Type: multipart/alternative; boundary=94eb2c07c5fac4beca052ebea3ce --94eb2c07c5fac4beca052ebea3ce Content-Type: text/plain; charset=UTF-8 Hi Yi, This is aonther issue. I overwrote it in our rapper library called athena-core-lib. The mvn dependency tree shows that it's picking up the version 1.9.13, plus the command line that runs the program shows 1.9.13. We are currently testing on different Yarn versions to see if that's related since our current version is 2.6.0. Thanks! Yuanchi On Wed, Mar 23, 2016 at 2:57 PM, Yi Pan wrote: > Hi, Yuanchi, > > Is this related w/ the issue you reported earlier regarding to "problem > picking up checkpoint after upgrade" in another thread? I assume that you > are using the official Samza 0.10 release? That has jackson version 1.8.5 > by default. How do you change it in your own build/package to 1.9.13? > > Thanks! > > -Yi > > On Wed, Mar 23, 2016 at 12:00 PM, Yuanchi Ning > wrote: > >> Hi All, >> >> When we upgrade our existing samza job from 0.9 to 0.10, we found this >> error below: >> >> >> /var/lib/hadoop-yarn/data/samza-yarn/usercache/yuanchi/appcache/application_1458330755534_0075/container_1458330755534_0075_02_000001/__package/bin/run-class.sh: >> line 63: OpenJDK: command not found >> java version "1.7.0_95" >> OpenJDK Runtime Environment (IcedTea 2.6.4) (7u95-2.6.4-0ubuntu0.12.04.1) >> OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode) >> Exception in thread "main" java.lang.NullPointerException >> at java.io.StringReader.(StringReader.java:50) >> at >> org.codehaus.jackson.JsonFactory.createJsonParser(JsonFactory.java:636) >> at >> org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1863) >> at >> org.apache.samza.job.yarn.SamzaAppMaster$.main(SamzaAppMaster.scala:71) >> at >> org.apache.samza.job.yarn.SamzaAppMaster.main(SamzaAppMaster.scala) >> >> >> And that line 71 in SamzaAppMaster is: >> >> val coordinatorSystemConfig = new >> >> MapConfig(SamzaObjectMapper.getObjectMapper.readValue(System.getenv(ShellCommandConfig.ENV_COORDINATOR_SYSTEM_CONFIG), >> classOf[Config])) >> >> And our checkpoint/ new version related properties are as below: >> >> ##################### Job config ##################### >> >> job.factory.class=org.apache.samza.job.yarn.YarnJobFactory >> >> job.name=trip-counter >> >> job.datacenter=sjc1 >> >> job.environment=sandbox >> >> job.coordinator.system=kafka >> >> job.coordinator.replication.factor=3 >> >> >> ##################### Task config ##################### >> >> task.class=com.uber.athena.TripCounterTask >> >> task.inputs=kafka.trip_details,kafka.hp-api-client_signups >> >> task.outputTripTopic=trip_count_details >> >> task.outputClientSignUpsTopic=client_sign_ups_count_details >> >> >> task.checkpoint.factory=org.apache.samza.checkpoint.kafka.KafkaCheckpointManagerFactory >> >> task.checkpoint.system=kafka >> >> task.checkpoint.replication.factor=3 >> >> >> Plus our version for codehaus jackson dependencies is 1.9.13 as >> suggested in other dev email threads. >> >> >> Any ideas what would cause this issue? >> >> Thanks! >> >> Yuanchi >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> -- >> Yuanchi Ning >> > > --94eb2c07c5fac4beca052ebea3ce--