From user-return-4205-apmail-thrift-user-archive=thrift.apache.org@thrift.apache.org Sat May 16 18:50:56 2015 Return-Path: X-Original-To: apmail-thrift-user-archive@www.apache.org Delivered-To: apmail-thrift-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1FE0D10B1C for ; Sat, 16 May 2015 18:50:56 +0000 (UTC) Received: (qmail 6929 invoked by uid 500); 16 May 2015 18:50:50 -0000 Delivered-To: apmail-thrift-user-archive@thrift.apache.org Received: (qmail 6885 invoked by uid 500); 16 May 2015 18:50:50 -0000 Mailing-List: contact user-help@thrift.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@thrift.apache.org Delivered-To: mailing list user@thrift.apache.org Received: (qmail 6870 invoked by uid 99); 16 May 2015 18:50:50 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 16 May 2015 18:50:50 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id E8C511A2DA7 for ; Sat, 16 May 2015 18:50:49 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.879 X-Spam-Level: ** X-Spam-Status: No, score=2.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id 0lteqCdzRFPm for ; Sat, 16 May 2015 18:50:49 +0000 (UTC) Received: from mail-oi0-f52.google.com (mail-oi0-f52.google.com [209.85.218.52]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id 3897223164 for ; Sat, 16 May 2015 18:50:48 +0000 (UTC) Received: by oica37 with SMTP id a37so104389876oic.0 for ; Sat, 16 May 2015 11:50:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=drnZ05Y+Grhg+/pTcVczjLQeo9R0Bw0mnTaXQLSSAKg=; b=d2+va5aY0QjzRcrG6mOieiw0zcpkf6SioiJEu1VIN6U8hSOcFxGKAOAP8NqwhDzCm0 FfNxcsLTOeFUJCx2qJNHFoOpLWbtbSLX5Jt3mks4PeCzYscbzraDm81gca3VCPWr+sTT wV9Su+guMvZm7uzsgYHEZjhSXv+l8XwIhqBJC5Aghu4UmVg0P8pNAUsk8mr84HiN8oeC 4B2rqUD5P2954gQZ1mS0CMv8DDrB4TbMBwxHUtosRi0zi5kzXPnPfms/uSOsCl9A0O3a KM4vbcBfXvS0ItfkTqmnF+SDPGVaeWfPhjFOupHMRig0YuknA4fe9CY+E31LZFiRLdN6 O4oQ== MIME-Version: 1.0 X-Received: by 10.202.206.144 with SMTP id e138mr13163116oig.132.1431802246693; Sat, 16 May 2015 11:50:46 -0700 (PDT) Received: by 10.202.231.148 with HTTP; Sat, 16 May 2015 11:50:46 -0700 (PDT) Date: Sat, 16 May 2015 14:50:46 -0400 Message-ID: Subject: String encoding in Haskell Thrift From: Andreas Voellmy To: user@thrift.apache.org Content-Type: multipart/alternative; boundary=001a113ad9f04d874f05163770fc --001a113ad9f04d874f05163770fc Content-Type: text/plain; charset=UTF-8 Hi all, I've got a service written in one language (C++ or Python) that has a service API with a string parameter that really is a byte array where the bytes can have values > 127. The server does not seem to use UTF8 for decoding. Then I've got a client using the Haskell thrift library + generated code sending the byte array as string to the server. The problem is that the Haskell code generated by Thrift first encodes all strings using UTF8, which are then interpreted badly on the server side whenever the byte values are over 127. I noticed a long discussion [1] relating to exactly the same issue noticing that Java does the UTF8 encoding where Python doesn't. I think this is the same issue, this time with Haskell. For the time being, I've changed the Haskell code-generator so that the Haskell code does not apply UTF encoding. What is the right way to resolve this issue of sending a byte array from one thrift client to another? [1] https://issues.apache.org/jira/browse/THRIFT-395 -Andi --001a113ad9f04d874f05163770fc--