www-announce mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajini Sivaram <rsiva...@apache.org>
Subject [ANNOUNCE] Apache Kafka 2.0.0 Released
Date Mon, 30 Jul 2018 10:25:35 GMT
The Apache Kafka community is pleased to announce the release for

Apache Kafka 2.0.0.

This is a major release and includes significant new features from

40 KIPs. It contains fixes and improvements from 246 JIRAs, including

a few critical bugs. Here is a summary of some notable changes:

** KIP-290 adds support for prefixed ACLs, simplifying access control
management in large secure deployments. Bulk access to topics,
consumer groups or transactional ids with a prefix can now be granted
using a single rule. Access control for topic creation has also been
improved to enable access to be granted to create specific topics or
topics with a prefix.

** KIP-255 adds a framework for authenticating to Kafka brokers using
OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
customizable using callbacks for token retrieval and validation.

**Host name verification is now enabled by default for SSL connections
to ensure that the default SSL configuration is not susceptible to
man-in-the middle attacks. You can disable this verification for
deployments where validation is performed using other mechanisms.

** You can now dynamically update SSL trust stores without broker restart.
You can also configure security for broker listeners in ZooKeeper before
starting brokers, including SSL key store and trust store passwords and
JAAS configuration for SASL. With this new feature, you can store sensitive
password configs in encrypted form in ZooKeeper rather than in cleartext
in the broker properties file.

** The replication protocol has been improved to avoid log divergence
between leader and follower during fast leader failover. We have also
improved resilience of brokers by reducing the memory footprint of
message down-conversions. By using message chunking, both memory
usage and memory reference time have been reduced to avoid
OutOfMemory errors in brokers.

** Kafka clients are now notified of throttling before any throttling is
when quotas are enabled. This enables clients to distinguish between
network errors and large throttle times when quotas are exceeded.

** We have added a configuration option for Kafka consumer to avoid
indefinite blocking in the consumer.

** We have dropped support for Java 7 and removed the previously
deprecated Scala producer and consumer.

** Kafka Connect includes a number of improvements and features.
KIP-298 enables you to control how errors in connectors, transformations
and converters are handled by enabling automatic retries and controlling the
number of errors that are tolerated before the connector is stopped. More
contextual information can be included in the logs to help diagnose problems
and problematic messages consumed by sink connectors can be sent to a
dead letter queue rather than forcing the connector to stop.

** KIP-297 adds a new extension point to move secrets out of connector
configurations and integrate with any external key management system.
The placeholders in connector configurations are only resolved before
sending the configuration to the connector, ensuring that secrets are stored
and managed securely in your preferred key management system and
not exposed over the REST APIs or in log files.

** We have added a thin Scala wrapper API for our Kafka Streams DSL,
which provides better type inference and better type safety during compile
time. Scala users can have less boilerplate in their code, notably regarding
Serdes with new implicit Serdes.

** Message headers are now supported in the Kafka Streams Processor API,
allowing users to add and manipulate headers read from the source topics
and propagate them to the sink topics.

** Windowed aggregations performance in Kafka Streams has been largely
improved (sometimes by an order of magnitude) thanks to the new
single-key-fetch API.

** We have further improved unit testibility of Kafka Streams with the
kafka-streams-testutil artifact.

All of the changes in this release can be found in the release notes:


You can download the source and binary release (Scala 2.11 and Scala 2.12)



Apache Kafka is a distributed streaming platform with four core APIs:

** The Producer API allows an application to publish a stream records to

one or more Kafka topics.

** The Consumer API allows an application to subscribe to one or more

topics and process the stream of records produced to them.

** The Streams API allows an application to act as a stream processor,

consuming an input stream from one or more topics and producing an

output stream to one or more output topics, effectively transforming the

input streams to output streams.

** The Connector API allows building and running reusable producers or

consumers that connect Kafka topics to existing applications or data

systems. For example, a connector to a relational database might

capture every change to a table.

With these APIs, Kafka can be used for two broad classes of application:

** Building real-time streaming data pipelines that reliably get data

between systems or applications.

** Building real-time streaming applications that transform or react

to the streams of data.

Apache Kafka is in use at large and small companies worldwide, including

Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,

Target, The New York Times, Uber, Yelp, and Zalando, among others.

A big thank you for the following 131 contributors to this release!

Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,

Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,

Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,

Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,

Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,

Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,

Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,

Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander,

Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain,

Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,

Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,

Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,

Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan Goyeau,

Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,

Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,

Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,

Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,

manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,

Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick Travers,

nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,

Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,

Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,

Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane

Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas Leplus,

UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,

Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,

Yaswanth Kumar, ying-zheng, Yu

We welcome your help and feedback. For more information on how to

report problems, and to get involved, visit the project website at


Thank you!



View raw message