- Netty / All things NIO
- Author of Netty in Action
- @normanmaurer
- github.com/normanmaurer
Because writing fast, non-blocking network code is non-trivial.
Why Netty ?
http://www.techempower.com/benchmarks/#section=data-r9&hw=i7&test=plaintext
ResponseHTTP/1.1 200 OK
Content-Length: 15
Content-Type: text/plain; charset=UTF-8
Server: Example
Date: Wed, 17 Apr 2013 12:00:00 GMT
Hello, World!| 256 concurrent connections |
| 256 requests pipelined |
24 cores[nmaurer@xxx]~% wrk/wrk -H 'Host: localhost' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Connection: keep-alive' -d 120 -c 256 -t 16 --pipeline 256 http://xxx:8080/plaintext
Running 2m test @ http://xxx:8080/plaintext
16 threads and 256 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 18.77ms 16.68ms 452.00ms 92.67%
Req/Sec 225.82k 41.95k 376.21k 67.34%
429966998 requests in 2.00m, 57.66GB read
Requests/sec: 3583411.40
Transfer/sec: 492.11MB
https://www.flickr.com/photos/owyzzz/4934057731/
| Asychronous from the ground up |
| Using java.nio or native method calls for non-blocking io |
| Futures and callbacks provided for easy composing |
Don’t call us, we’ll call you.
Hollywood principle
Hides all the complexity involved when you use java.nio or java.nio2 |
| Still powers you with a lot of flexibility |
| Unified API for every transport |
| Allows easy testing of your custom code. |
Supports TCP, UDP, UDT, SCTP out of the box |
| Contains codecs for different protocols on top of these |
| HTTP, WebSockets ( + compression) , SPDY, HTTP 2 |
| SSL/TLS, Zlib, Deflate |
| Protobufs, JBoss Marshalling, Java Serialization |
| DNS |
| Memcached, Stomp, Proxy, MQTT |
| …add your prefered protocol here… |
| Having inbound and outbound events handled by the same Thread simplifies concurrency handling a lot! |
Allows to react on each state change by intercept the states via ChannelHandler. |
| Allows flexible handling depending on the needs. |

| Interceptor pattern |
Allows to add building-blocks (ChannelHandler) on-the-fly that transform data or react on events |
Kind of a unix-pipe-like thing…$ echo "Netty is shit...." | sed -e 's/is /is the /' | cat (1)
Netty is the shit....
| 1 | Think of the whole line to be the ChannelPipeline and echo, sed and cat the ChannelHandler s that allow to transform data. |
Inbound and outbound events flow through the ChannelHandler s in the ChannelPipeline and so allow to react one these events. |

Compose complex processing logic via multiple ChannelHandler. |
public class MyChannelInitializer extends ChannelInitializer<Channel> {
@Override
public void initChannel(Channel ch) {
ChannelPipeline p = ch.pipeline();
p.addLast(new SslHandler(...)); (1)
p.addLast(new HttpServerCodec(...)); (2)
p.addLast(new YourRequestHandler()); (3)
}
}| 1 | Encrypt traffic |
| 2 | Support HTTP |
| 3 | Your handler that receive the HTTP requests. |
@Sharable
public class EchoHandler extends ChannelInboundHandlerAdapter {
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) { (1)
ctx.writeAndFlush(msg);
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) { (2)
cause.printStacktrace();
ctx.close();
}
}| 1 | Intercept received message and write it back to the remote peer |
| 2 | React on Throwable and close the connection |
| Different abstract base classes for Decoder and Encoder |
| Handles buffering for you if needed (remember everything is non-blocking!) |
Transform received ByteBuf to Stringpublic class StringDecoder extends MessageToMessageDecoder<ByteBuf> {
@Override
protected void decode(ChannelHandlerContext ctx, ByteBuf msg, List<Object> out) {
out.add(msg.toString(charset));
}
}Transform to be send String to ByteBufpublic class StringEncoder extends MessageToMessageEncoder<String> {
@Override
protected void encode(ChannelHandlerContext ctx, String msg, List<Object> out) {
if (msg.length() == 0) return;
out.add(ByteBufUtil.encodeString(ctx.alloc(), CharBuffer.wrap(msg), charset));
}
}Adding more processing logic is often a matter of adding just-another ChannelHandler to the ChannelPipeline. |

http://memegenerator.net/instance/43005548
public class PortUnificationServerHandler extends ByteToMessageDecoder {
@Override
protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) {
if (in.readableBytes() < 5) return (1)
if (isSsl(in)) { (2)
ctx.pipeline().addLast("ssl", sslCtx.newHandler(ctx.alloc()));
ctx.pipeline().remove(this);
} else if (isGzip(in)) { ...
} else { (3)
ctx.close();
}
}
}| 1 | Will use the first five bytes to detect a protocol. |
| 2 | Check if SSL is used and if so add SslHandler to the ChannelPipeline. |
| 3 | Unknown protocol, just close the connection |
Channel.write(...) ⇒ write through the ChannelPipeline but NOT trigger syscall like write or writev. |
Channel.flush() ⇒ writes all pending data to the socket. |
| Gives more flexibility for when things are written and also allows efficient pipelining. |
public class HttpPipeliningHandler extends SimpleChannelInboundHandler<HttpRequest> {
@Override
public void channelRead(ChannelHandlerContext ctx, HttpRequest req) {
ChannelFuture future = ctx.writeAnd(createResponse(req)); (1)
if (!HttpHeaders.isKeepAlive(req)) {
future.addListener(ChannelFutureListener.CLOSE); (2)
}
}
@Override
public void channelReadComplete(ChannelHandlerContext ctx) {
ctx.flush(); (3)
}
}| 1 | Write to the Channel (No syscall!) but not flush yet |
| 2 | After written close socket |
| 3 | Flush out to the socket once everything was ready from the `Channel |

https://www.flickr.com/photos/kwl/4514986410
public class StateHandler extends ChannelInboundHandlerAdapter {
@Override
public void channelWritabilityChanged(ChannelHandlerContext ctx) { } (1)
}| 1 | Is triggered once Channel.isWritable() changes. |
Don’t block as these will effect all Channel s that are served by the same Thread. |
ChannelPipeline allows to add ChannelHandler that are exected on different Thread to free up IO Thread (EventLoop). |
Channel ch = ...;
ChannelPipeline p = ch.pipeline();
EventExecutor e1 = new DefaultEventExecutor(16);
p.addLast(new MyProtocolCodec()); (1)
p.addLast(e1, new MyDatabaseAccessingHandler()); (2)| 1 | Executed in EventLoop (and so the Thread bound to it) |
| 2 | Executed in one of the EventExecutors of e1 |
Use pooling / ThreadLocal s to prevent GC pressure |
| Prefer direct method invocation over fire event object |
http://25.media.tumblr.com/tumblr_me2eq0PnBx1rtu0cpo1_1280.jpg
| Reduces GC-Pressure a lot! |
| Separate index for reader/writer |
| Direct, Heap and Composite |
| Resizable with max capacity |
| Reference counting / pooling |
uses sun.misc.Unsafe for maximal performance |
ByteBuf buf = ...;
buf.writeInt(1).writeBytes(data).writeBoolean(true)...SlowSearch for ByteBuf :(int index = -1;
for (int i = buf.readerIndex(); index == -1 && i < buf.writerIndex(); i++) {
if (buf.getByte(i) == '\n') {
index = i;
}
}FastSearch for ByteBuf :)int index = buf.forEachByte(new ByteBufProcessor() {
@Override
public boolean process(byte value) {
return value != '\n';
}
});| Pooling pays off for direct and heap buffers! |
https://blog.twitter.com/2013/netty-4-at-twitter-reduced-gc-overhead
| Algorithm is a hybrid of jemalloc and buddy-allocation |
ThreadLocal caches for lock-free allocation |
| Synchronization per area that holds different chunks of memory, when not be able to serve via cache |
Before caches were added[nmaurer@xxx]~% wrk/wrk -H 'Host: localhost' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
-H 'Connection: keep-alive' -d 120 -c 256 -t 16 --pipeline 256 http://xxx:8080/plaintext
...
Requests/sec: 2812559.99
Transfer/sec: 388.93MB
With caches[nmaurer@xxx]~% wrk/wrk -H 'Host: localhost' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
-H 'Connection: keep-alive' -d 120 -c 256 -t 16 --pipeline 256 http://xxx:8080/plaintext
...
Requests/sec: 3022942.17
Transfer/sec: 418.02MB
simple leak reportingLEAK: ByteBuf.release() was not called before it's garbage-collected....advanced leak reportingLEAK: ByteBuf.release() was not called before it's garbage-collected.
Recent access records: 1
#1:
io.netty.buffer.AdvancedLeakAwareByteBuf.toString(AdvancedLeakAwareByteBuf.java:697)
...
Created at:
...
io.netty.handler.codec.xml.XmlFrameDecoderTest.testDecodeWithXml(XmlFrameDecoderTest.java:147)| OpenSSL based SslEngine to reduce memory usage and latency. |
| Native transport for linux using epoll ET for more performance and less CPU usage. |
| Native transport also supports SO_REUSEPORT and TCP_CORK :) |

Using NIO transportBootstrap bootstrap = new Bootstrap().group(new NioEventLoopGroup());
bootstrap.channel(NioSocketChannel.class);Using native transportBootstrap bootstrap = new Bootstrap().group(new EpollEventLoopGroup());
bootstrap.channel(EpollSocketChannel.class);Dynamic Channel re-register (based on metrics) - GSOC |
ForkJoinPool based EventLoop - GSOC |
| Metrics |
https://www.flickr.com/photos/mrpinkeyes/5140672916/
https://www.flickr.com/photos/opensourceway/5751540267

| … and many more … |
| … and many more … |
| Mailinglist - https://groups.google.com/forum/#!forum/netty |
| IRC - #netty irc.freenode.org |
| Website - http://netty.io |
| Source / issue tracker - https://github.com/netty/netty/ |
| Buy my book Netty in Action and make me RICH. |
http://www.manning.com/maurer
$ KA-CHING $
| Netty - http://netty.io |
| Slides generated with Asciidoctor and DZSlides backend |
| Original slide template - Dan Allen & Sarah White |
All pictures licensed with Creative Commons Attribution orCreative Commons Attribution-Share Alike |