- HTTP / Pipelining
- Writing gracefully
- Buffers best-practises
- EventLoop
public class HttpHandler extends SimpleChannelInboundHandler<HttpRequest> {
@Override
public void channelRead(ChannelHandlerContext ctx, HttpRequest req) {
ChannelFuture future = ctx.writeAndFlush(createResponse(req)); (1)
if (!isKeepAlive(req)) {
future.addListener(ChannelFutureListener.CLOSE); (2)
}
}
}| 1 | Write to the Channel and flush out to the Socket. |
| 2 | After written close Socket |
public class HttpPipeliningHandler extends SimpleChannelInboundHandler<HttpRequest> {
@Override
public void channelRead(ChannelHandlerContext ctx, HttpRequest req) {
ChannelFuture future = ctx.write(createResponse(req)); (1)
if (!isKeepAlive(req)) {
future.addListener(ChannelFutureListener.CLOSE); (2)
}
}
@Override
public void channelReadComplete(ChannelHandlerContext ctx) {
ctx.flush(); (3)
}
}| 1 | Write to the Channel (No syscall!) but don’t flush yet |
| 2 | Close socket when done writing |
| 3 | Flush out to the socket. |
Validate headers for US-ASCIIChannelPipeline pipeline = channel.pipeline();
pipeline.addLast(new HttpRequestDecoder(4096, 8192, 8192));
HttpResponse response = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpStatus.OK);Not validate headers for US-ASCIIChannelPipeline pipeline = channel.pipeline();
pipeline.addLast(new HttpRequestDecoder(4096, 8192, 8192, false));
HttpResponse response = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpStatus.OK, false);| Validation takes time and most of the times it is not needed directly in the decoder/encoder. |
private static final CharSequence X_HEADER_NAME = HttpHeaders.newEntity("X-Header"); (1)
private static final CharSequence X_VALUE = HttpHeaders.newEntity("Value");
pipeline.addLast(new SimpleChannelInboundHandler<FullHttpRequest>() {
@Override
public void channelRead(ChannelHandlerContext ctx, FullHttpRequest req) {
FullHttpResponse response = new FullHttpResponse(HTTP_1_1, OK);
response.headers().set(X_HEADER_NAME, X_VALUE); (2)
...
ctx.writeAndFlush(response);
}
});| 1 | Create CharSequence for often used header names and values. |
| 2 | Add to HttpHeader of FullHttpResponse |
Created CharSequence is faster to encode and faster to find in HttpHeaders. |
write(msg) ⇒ pass through pipeline
flush() ⇒ gathering write of previous written msgs
writeAndFlush() ⇒ short-cut for write(msg) and flush()
| Limit flushes as much as possible as syscalls are quite expensive. |
But also limit write(...) as much as possible as it need to traverse the whole pipeline. |
public class BlindlyWriteHandler extends ChannelInboundHandlerAdapter {
@Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
while(needsToWrite) { (1)
ctx.writeAndFlush(createMessage());
}
}
}| 1 | Writes till needsToWrite returns false. |
| Risk of OutOfMemoryError if writing too fast and having slow receiver! |
public class GracefulWriteHandler extends ChannelInboundHandlerAdapter {
@Override
public void channelActive(ChannelHandlerContext ctx) {
writeIfPossible(ctx.channel());
}
@Override
public void channelWritabilityChanged(ChannelHandlerContext ctx) {
writeIfPossible(ctx.channel());
}
private void writeIfPossible(Channel channel) {
while(needsToWrite && channel.isWritable()) { (1)
channel.writeAndFlush(createMessage());
}
}
}| 1 | Make proper use of Channel.isWritable() to prevent OutOfMemoryError |
| Set sane WRITE_BUFFER_HIGH_WATER_MARK and WRITE_BUFFER_LOW_WATER_MARK |
ServerServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.childOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, 32 * 1024);
bootstrap.childOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, 8 * 1024);ClientBootstrap bootstrap = new Bootstrap();
bootstrap.option(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, 32 * 1024);
bootstrap.option(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, 8 * 1024);| Use unpooled buffers with caution! |
| Use pooled buffers! |
Bootstrap bootstrap = new Bootstrap();
bootstrap.option(ChannelOption.ALLOCATOR, PooledByteBufAllocator.DEFAULT);
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.childOption(ChannelOption.ALLOCATOR, PooledByteBufAllocator.DEFAULT);| Pooling pays off for direct and heap buffers! |
https://blog.twitter.com/2013/netty-4-at-twitter-reduced-gc-overhead
| OpenJDK and Oracle JDK copy otherwise to direct buffer by itself! |
Only use heap buffers if need to operate on byte[]` in ChannelOutboundHandler! By default direct ByteBuf` will be returned by ByteBufAllocator.buffer(...).
Take this as rule of thumb
SlowSearch :(ByteBuf buf = ...;
int index = -1;
for (int i = buf.readerIndex(); index == -1 && i < buf.writerIndex(); i++) {
if (buf.getByte(i) == '\n') {
index = i;
}
}FastSearch :)ByteBuf buf = ...;
int index = buf.forEachByte(new ByteBufProcessor() {
@Override
public boolean process(byte value) {
return value != '\n';
}
});ByteBuf payload ⇒ extend DefaultByteBufHolder
http://www.flickr.com/photos/za3tooor/65911648/
| Use zero-memory-copy for efficient transfer of raw file content |
Channel channel = ...;
FileChannel fc = ...;
channel.writeAndFlush(new DefaultFileRegion(fc, 0, fileLength));This only works if you don’t need to modify the data on the fly. If so use ChunkedWriteHandler and NioChunkedFile. |
Thread.sleep() |
CountDownLatch.await() or any other blocking operation from
java.util.concurrent |
| Long-lived computationally intensive operations |
| Blocking operations that might take a while (e.g. DB query) |
http://www.flickr.com/photos/za3tooor/65911648/
Bootstrap bootstrap = new Bootstrap().group(new NioEventLoopGroup());
Bootstrap bootstrap2 = new Bootstrap().group(new NioEventLoopGroup());Share EventLoopGroup between different BootstrapsEventLoopGroup group = new NioEventLoopGroup();
Bootstrap bootstrap = new Bootstrap().group(group);
Bootstrap bootstrap2 = new Bootstrap().group(group);Sharing the same EventLoopGroup allows to keep the resource usage (like Thread-usage) to a minimum. |
public class ProxyHandler extends ChannelInboundHandlerAdapter {
@Override
public void channelActive(ChannelHandlerContext ctx) { (1)
final Channel inboundChannel = ctx.channel();
Bootstrap b = new Bootstrap();
b.group(new NioEventLooopGroup()); (2)
...
ChannelFuture f = b.connect(remoteHost, remotePort);
...
}
}| 1 | Called once a new connection was accepted |
| 2 | Use a new EventLoopGroup instance to handle the connection to the remote peer |
| Don’t do this! This will tie up more resources than needed and introduce extra context-switching overhead. |
public class ProxyHandler extends ChannelInboundHandlerAdapter {
@Override
public void channelActive(ChannelHandlerContext ctx) { (1)
final Channel inboundChannel = ctx.channel();
Bootstrap b = new Bootstrap();
b.group(inboundChannel.eventLoop()); (2)
...
ChannelFuture f = b.connect(remoteHost, remotePort);
...
}
}| 1 | Called once a new connection was accepted |
| 2 | Share the same EventLoop between both Channels. This means all IO for both connected Channels are handled by the same Thread. |
| Always share EventLoop in those Applications |
public class YourHandler extends ChannelInboundHandlerAdapter {
@Override
public void channelActive(ChannelHandlerContext ctx) {
// BAD (most of the times)
ctx.channel().writeAndFlush(msg); (1)
// GOOD
ctx.writeAndFlush(msg); (2)
}
}| 1 | Channel.* methods ⇒ the operation will start at the tail of the ChannelPipeline |
| 2 | ChannelHandlerContext.* methods => the operation will start from this `ChannelHandler to flow through the ChannelPipeline. |
| Use the shortest path as possible to get the maximal performance. |
@ChannelHandler.Shareable (1)
public class StatelessHandler extends ChannelInboundHandlerAdapter {
@Override
public void channelActive(ChannelHandlerContext ctx) {
logger.debug("Now client from " + ctx.channel().remoteAddress().toString());
}
}
public class MyInitializer extends ChannelInitializer<Channel> {
private static final ChannelHandler INSTANCE = new StatelessHandler();
@Override
public void initChannel(Channel ch) {
ch.pipeline().addLast(INSTANCE);
}
}| 1 | Annotate ChannelHandler that are stateless with @ChannelHandler.Shareable and use the same instance accross Channels to reduce GC. |
public class OneTimeHandler extends ChannelInboundHandlerAdapter {
@Override
public void channelActive(ChannelHandlerContext ctx) {
doOneTimeAction();
ctx.channel().pipeline().remove(this); (1)
}
}| 1 | Remove ChannelHandler once not needed anymore. |
This keeps the ChannelPipeline as short as possible and so eliminate overhead of traversing as much as possible. |
ChannelPipelineYour custom eventspublic enum CustomEvents {
MyCustomEvent
}
public class CustomEventHandler extends ChannelInboundHandlerAdapter {
@Override
public void userEventTriggered(ChannelHandlerContext ctx, Object evt) {
if (evt == MyCustomEvent) { // do something}
}
}
ChannelPipeline pipeline = channel.pipeline();
pipeline.fireUserEventTriggered(MyCustomEvent);| Good fit for handshake notifications and more |
public class EncodeActsOnByteArray extends MessageToByteEncoder<YourMessage> {
public EncodeActsOnByteArray() { super(false); } (1)
@Override
public encode(ChannelHandlerContext ctx, YourMessage msg, ByteBuf out) {
byte[] array = out.array(); (2)
int offset = out.arrayOffset() + out.writerIndex();
out.writeIndex(out.writerIndex() + encode(msg, array, offset)); (3)
}
private int encode(YourMessage msg, byte[] array, int offset, int len) { ... }
}| 1 | Ensure heap buffers are used when pass into encode(...) method. This way you can access the backing array directly |
| 2 | Access the backing array and also calculate offset |
| 3 | Update writerIndex to reflect written bytes |
| This saves extra byte copies. |
By default Netty will keep on reading data from the Channel once something is ready.
Need more fine grained control ?channel.config().setAutoRead(false); (1)
channel.read(); (2)
channel.config().setAutoRead(true); (3)| 1 | Disable auto read == no more data will be read automatically from this Channel. |
| 2 | Tell the Channel to do one read operation once new data is ready |
| 3 | Enable again auto read == Netty will automatically read again |
| This can also be quite useful when writing proxy like applications! |
| OpenSSL based SslEngine to reduce memory usage and latency. |
| Native transport for Linux using Epoll ET for more performance and less CPU usage. |
| Native transport also supports SO_REUSEPORT and TCP_CORK :) |

Using NIO transportBootstrap bootstrap = new Bootstrap().group(new NioEventLoopGroup());
bootstrap.channel(NioSocketChannel.class);Using native transportBootstrap bootstrap = new Bootstrap().group(new EpollEventLoopGroup());
bootstrap.channel(EpollSocketChannel.class);| Buy my book Netty in Action and make me RICH. |
http://www.manning.com/maurer
$ KA-CHING $
| Netty - http://netty.io |
| Slides generated with Asciidoctor and DZSlides backend |
| Original slide template - Dan Allen & Sarah White |
All pictures licensed with Creative Commons Attribution orCreative Commons Attribution-Share Alike |