What is nftlb?

Overview

nftlb stands for nftables load balancer, the next generation linux firewall that will replace iptables is adapted to behave as a complete load balancer and traffic distributor.

nftlb is a nftables rules manager to create virtual services for load balancing at layer 2, layer 3 and layer 4, minimizing the number of rules and using structures to match efficiently the packets. It’s also provided with an easy JSON API service to have the flexibility to interact with nftlb programmatically and to meet automation. So you can use your preferred health checker to be integrated with nftlb very easily.

The philosophy of nftlb is to maintain the data path into the kernel, in order to achieve the most performance possible, but the control plane and heath checks into user space to have the flexibility to change the behavior easily but also to be compatible with the rest of the linux stack.

At Zevenet, we’ve been using iptables and the netfilter infrastructure for years to create a full featured load balancer, hence we know very well the limitations of such approach that we’re saving with nftlb.

Why is nftlb needed?

The linux kernel already counts of an internal load balancer called IPVS, or also known as LVS (Linux Virtual Server), which is a complete piece of software and very stable that has been used for years. But such load balancer has some limitations: the kernel side is used for tasks that should be performed by the user space, for certain capabilities it duplicates the infrastructure that currently netfilter provides, and it relies on iptables and other pieces of software if is required something more complex (like transparent proxy, multiport or multiprotocol). It provides SNAT and DSR topologies, but not DNAT.

With the iptables approach the main limitations are the number of rules to be created by virtual service and number of backends (a minimum of ~2 rules per backend with several matches included in it) and an increasing linear complexity according to the number of backends added. The sequentially processing of rules also slows down the performance if too much rules are included, and this is even worse due to the classic iptables locking problem. In order to provide IPv6 load balancing it has the inconvenience of the use a different command, ip6tables. In addition, this approach is able to provide DNAT (Destination NAT for transparency) and SNAT (Source NAT) load balancing but not able to work in DSR (Direct Server Return) topologies.

With nftlb based on nftables we save all of these issues:
It can build all these topologies and change from one to another very easily.
It can handle multiport and multiprotocol natively.
It can manage IPv4 and IPv6 traffic seamlessly.
It’s used just one interface to provide all the required capabilities for load balancing.
nftables provides a more expressive language so we can use 2 rules to build a complete load balancer with constant complexity!
The matches are indexed per virtual service, so we don’t need to sequentially process all of them.
It is provided of the RCU subsystem so there is no locking problem when updating rules.
The data path remains in the kernel space but providing the flexibility in the user space for the control plane.
It has been proven that it can perform 10x faster than LVS.

nftlb features

Currently, nftlb provides the following capabilities:
Topologies supported: Destination NAT, Source NAT, Direct Server Return and Stateless DNAT. This enables the use of the load balancer in one-armed and two-armed network architectures.
Support for both IPv4 and IPv6 families.
Multilayer load balancer: DSR in layer 2, IP based load balancing with protocol agnostic at layer 3, and support of load balancing of UDP, TCP and SCTP at layer 4.
Multiport support for ranges and lists of ports.
Multiple virtual services (or farms) support.
Schedulers available: weight, round robin, configurable hash (per IP, port, MAC or combination of them) and symmetric hash.
Support of configurable persistence or client-backend affinity with a timeout (per IP, port, MAC or combination of them).
Support of security policies per service: white and blacklists (from ingress), queuing to user space filter, filtering of bogus TCP frames, maximum number of established connections, limit TCP RST per second, limit new connections per second and more.
Priority support per backend.
Live management of virtual services and backends programmatically through a JSON API.
Web service authentication with a security Key.
Automated testbed included.

Official git repository: https://github.com/zevenet/nftlb

Changelog

nftlb 0.4 (18 Mar 2019)

New features
– farms: add persistence between client and backend during a timeout
– policies: support of security policies per virtual service
– farms: support of queuing packets to user space per service
– farms: support of tcp flow validation per service
– farms: support of max established connections per virtual service per source address
– farms: support of tcp resets per second allowed per virtual service per source address
– farms: support of new connections limit per second per virtual service and optional burst
– farms: add configurable hashing parameters
– src: support of delete all farms at once

Improvements
– nft: refactor farm rules generation code
– server: add long body support
– config: parsing json values hardening
– nft: fix helpers rules according to protocol
– readme: update the new parameter tcp-strict to avoid bogus tcp attacks
– farms: enable mac discovery for stateless dnat
– main: hide the key parameter when the process is running for security reasons
– nft: separate services by interface name for ingress modes
– farms: force the network data reload when changing the virtual ip
– farm: set masquerade if source addr is empty
– nft: add prerouting filter chain for marking and helpers
– buffer: remove debug messages
– farm: set default scheduler parameter for hash algorithm only
– config: use string keys as much as possible
– readme: add stateless nat mode option
– tests: allow launch of one single test without service
– buffer: fix code identation
– backends: only actionable if the backend is available
– backends: declare actionable functions
– buffer: support of scalable buffer
– backends: enable restart of backends after configuration
– nft: apply reset action per farm and backends
– nft: generalize actions for add or deletion postrouting elements
– farms: rename farm source-addr attribute instead of src-addr
– config: print marks in hex format
– tests: support to launch tests through web api
– build: move -lev to LDADD
– build: move preprocessor flags to CPPFLAGS

Bugfixes
– config: return error when an object has not been selected
– backends: avoid go to config_error after setting dnat ip addresses
– nft: fix stateless dnat rules when the input and output interfaces are different
– nft: fix service name for stateless nat
– backend: fix backend validation during automated mac address request
– network: protect double free in handle
– server: fix double free segfault
– backends: fix backend validation when applying dsr mode
– farms: strim virtual interfaces for ingress chains
– nft: fix add element filter rules in reload
– nft: fix stateless dnat rules actions
– nft: avoid the use of filter chain and backend marks for ingress
– nft: avoid empty rules in filter chain when there is no backends
– backends: apply reload if changing the state of a backend
– nft: fix skb mark insertion from ct mark in filter chain
– nft: fix flush and delete chain filter
– nft: fix delete filter service and chain
– nft: fix delete elements from filter chain
– nft: avoid rules generation if there is no backend available
– backends: fix backend availability for ingress modes
– logs: fix set log level at startup
– objects: avoid buffer copy overlap
– buffer: fix typo in error message
– backends: fix backend going down
– Remove config.h file from .gitignore
– src: fix string copy sizes
– src: add a cleanup parsing structures to avoid null objects references
– config: fix farm mark json dump
– server: fix parse input body that produces buffer parsing error

nftlb 0.3 (15 Nov 2018)

This release is integrated in kubernetes as kube-nftlb https://github.com/zevenet/kube-nftlb

New features
– network: generalize netlink request to ask for routing data
– farms: new mode stateless dnat
– farms: add l7 helpers support
– farms: add input logging support
– farms: support of farm renaming with the ‘newname’ attribute
– farms: add mark flow support per virtual service
– nft: add flow mark per backend and farm using masks
– src: add custom source ip address configuration instead of masquerading

Improvements
– events: generalize event loop
– farms: include new attributes for interface and mac address management
– network: add support to interoperate with some network discovery functions
– src: refactorization and api simplification
– events: generalize netlink event for dsr
– farms: make dsr counter global
– backends: include a new backend state config_error
– src: silent fallthrough warning
– backends: ensure the backends list is empty when configuring the
output interface
– farms: validate and rulerize per farm
– config: avoid to print auto-generated information of a farm
– farms: validate and check the farm status before rulerize
– server: expand the server buffer data
– readme: add new examples
– tests: improve diff output format
– nft: improve modularization of nft rules generation
– server: set SO_REUSEADDR socket flag
– main: initial signal handler skeleton
– server: add struct nftlb_client
– server: add struct nftlb_http_state
– server: add nftlb_http_send_response()
– server: add body response field to struct nftlb_http_state
– src: do no use EXIT_{SUCCESS,FAILURE}
– server: statify objects that are only used from server.c
– server: remove unnecessary definitions

Bugfixes
– config: dump configuration with indented JSON
– nft: fix dsr rules to set the mac address instead of matching
– backend: fix update backend status when switching from down to up
– nft: avoid add rules if no backends are available
– objects: set right initial state for farms and backends
– farms: fix start-stop actions
– backends: input validation for net_get_neigh_ether()
– nft: fix stateless nat backend to client rule
– nft: fix udp ipv6 services name
– server: fix some web server memory leaks
– tests: fix some tests cases

nftlb 0.2 (14 May 2018)

– 3 topologies supported: Destination NAT, Source NAT and Direct
Server Return. This enables the load balancer to be setup in
one-armed and two-armed network architectures.
– support for both IPv4 and IPv6 families.
– multilayer capabilities: MAC based LB in layer 2, IP based LB
with protocol agnostic at layer 3, and support of UDP, TCP and
SCTP LB at layer 4.
– multiport support for ranges and lists of ports.
– support of multiple virtual services setup.
– schedulers available: weight, round robin, hash and symmetric
hash.
– priority support per backend.
– JSON API service for monitoring, automation and managemnet.
– web service authentication with a security key.
– automated testbed.

nftlb 0.1 (27 Feb 2018)

– Initial version

Share on:

Documentation under the terms of the GNU Free Documentation License.

Was this article helpful?

Related Articles