2021-12-07 Updated version
Please additional see Follow up – Docker and fail2ban – How I solved it (for me)
Docker is great when running your own services in an isolated ephemeral setup.
I’ve been using this pattern for quite a while now (approx since Linux VServer had been introduced, afterwards with plain LXC and then with Docker).
But SMTP, IMAP and other services are very attractive to not so nice people. And one might want to add some security to the services, eg. using layer 7 level information to block network traffic on layer 3/4.
So as soon as there are like bruteforce login attacks, drop the eg. TCP/IP paket for the IP of the attacker.
Normally you’d use fail2ban out of the box. It provides pretty good detections for the most well known service implementations and integrates the counter measures (eg. iptable based actions) very good into your OS.
With docker this is a little bit different. Docker also uses the iptables network utils for passing the incomming traffic on public IP towards an private IP by DNATting (Destination Network Address Translation) into a docker container private network.
Let’s have a look at the iptables chain flows:
Let’s check the acual config:
$ iptables -t nat -L Chain PREROUTING (policy ACCEPT) target prot opt source destination DOCKER all -- anywhere anywhere ADDRTYPE match dst-type LOCAL Chain INPUT (policy ACCEPT) target prot opt source destination Chain POSTROUTING (policy ACCEPT) target prot opt source destination ... Chain DOCKER (2 references) target prot opt source destination ... DNAT tcp -- anywhere myhost.tdl tcp dpt:https to:10.10.10.10:443
So at a very early stage the traffic is passed on via DNAT. Solutions you can find for Docker and fail2ban mostly deal with the FORWARD chain.
This did not really work oput for me. Therefore I did setup my own fail2ban action (eg. /etc/fail2ban/action.d/iptables-mangle-allports.conf):
[INCLUDES]
before = iptables-common.conf
[Definition]
# Option: actionstart
# Notes.: command executed once at the start of Fail2Ban.
# Values: CMD
#
actionstart = -t filter -N f2b-<name>
-t filter -A f2b-<name> -j
-t filter -I INPUT -p -j f2b-<name>
-t filter -I FORWARD -p -j f2b-<name>
-t filter -I OUTPUT -p -j f2b-<name>
-t filter -I DOCKER -p -j f2b-<name>
# Option: actionstop
# Notes.: command executed once at the end of Fail2Ban
# Values: CMD
#
actionstop = -t filter -D INPUT -p -j f2b-
-t filter -D FORWARD -p -j f2b-<name>
-t filter -D OUTPUT -p -j f2b-<name>
-t filter -D DOCKER -p -j f2b-<name>
<actionflush>
-t filter -X f2b-<name>
# Option: actioncheck
# Notes.: command executed once before each actionban command
# Values: CMD
#
actioncheck = -t filter -n -L | grep -q 'f2b-[ \t]'
# Option: actionban
# Notes.: command executed when banning an IP. Take care that the command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionban = -t filter -I f2b-<name> 1 -s -j REJECT --reject-with icmp-host-unreachable
# Option: actionunban
# Notes.: command executed when unbanning an IP. Take care that the
command is executed with Fail2Ban user rights.
# Tags: See jail.conf(5) man page
# Values: CMD
#
actionunban = -t filter -D f2b-<name> -s -j REJECT --reject-with icmp-host-unreachable
[Init]
I am now able to DROP the packages at a very eraly stage. Which helped me a lot.
Some jail example for my docker based mail setup, /etc/fail2ban/jail.d/mymail.conf:
[emailserver] enabled = true filter = mymail logpath = /var/log/syslog maxretry = 2 findtime = 72000 bantime = 7200 chain = PREROUTING banaction = iptables-mangle-allports[name="emailserver", chain="PREROUTING"]
This uses the mymail filter, defined in /etc/fail2ban/filter.d/mymail.conf:
3 ban in 1 hour > Ban for 1 hour [mymail] enabled = true filter = mymail logpath = /var/log/syslog maxretry = 2 findtime = 86400 bantime = 86400 banaction = iptables-mangle-allports[name="mymail"]
In my docker-compose.yaml, I’ve added a logging towards journald for the auth service used for the mail server:
version: '3'
services:
auth:
image: {{auth-service-image you use}}
logging:
driver: "journald"
options:
env: "mail_auth=true"
tag: "{{.Name}}/{{.ID}}"
For people like me coming from google, there are some problems/nits with his iptables-mangle-allports.conf:
1. After -s in the actionban/actionunban, you need to add to tell it what ip to actually ban.
2. You have to prepend to all the commands, or else sh just tries to interpret the arguments as commands.
3. You can remove „-t filter“ from all the commands, as that is the default table.
4. You can remove the blank „f2b-“ chain, just update the grep to „grep -q ‚f2b-‚“
5. My version of iptables didn’t have REJECT or –reject-with, so I removed –reject-with and changed REJECT to DROP.
6. You probably don’t want this filter applied to the OUTPUT chain, as if I understand correctly it shouldn’t match anything. I removed it.
Hallo,
ich bin durch Zufall auf deinen Post gestoßen, da ich vermutlich das gleiche Problem habe. Ich habe deine Anpassungen versucht zu übernehmen, aber es funktioniert leider trotzdem nicht. Würde mich freuen, wenn du mir helfen kannst.
Kurz mein Setting:
Ich habe einen vServer bei netcup mit Ubuntu 20.04 und root Zugang. Auf diesem ist Docker installiert. Auf Docker läuft eine Nextcloud Instanz und Nginx Proxy Manager, der als Proxy fungiert. Das Problem ist nun, dass ich Fail2ban analog dieser Anleitung https://www.c-rieger.de/nextcloud-installationsanleitung/ installiert und dann deine Anpassungen vorgenommen habe. Beide Varianten funktionieren leider nicht. Gebe ich mehrmals falsche Anmeldedaten ein, bannt f2b eine private IP, nämlich die von Docker und ich kann mich weiterhin versuchen auf Nextcloud anzumelden.
Meine ipTables sehen so aus:
$ iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
DOCKER all — anywhere anywhere ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
DOCKER all — anywhere !localhost/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
MASQUERADE all — 172.21.0.0/16 anywhere
MASQUERADE all — 172.19.0.0/16 anywhere
MASQUERADE all — 172.17.0.0/16 anywhere
MASQUERADE all — 172.18.0.0/16 anywhere
MASQUERADE all — 192.168.32.0/20 anywhere
MASQUERADE tcp — 192.168.32.2 192.168.32.2 tcp dpt:http
MASQUERADE tcp — 172.17.0.2 172.17.0.2 tcp dpt:9000
MASQUERADE tcp — 172.18.0.3 172.18.0.3 tcp dpt:81
MASQUERADE tcp — 172.18.0.3 172.18.0.3 tcp dpt:https
MASQUERADE tcp — 172.18.0.3 172.18.0.3 tcp dpt:http
MASQUERADE tcp — 172.21.0.4 172.21.0.4 tcp dpt:http
MASQUERADE tcp — 172.19.0.4 172.19.0.4 tcp dpt:http
Chain DOCKER (2 references)
target prot opt source destination
RETURN all — anywhere anywhere
RETURN all — anywhere anywhere
RETURN all — anywhere anywhere
RETURN all — anywhere anywhere
RETURN all — anywhere anywhere
DNAT tcp — anywhere anywhere tcp dpt:8833 to:192.168.32.2:80
DNAT tcp — anywhere anywhere tcp dpt:9199 to:172.17.0.2:9000
DNAT tcp — anywhere anywhere tcp dpt:9181 to:172.18.0.3:81
DNAT tcp — anywhere anywhere tcp dpt:https to:172.18.0.3:443
DNAT tcp — anywhere anywhere tcp dpt:http to:172.18.0.3:80
DNAT tcp — anywhere anywhere tcp dpt:9081 to:172.21.0.4:80
DNAT tcp — anywhere anywhere tcp dpt:9080 to:172.19.0.4:80
Ich kann damit nun leider gar nichts anfangen. f2b meldet als gebannt 172.19.0.1. Und das macht nun gar keinen Sinn.
Wäre nett, wenn du mir helfen kannst.
Viele Grüße
Christian
Hi Tobias, for my Docker network I don’t have any prerouting or postrouting chain, I have my fail2ban running inside docker container and I see ip’s banned in it , but in the actually iptables outside docker, I don’t see any IP’s blocked, and it allows the IP still login, This is my IP tables list:
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
5019 3594K DOCKER-USER all — * * 0.0.0.0/0 0.0.0.0/0
5019 3594K DOCKER-ISOLATION-STAGE-1 all — * * 0.0.0.0/0 0.0.0.0/0
767 468K ACCEPT all — * docker0 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
8 416 DOCKER all — * docker0 0.0.0.0/0 0.0.0.0/0
916 952K ACCEPT all — docker0 !docker0 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all — docker0 docker0 0.0.0.0/0 0.0.0.0/0
5795 2549K ACCEPT all — * br-8f06c3dc391f 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 DOCKER all — * br-8f06c3dc391f 0.0.0.0/0 0.0.0.0/0
7675 747K ACCEPT all — br-8f06c3dc391f !br-8f06c3dc391f 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all — br-8f06c3dc391f br-8f06c3dc391f 0.0.0.0/0 0.0.0.0/0
43965 23M ACCEPT all — * br-288ddad3c4ae 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
1083 59844 DOCKER all — * br-288ddad3c4ae 0.0.0.0/0 0.0.0.0/0
22862 17M ACCEPT all — br-288ddad3c4ae !br-288ddad3c4ae 0.0.0.0/0 0.0.0.0/0
22 1320 ACCEPT all — br-288ddad3c4ae br-288ddad3c4ae 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain DOCKER (3 references)
pkts bytes target prot opt in out source destination
8 416 ACCEPT tcp — !docker0 docker0 0.0.0.0/0 172.17.0.2 tcp dpt:9000
0 0 ACCEPT tcp — !docker0 docker0 0.0.0.0/0 172.17.0.2 tcp dpt:8000
11 540 ACCEPT tcp — !br-288ddad3c4ae br-288ddad3c4ae 0.0.0.0/0 172.18.0.2 tcp dpt:443
4 208 ACCEPT tcp — !br-288ddad3c4ae br-288ddad3c4ae 0.0.0.0/0 172.18.0.2 tcp dpt:81
0 0 ACCEPT tcp — !br-288ddad3c4ae br-288ddad3c4ae 0.0.0.0/0 172.18.0.2 tcp dpt:80
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
pkts bytes target prot opt in out source destination
916 952K DOCKER-ISOLATION-STAGE-2 all — docker0 !docker0 0.0.0.0/0 0.0.0.0/0
184 17672 DOCKER-ISOLATION-STAGE-2 all — br-8f06c3dc391f !br-8f06c3dc391f 0.0.0.0/0 0.0.0.0/0
720 189K DOCKER-ISOLATION-STAGE-2 all — br-288ddad3c4ae !br-288ddad3c4ae 0.0.0.0/0 0.0.0.0/0
5019 3594K RETURN all — * * 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-USER (1 references)
pkts bytes target prot opt in out source destination
102K 73M RETURN all — * * 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-ISOLATION-STAGE-2 (3 references)
pkts bytes target prot opt in out source destination
0 0 DROP all — * docker0 0.0.0.0/0 0.0.0.0/0
0 0 DROP all — * br-8f06c3dc391f 0.0.0.0/0 0.0.0.0/0
0 0 DROP all — * br-288ddad3c4ae 0.0.0.0/0 0.0.0.0/0
1820 1159K RETURN all — * * 0.0.0.0/0 0.0.0.0/0
# Warning: iptables-legacy tables present, use iptables-legacy to see them
Trying to resolve this from long back but couldn’t ban the IP’s totally can you please help me?
Hi,
I haven’t done any attempt to have fail2ban run as a container itself. It might be related to the cgroup isolation and the fail2ban container might need to run as a priviledged container – but that’s just guessing.
Unfortunately I don’t have any insights about your setup.
I myself changed to use ipset+iptables to have less iptable rules and improve the performance.
I do hook into the INPUT and FORWARD chain – no prerouting/postrouting.
There are – at leat in debian – some examples for some ipset action in fail2ban.