Discussion:
stunnel transparent mode
Julian D. Seifert
2011-08-18 00:59:30 UTC
Permalink
Hi List,

I have a little issue trying to get stunnel running for encapsulating an
existing service with SSL. We are running a really old legacy ircd
service and would like to provide SSL for the users without having to
actually hack that into the really old code base. (Yes we could just use
another more modern software but let's just say there are "political"
reasons as to why this is not an option) For an ircd we need
transparency as in "the ircd should think the connection is not coming
from localhost")
ANYWAY... so I am working with stunnel 4.39 right now.

First a question if I understand it correctly how stunnel will work.
I have the ircd running on IP1:6667. I created a service section like this.

[ircd]
accept = 7000
connect = 6667
transparent = source

(And applied the iptables/routing related stuff mentioned in the
documentation)
Should this already work? Will stunnel "manipulate" packets so that the
IRCD "sees" connections from the Peers IP's? And will the Answers the
ircd sends to the peers get "intercepted" by stunnel so it can
"remanipulate" the answers into the SSL connection?
If I try it like I get "local_bind (original port): Cannot assign
requested address (99) stunnel" and clients get connection abort.

Or is this what "transparent = both" is for? With 4.39 I had an issue
trying to use "transparent = both". Stunnel always displayed an error
message saying "Each service must define two endpoints" (more or less).
I got it to compile by commenting out the one line in options.c causing
the error...
#### snip ####
} else { /* standalone mode checks */
if((unsigned int)section->option.accept
+ (unsigned int)section->option.program
+ (unsigned int)section->option.remote
+ (unsigned int)section->option.sni
#ifndef USE_WIN32
+ (unsigned int)section->option.transparent_dst
#endif /* USE_WIN32 */
!=2) {
section_error(last_line, section->servname,
"Each service must define two endpoints");
return 0;
}
### snap ###
Commented out the line within the ifndef.

Anyway... I hope somebody can clarify for me how stunnel works and what
transparent mode is the one I need. (I want to transparently provide the
ircd service with connections so it seems they are directly from the
peers and not from stunnel(localhost))

regards,

Julian Seifert

-- I hate mondays...
Michal Trojnara
2011-08-18 08:19:33 UTC
Permalink
Post by Julian D. Seifert
If I try it like I get "local_bind (original port): Cannot assign
requested address (99) stunnel" and clients get connection abort.
I guess either you're not running one of the supported kernels, or
you're not running stunnel as root.

Please send your configuration file.
Post by Julian D. Seifert
Or is this what "transparent = both" is for?
Could you Read The Fine Manual, please?
http://stunnel.org/static/stunnel.html
http://catb.org/~esr/faqs/smart-questions.html

Mike
Julian D. Seifert
2011-08-18 10:58:44 UTC
Permalink
Post by Michal Trojnara
Post by Julian D. Seifert
If I try it like I get "local_bind (original port): Cannot assign
requested address (99) stunnel" and clients get connection abort.
I guess either you're not running one of the supported kernels, or
you're not running stunnel as root.
I checked (with ps) that stunnel is running as root(I commented out the
setuid setgid options in the config file)
I had two setups, one with standard ubuntu lucid lts kernel and one with
2.6.32. (I can also provide the kernelconfigurations or the settings of
the options that are necessary)
Post by Michal Trojnara
Please send your configuration file.
stunnel.conf:
cert = /etc/stunnel/stunnel.pem
key = /etc/stunnel/key.pem
sslVersion = all
pid = /stunnel4.pid
socket = r:SO_LINGER=1:60
debug = 7
[ircd]
accept = 7000
connect = 91.143.93.209:6667
transparent = source
TIMEOUTclose = 45


regards,

Julian 'alamar' Seifert
Julian D. Seifert
2011-08-18 11:05:00 UTC
Permalink
Post by Julian D. Seifert
Post by Michal Trojnara
Post by Julian D. Seifert
If I try it like I get "local_bind (original port): Cannot assign
requested address (99) stunnel" and clients get connection abort.
I guess either you're not running one of the supported kernels, or
you're not running stunnel as root.
I checked (with ps) that stunnel is running as root(I commented out the
setuid setgid options in the config file)
I had two setups, one with standard ubuntu lucid lts kernel and one with
2.6.32. (I can also provide the kernelconfigurations or the settings of
the options that are necessary)
Linux ubuntu 2.6.32-21-generic #32-Ubuntu SMP Fri Apr 16 08:10:02 UTC
2010 i686 GNU/Linux
lsmod | grep -ie tprox
xt_TPROXY 1165 0
nf_defrag_ipv4 1073 2 xt_TPROXY,xt_socket
nf_tproxy_core 1608 2 xt_TPROXY,xt_socket,[permanent]
x_tables 14299 4 xt_TPROXY,xt_MARK,xt_socket,ip_tables


ps:
root 2024 0.0 0.0 3876 492 pts/0 S 04:01 0:00
./stunnel stunnel.conf
root 2025 0.0 0.0 3876 492 pts/0 S 04:01 0:00
./stunnel stunnel.conf


stunnel version:
No limit detected for the number of clients
signal_pipe: FD=3 allocated (non-blocking mode)
signal_pipe: FD=4 allocated (non-blocking mode)
stunnel 4.39 on i686-pc-linux-gnu platform
Compiled/running with OpenSSL 0.9.8k 25 Mar 2009
Threading:PTHREAD SSL:ENGINE Auth:LIBWRAP Sockets:POLL,IPv6
stunnel 4.39 on i686-pc-linux-gnu platform
Compiled/running with OpenSSL 0.9.8k 25 Mar 2009
Threading:PTHREAD SSL:ENGINE Auth:LIBWRAP Sockets:POLL,IPv6

Global option defaults
debug = daemon.notice
pid = /usr/local/var/run/stunnel/stunnel.pid
RNDbytes = 64
RNDfile = /dev/urandom
RNDoverwrite = yes

Service-level option defaults
cert = /usr/local/etc/stunnel/stunnel.pem
ciphers = RC4-MD5:HIGH:!aNULL:!SSLv2
curve = sect163r2
session = 300 seconds
sslVersion = TLSv1 for client, all for server
stack = 65536 bytes
TIMEOUTbusy = 300 seconds
TIMEOUTclose = 60 seconds
TIMEOUTconnect = 10 seconds
TIMEOUTidle = 43200 seconds
verify = none
Michal Trojnara
2011-08-18 11:13:56 UTC
Permalink
Post by Julian D. Seifert
lsmod | grep -ie tprox
xt_TPROXY 1165 0
nf_defrag_ipv4 1073 2 xt_TPROXY,xt_socket
nf_tproxy_core 1608 2 xt_TPROXY,xt_socket,[permanent]
x_tables 14299 4 xt_TPROXY,xt_MARK,xt_socket,ip_tables
Interesting. What is the output of:
iptables -t mangle -v -n -L
?

Please also include a longer sample of the debug log.

Mike
Julian D. Seifert
2011-08-18 11:26:06 UTC
Permalink
Post by Michal Trojnara
Post by Julian D. Seifert
lsmod | grep -ie tprox
xt_TPROXY 1165 0
nf_defrag_ipv4 1073 2 xt_TPROXY,xt_socket
nf_tproxy_core 1608 2 xt_TPROXY,xt_socket,[permanent]
x_tables 14299 4 xt_TPROXY,xt_MARK,xt_socket,ip_tables
iptables -t mangle -v -n -L
?
iptables -t mangle -L -v -n
Chain PREROUTING (policy ACCEPT 212 packets, 53634 bytes)
pkts bytes target prot opt in out source
destination
1068 1567K DIVERT tcp -- * * 0.0.0.0/0
0.0.0.0/0 socket

Chain INPUT (policy ACCEPT 1127 packets, 1574K bytes)
pkts bytes target prot opt in out source
destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination

Chain OUTPUT (policy ACCEPT 359 packets, 26078 bytes)
pkts bytes target prot opt in out source
destination

Chain POSTROUTING (policy ACCEPT 359 packets, 26078 bytes)
pkts bytes target prot opt in out source
destination

Chain DIVERT (1 references)
pkts bytes target prot opt in out source
destination
1068 1567K MARK all -- * * 0.0.0.0/0
0.0.0.0/0 MARK xset 0x1/0xffffffff
1068 1567K ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0
Post by Michal Trojnara
Please also include a longer sample of the debug log.
It's from the lab environment (As all infos are) the only difference is
private ip space
Client is 192.168.0.102, Server(stunnel)192.168.0.103

Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078105408]: local socket:
FD=0 allocated (n
on-blocking mode)
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078105408]: Service ircd
accepted FD=0 from
192.168.0.102:63723
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: Service ircd started
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: Option
TCP_NODELAY set on local
socket
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: Waiting for a
libwrap process
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: Acquired libwrap
process #0
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: Releasing libwrap
process #0
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: Released libwrap
process #0
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: Service ircd
permitted by libwr
ap from 192.168.0.102:63723
Aug 18 04:23:09 ubuntu stunnel: LOG5[2029:3078101872]: Service ircd
accepted connectio
n from 192.168.0.102:63723
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: SSL state
(accept): before/acce
pt initialization
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: SSL state
(accept): SSLv3 read
client hello A
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: SSL state
(accept): SSLv3 write
server hello A
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: SSL state
(accept): SSLv3 write certificate A
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: SSL state
(accept): SSLv3 write server done A
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: SSL state
(accept): SSLv3 flush data
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: SSL state
(accept): SSLv3 read client key exchange A
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: SSL state
(accept): SSLv3 read finished A
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: SSL state
(accept): SSLv3 write change cipher spec A
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: SSL state
(accept): SSLv3 write finished A
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: SSL state
(accept): SSLv3 flush data
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: 4 items in the
session cache
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: 0 client
connects (SSL_connect())
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: 0 client
connects that finished
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: 0 client
renegotiations requested
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: 4 server
connects (SSL_accept())
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: 4 server
connects that finished
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: 0 server
renegotiations requested
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: 0 session
cache hits
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: 0 external
session cache hits
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: 0 session
cache misses
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: 0 session
cache timeouts
Aug 18 04:23:09 ubuntu stunnel: LOG6[2029:3078101872]: SSL accepted: new
session negotiated
Aug 18 04:23:09 ubuntu stunnel: LOG6[2029:3078101872]: Negotiated
ciphers: AES256-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(256) Mac=SHA1
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: remote socket:
FD=1 allocated (non-blocking mode)
Aug 18 04:23:09 ubuntu stunnel: LOG6[2029:3078101872]: local_bind
succeeded on the original port
Aug 18 04:23:09 ubuntu stunnel: LOG6[2029:3078101872]: connect_blocking:
connecting 192.168.0.103:6667
Aug 18 04:23:09 ubuntu stunnel: LOG7[2029:3078101872]: connect_blocking:
s_poll_wait 192.168.0.103:6667: waiting 10 seconds
Aug 18 04:23:19 ubuntu stunnel: LOG3[2029:3078101872]: connect_blocking:
s_poll_wait 192.168.0.103:6667: TIMEOUTconnect exceeded
Aug 18 04:23:19 ubuntu stunnel: LOG5[2029:3078101872]: Connection reset:
0 bytes sent to SSL, 0 bytes sent to socket
Aug 18 04:23:19 ubuntu stunnel: LOG7[2029:3078101872]: Service ircd
finished (0 left)
Aug 18 04:23:19 ubuntu stunnel: LOG7[2029:3078101872]: str_stats: 0
block(s), 0 byte(s)
Post by Michal Trojnara
Mike
_______________________________________________
stunnel-users mailing list
http://stunnel.mirt.net/mailman/listinfo/stunnel-users
Michal Trojnara
2011-08-18 11:32:32 UTC
Permalink
Post by Julian D. Seifert
Aug 18 04:23:09 ubuntu stunnel: LOG6[2029:3078101872]: local_bind
succeeded on the original port
This time local_bind worked!
Post by Julian D. Seifert
connecting 192.168.0.103:6667
s_poll_wait 192.168.0.103:6667: waiting 10 seconds
s_poll_wait 192.168.0.103:6667: TIMEOUTconnect exceeded
Now there is a problem with routing.

Make sure you've configured:
ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100
echo 0 >/proc/sys/net/ipv4/conf/lo/rp_filter

Use tcpdump to diagnose problems.

Mike
Julian D. Seifert
2011-08-18 11:50:46 UTC
Permalink
Post by Michal Trojnara
Post by Julian D. Seifert
Aug 18 04:23:09 ubuntu stunnel: LOG6[2029:3078101872]: local_bind
succeeded on the original port
This time local_bind worked!
Sorry I think the bind error happened with transparent=both but I'll
have to recheck my log on that.
Post by Michal Trojnara
Post by Julian D. Seifert
connecting 192.168.0.103:6667
s_poll_wait 192.168.0.103:6667: waiting 10 seconds
s_poll_wait 192.168.0.103:6667: TIMEOUTconnect exceeded
Now there is a problem with routing.
ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100
echo 0 >/proc/sys/net/ipv4/conf/lo/rp_filter
Well I just run a script with the contents of the stunnel documentation
--- snip ---
iptables -t mangle -N DIVERT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT
ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100
echo 0 >/proc/sys/net/ipv4/conf/lo/rp_filter
--- snap ---
***@ubuntu:~/stunnel-4.39/src# cat /proc/sys/net/ipv4/conf/lo/rp_filter
0
***@ubuntu:~/stunnel-4.39/src# ip rule show
0: from all lookup local
32765: from all fwmark 0x1 lookup 100
32766: from all lookup main
32767: from all lookup default
***@ubuntu:~/stunnel-4.39/src# ip ro show table 100
local default dev lo scope host
Post by Michal Trojnara
Use tcpdump to diagnose problems.
Well I created a dump file (see appendix) but I'm not sure if there is a
problem

Loading...