performance ?

[ple1n]

------------------------------------------------------------
Client connecting to 192.168.115.23, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  1] local 100.64.0.2 port 52176 connected with 192.168.115.23 port 5001 (icwnd/mss/irtt=5/536/785)
[ ID] Interval       Transfer     Bandwidth
[  1] 0.0000-20.0570 sec   414 MBytes   173 Mbits/sec

heres a release build

see https://github.com/xjasonlyu/tun2socks

[SajjadPourali]

Please only check ipstack with its examples or create some tests to measure or reproduce bugs.
I can not investigate use of it on other tools at this time.

[SajjadPourali]

Ipstack + tun:

iperf3-darwin -c 10.0.0.2 -p 5201 -R
Connecting to host 10.0.0.2, port 5201
Reverse mode, remote host 10.0.0.2 is sending
[  6] local 10.0.0.1 port 53644 connected to 10.0.0.2 port 5201
[ ID] Interval           Transfer     Bitrate         Rwnd
[  6]   0.00-1.00   sec   390 MBytes  3.27 Gbits/sec  64.0 KBytes
[  6]   1.00-2.00   sec   395 MBytes  3.32 Gbits/sec  64.0 KBytes
[  6]   2.00-3.00   sec   395 MBytes  3.31 Gbits/sec  64.0 KBytes
[  6]   3.00-4.00   sec   396 MBytes  3.32 Gbits/sec  64.0 KBytes
[  6]   4.00-5.00   sec   393 MBytes  3.29 Gbits/sec  64.0 KBytes
[  6]   5.00-6.00   sec   392 MBytes  3.29 Gbits/sec  64.0 KBytes
[  6]   6.00-7.00   sec   395 MBytes  3.31 Gbits/sec  64.0 KBytes
[  6]   7.00-8.00   sec   396 MBytes  3.32 Gbits/sec  64.0 KBytes
[  6]   8.00-9.00   sec   396 MBytes  3.32 Gbits/sec  64.0 KBytes
[  6]   9.00-10.00  sec   391 MBytes  3.28 Gbits/sec  64.0 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  6]   0.00-10.00  sec  3.85 GBytes  3.30 Gbits/sec    0             sender
[  6]   0.00-10.00  sec  3.85 GBytes  3.30 Gbits/sec                  receiver

iperf Done.

iperf3-darwin -c 10.0.0.2 -p 5201
Connecting to host 10.0.0.2, port 5201
[  6] local 10.0.0.1 port 53648 connected to 10.0.0.2 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd          RTT
[  6]   0.00-1.00   sec  37.2 MBytes   312 Mbits/sec    0   64.0 KBytes   1ms
[  6]   1.00-2.00   sec  36.7 MBytes   308 Mbits/sec    0   64.0 KBytes   1ms
[  6]   2.00-3.00   sec  36.8 MBytes   309 Mbits/sec    0   64.0 KBytes   1ms
[  6]   3.00-4.00   sec  36.0 MBytes   302 Mbits/sec    0   64.0 KBytes   1ms
[  6]   4.00-5.00   sec  36.2 MBytes   304 Mbits/sec    0   64.0 KBytes   1ms
[  6]   5.00-6.00   sec  36.0 MBytes   302 Mbits/sec    0   64.0 KBytes   1ms
[  6]   6.00-7.00   sec  35.7 MBytes   299 Mbits/sec    0   64.0 KBytes   1ms
[  6]   7.00-8.00   sec  37.3 MBytes   313 Mbits/sec    0   64.0 KBytes   1ms
[  6]   8.00-9.00   sec  36.3 MBytes   305 Mbits/sec    0   64.0 KBytes   1ms
[  6]   9.00-10.00  sec  35.9 MBytes   301 Mbits/sec    0   64.0 KBytes   1ms
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  6]   0.00-10.00  sec   364 MBytes   305 Mbits/sec    0             sender
[  6]   0.00-10.00  sec   364 MBytes   305 Mbits/sec                  receiver

iperf Done.

[SajjadPourali]

Localhost:

iperf3-darwin -c 127.0.0.1 -p 5201 -R
Connecting to host 127.0.0.1, port 5201
Reverse mode, remote host 127.0.0.1 is sending
[  5] local 127.0.0.1 port 53655 connected to 127.0.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Rwnd
[  5]   0.00-1.00   sec  9.89 GBytes  84.9 Gbits/sec  3.80 MBytes
[  5]   1.00-2.00   sec  9.77 GBytes  83.9 Gbits/sec  3.87 MBytes
[  5]   2.00-3.00   sec  9.50 GBytes  81.6 Gbits/sec  3.62 MBytes
[  5]   3.00-4.00   sec  9.81 GBytes  84.3 Gbits/sec  3.87 MBytes
[  5]   4.00-5.00   sec  8.59 GBytes  73.7 Gbits/sec  3.25 MBytes
[  5]   5.00-6.00   sec  7.13 GBytes  61.2 Gbits/sec  3.69 MBytes
[  5]   6.00-7.00   sec  9.06 GBytes  77.9 Gbits/sec  4.00 MBytes
[  5]   7.00-8.00   sec  8.86 GBytes  76.1 Gbits/sec  3.56 MBytes
[  5]   8.00-9.00   sec  7.76 GBytes  66.7 Gbits/sec  4.00 MBytes
[  5]   9.00-10.00  sec  8.51 GBytes  73.1 Gbits/sec  3.94 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  88.9 GBytes  76.3 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  88.9 GBytes  76.3 Gbits/sec                  receiver

iperf Done.

iperf3-darwin -c 127.0.0.1 -p 5201
Connecting to host 127.0.0.1, port 5201
[  5] local 127.0.0.1 port 53653 connected to 127.0.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd          RTT
[  5]   0.00-1.00   sec  10.0 GBytes  86.1 Gbits/sec    0   4.00 MBytes   1ms
[  5]   1.00-2.00   sec  10.1 GBytes  87.0 Gbits/sec    0   4.00 MBytes   1ms
[  5]   2.00-3.00   sec  9.40 GBytes  80.8 Gbits/sec    0   4.00 MBytes   1ms
[  5]   3.00-4.00   sec  9.49 GBytes  81.5 Gbits/sec    0   4.00 MBytes   1ms
[  5]   4.00-5.00   sec  9.45 GBytes  81.2 Gbits/sec    0   4.00 MBytes   1ms
[  5]   5.00-6.00   sec  9.77 GBytes  83.9 Gbits/sec    0   4.00 MBytes   1ms
[  5]   6.00-7.00   sec  10.0 GBytes  85.9 Gbits/sec    0   4.00 MBytes   1ms
[  5]   7.00-8.00   sec  9.53 GBytes  81.8 Gbits/sec    0   4.00 MBytes   1ms
[  5]   8.00-9.00   sec  9.75 GBytes  83.7 Gbits/sec    0   4.00 MBytes   1ms
[  5]   9.00-10.00  sec  9.61 GBytes  82.5 Gbits/sec    0   4.00 MBytes   1ms
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  97.2 GBytes  83.5 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  97.2 GBytes  83.5 Gbits/sec                  receiver

iperf Done.

[ple1n]

noume@proxied ~/nsproxy (tun2socks5)> iperf3 -c 192.168.0.107 -R 
Connecting to host 192.168.0.107, port 5201
Reverse mode, remote host 192.168.0.107 is sending
[  5] local 100.64.0.2 port 46298 connected to 192.168.0.107 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   278 MBytes  2.33 Gbits/sec                  
[  5]   1.00-2.00   sec   268 MBytes  2.25 Gbits/sec                  
[  5]   2.00-3.00   sec   245 MBytes  2.05 Gbits/sec                  
[  5]   3.00-4.00   sec   226 MBytes  1.90 Gbits/sec                  
[  5]   4.00-5.00   sec   212 MBytes  1.78 Gbits/sec                  
[  5]   5.00-6.00   sec   199 MBytes  1.67 Gbits/sec                  
[  5]   6.00-7.00   sec   190 MBytes  1.60 Gbits/sec                  
[  5]   7.00-8.00   sec   180 MBytes  1.51 Gbits/sec                  
[  5]   8.00-9.00   sec   169 MBytes  1.42 Gbits/sec                  
[  5]   9.00-10.00  sec   161 MBytes  1.35 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.09 GBytes  1.80 Gbits/sec  190             sender
[  5]   0.00-10.00  sec  2.08 GBytes  1.78 Gbits/sec                  receiver

iperf Done.
noume@proxied ~/nsproxy (tun2socks5)> iperf3 -c 192.168.0.107 
Connecting to host 192.168.0.107, port 5201
[  5] local 100.64.0.2 port 44332 connected to 192.168.0.107 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  44.2 MBytes   371 Mbits/sec    0   16.8 KBytes       
[  5]   1.00-2.00   sec  45.7 MBytes   383 Mbits/sec    0   16.8 KBytes       
[  5]   2.00-3.00   sec  45.1 MBytes   378 Mbits/sec    0   16.8 KBytes       
[  5]   3.00-4.00   sec  45.2 MBytes   379 Mbits/sec    0   16.8 KBytes       
[  5]   4.00-5.00   sec  44.8 MBytes   376 Mbits/sec    0   16.8 KBytes       
[  5]   5.00-6.00   sec  45.1 MBytes   378 Mbits/sec    0   16.8 KBytes       
[  5]   6.00-7.00   sec  46.0 MBytes   386 Mbits/sec    0   16.8 KBytes       
[  5]   7.00-8.00   sec  45.5 MBytes   382 Mbits/sec    0   16.8 KBytes       
[  5]   8.00-9.00   sec  44.8 MBytes   376 Mbits/sec    0   16.8 KBytes       
[  5]   9.00-10.00  sec  45.4 MBytes   381 Mbits/sec    0   16.8 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   452 MBytes   379 Mbits/sec    0             sender
[  5]   0.00-10.00  sec   452 MBytes   379 Mbits/sec                  receiver

iperf Done.

noume@noume-pc ~/nsproxy (tun2socks5)> iperf -c 127.0.0.1 -R
------------------------------------------------------------
Client connecting to 127.0.0.1, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
tcp connect failed: Connection refused
[  1] local 0.0.0.0 port 0 connected with 127.0.0.1 port 5001 (reverse)
noume@noume-pc ~/nsproxy (tun2socks5)> iperf3 -c 127.0.0.1 -R
Connecting to host 127.0.0.1, port 5201
Reverse mode, remote host 127.0.0.1 is sending
[  5] local 127.0.0.1 port 40580 connected to 127.0.0.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  4.58 GBytes  39.4 Gbits/sec                  
[  5]   1.00-2.00   sec  5.22 GBytes  44.9 Gbits/sec                  
[  5]   2.00-3.00   sec  4.80 GBytes  41.2 Gbits/sec                  
[  5]   3.00-4.00   sec  5.19 GBytes  44.6 Gbits/sec                  
[  5]   4.00-5.00   sec  5.15 GBytes  44.2 Gbits/sec                  
[  5]   5.00-6.00   sec  5.53 GBytes  47.5 Gbits/sec                  
[  5]   6.00-7.00   sec  5.11 GBytes  43.9 Gbits/sec                  
[  5]   7.00-8.00   sec  4.82 GBytes  41.4 Gbits/sec                  
[  5]   8.00-9.00   sec  5.06 GBytes  43.5 Gbits/sec                  
[  5]   9.00-10.00  sec  5.49 GBytes  47.2 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  50.9 GBytes  43.8 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  50.9 GBytes  43.8 Gbits/sec                  receiver

iperf Done.
noume@noume-pc ~/nsproxy (tun2socks5)> iperf3 -c 127.0.0.1 
Connecting to host 127.0.0.1, port 5201
[  5] local 127.0.0.1 port 49670 connected to 127.0.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  4.91 GBytes  42.2 Gbits/sec    0   1.37 MBytes       
[  5]   1.00-2.00   sec  4.41 GBytes  37.9 Gbits/sec    0   1.37 MBytes       
[  5]   2.00-3.00   sec  4.72 GBytes  40.5 Gbits/sec    0   1.37 MBytes       
[  5]   3.00-4.00   sec  4.80 GBytes  41.2 Gbits/sec    0   1.37 MBytes       
[  5]   4.00-5.00   sec  4.95 GBytes  42.5 Gbits/sec    0   1.37 MBytes       
[  5]   5.00-6.00   sec  4.61 GBytes  39.6 Gbits/sec    0   1.37 MBytes       
[  5]   6.00-7.00   sec  4.66 GBytes  40.0 Gbits/sec    0   1.37 MBytes       
[  5]   7.00-8.00   sec  4.94 GBytes  42.4 Gbits/sec    0   1.37 MBytes       
[  5]   8.00-9.00   sec  4.81 GBytes  41.3 Gbits/sec    0   1.37 MBytes       
[  5]   9.00-10.00  sec  5.02 GBytes  43.2 Gbits/sec    0   1.37 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  47.8 GBytes  41.1 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  47.8 GBytes  41.1 Gbits/sec                  receiver

iperf Done.

^ mtu 65535. Later I tried mtu 9000 the difference is negligible

[ple1n]

Benchmark of https://github.com/xjasonlyu/tun2socks

root@noume-pc ~/nsproxy# iperf3 -c 192.168.0.107 -R
Connecting to host 192.168.0.107, port 5201
Reverse mode, remote host 192.168.0.107 is sending
[  5] local 100.67.0.0 port 59682 connected to 192.168.0.107 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   115 MBytes   966 Mbits/sec                  
[  5]   1.00-2.00   sec   122 MBytes  1.03 Gbits/sec                  
[  5]   2.00-3.00   sec   124 MBytes  1.04 Gbits/sec                  
[  5]   3.00-4.00   sec   122 MBytes  1.03 Gbits/sec                  
[  5]   4.00-5.00   sec   124 MBytes  1.04 Gbits/sec                  
[  5]   5.00-6.00   sec   123 MBytes  1.03 Gbits/sec                  
[  5]   6.00-7.00   sec   123 MBytes  1.03 Gbits/sec                  
[  5]   7.00-8.00   sec   122 MBytes  1.03 Gbits/sec                  
[  5]   8.00-9.00   sec   121 MBytes  1.02 Gbits/sec                  
[  5]   9.00-10.00  sec   120 MBytes  1.01 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.20 GBytes  1.03 Gbits/sec  105             sender
[  5]   0.00-10.00  sec  1.19 GBytes  1.02 Gbits/sec                  receiver

iperf Done.
root@noume-pc ~/nsproxy# iperf3 -c 192.168.0.107 
Connecting to host 192.168.0.107, port 5201
[  5] local 100.67.0.0 port 33982 connected to 192.168.0.107 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   167 MBytes  1.40 Gbits/sec    0    526 KBytes       
[  5]   1.00-2.00   sec   172 MBytes  1.44 Gbits/sec    0    526 KBytes       
[  5]   2.00-3.00   sec   167 MBytes  1.40 Gbits/sec    0    526 KBytes       
[  5]   3.00-4.00   sec   171 MBytes  1.43 Gbits/sec    0    526 KBytes       
[  5]   4.00-5.00   sec   166 MBytes  1.39 Gbits/sec    0    526 KBytes       
[  5]   5.00-6.00   sec   170 MBytes  1.43 Gbits/sec    0    526 KBytes       
[  5]   6.00-7.00   sec   170 MBytes  1.43 Gbits/sec    0    526 KBytes       
[  5]   7.00-8.00   sec   170 MBytes  1.43 Gbits/sec    0    526 KBytes       
[  5]   8.00-9.00   sec   168 MBytes  1.41 Gbits/sec    0    526 KBytes       
[  5]   9.00-10.00  sec   168 MBytes  1.41 Gbits/sec    0    526 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.65 GBytes  1.42 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  1.65 GBytes  1.42 Gbits/sec                  receiver

iperf Done.

[ple1n]

@ssrlive hi

[ple1n]

The benchmark of golang tun2socks used a veth pair. Heres the control

root@noume-pc ~/nsproxy# iperf3 -c 100.67.0.1 -R
Connecting to host 100.67.0.1, port 5201
Reverse mode, remote host 100.67.0.1 is sending
[  5] local 100.67.0.0 port 44154 connected to 100.67.0.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  4.89 GBytes  42.0 Gbits/sec                  
[  5]   1.00-2.00   sec  5.08 GBytes  43.6 Gbits/sec                  
[  5]   2.00-3.00   sec  5.43 GBytes  46.7 Gbits/sec                  
[  5]   3.00-4.00   sec  5.03 GBytes  43.2 Gbits/sec                  
^C[  5]   4.00-4.51   sec  2.77 GBytes  46.6 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-4.51   sec  0.00 Bytes  0.00 bits/sec                  sender
[  5]   0.00-4.51   sec  23.2 GBytes  44.2 Gbits/sec                  receiver
iperf3: interrupt - the client has terminated
root@noume-pc ~/nsproxy [1]# iperf3 -c 100.67.0.1 
Connecting to host 100.67.0.1, port 5201
[  5] local 100.67.0.0 port 45844 connected to 100.67.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  4.99 GBytes  42.9 Gbits/sec    0    728 KBytes       
[  5]   1.00-2.00   sec  5.32 GBytes  45.7 Gbits/sec    0    773 KBytes       
[  5]   2.00-3.00   sec  5.14 GBytes  44.2 Gbits/sec    0    773 KBytes       
[  5]   3.00-4.00   sec  4.98 GBytes  42.8 Gbits/sec    0   1.14 MBytes       
[  5]   4.00-5.00   sec  5.30 GBytes  45.5 Gbits/sec    0   1.14 MBytes       
^C[  5]   5.00-5.30   sec  1.57 GBytes  44.8 Gbits/sec    0   1.14 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.30   sec  27.3 GBytes  44.2 Gbits/sec    0             sender
[  5]   0.00-5.30   sec  0.00 Bytes  0.00 bits/sec                  receiver
iperf3: interrupt - the client has terminated

[SajjadPourali]

Correctness has a higher priority at this time. The performance issue with sending data is not difficult to fix; however, I do not have plans for it at this time. Nevertheless, pull requests are highly welcomed. As a clue, the memory allocation and management on the TCB can be improved.

https://github.com/narrowlink/ipstack/blob/main/src/stream/tcb.rs#L43-L44

[ssrlive]

Hi @SajjadPourali . I check the file tcp.rs with ChatGPT, it tells me some tips. I hope this tips is helpful.

This Rust code is an implementation of the TCP protocol,
and it appears to be complete and well-structured.
However, there are some potential issues and areas for improvement:

1. Handling of TCP window size: In the poll_write method,
   if the size of the send window is less than half of the average send window,
   or if the send buffer is full, the write operation is blocked.
   This could lead to performance issues, as write operations could be unnecessarily blocked.
   A possible improvement is to use a more complex algorithm
   to dynamically adjust the window size to adapt to changes in network conditions.

2. Termination of TCP connections: In the poll_read and poll_write methods,
   if a packet with the RST flag is received, or if a timeout occurs,
   the TCP connection is immediately terminated and all unsent data is discarded.
   This could lead to data loss.
   A possible improvement is to attempt to retransmit unacknowledged packets before terminating the connection.

3. Establishment of TCP connections: In the new method,
   if the SYN flag of the received TCP packet is not true,
   a packet with the RST and ACK flags is sent and an error is returned.
   However, according to RFC 793, if the SYN flag of the received packet is not true,
   the packet should be ignored, not a packet with the RST flag sent.

4. Closure of TCP connections: In the poll_shutdown method,
   if the shutdown state is None,
   it is set to Pending and the poll_read method is called to read and discard all remaining data.
   However, this could lead to data loss.
   A possible improvement is to attempt to read and process all remaining data before closing the connection.

[SajjadPourali]

@ssrlive Thank you for the details. I honestly cannot trust the information generated by the LLM. There are two features that have not yet been implemented and might have a negative impact sometimes, but not too serious:

1. Support for FIN, PUSH, ACK:

   • This only affects some connections during shutdown, which does not significantly impact transmission performance.

2. Timers to resend unacknowledged data:

   • Since ipstack is used in the local network interface, it is rare to encounter unacknowledged data.

Regarding the comments you mentioned:

1. Controlling the window size is part of the flow control mechanism that I implemented to prevent the send buffer from becoming full. Additionally, if the size of the send buffer is less than average, the flow control decreases the payload of the data logarithmically until receiving a new update for the buffer size.

2. This behavior is expected in the current implementation.

3. See rfc793 - page 64 under the "SEGMENT ARRIVES" heading

     If the state is CLOSED (i.e., TCB does not exist) then
   
     all data in the incoming segment is discarded.  An incoming
     segment containing a RST is discarded.  An incoming segment not
     containing a RST causes a RST to be sent in response.  The
     acknowledgment and sequence field values are selected to make the
     reset sequence acceptable to the TCP that sent the offending
     segment.```
   

4. It is expected behavior, as we cannot call poll_read after calling poll_shutdown.

[SajjadPourali]

Ipstack + tun:

iperf3-darwin -c 10.0.0.2 -p 5201 -R
Connecting to host 10.0.0.2, port 5201
Reverse mode, remote host 10.0.0.2 is sending
[  6] local 10.0.0.1 port 53644 connected to 10.0.0.2 port 5201
[ ID] Interval           Transfer     Bitrate         Rwnd
[  6]   0.00-1.00   sec   390 MBytes  3.27 Gbits/sec  64.0 KBytes
[  6]   1.00-2.00   sec   395 MBytes  3.32 Gbits/sec  64.0 KBytes
[  6]   2.00-3.00   sec   395 MBytes  3.31 Gbits/sec  64.0 KBytes
[  6]   3.00-4.00   sec   396 MBytes  3.32 Gbits/sec  64.0 KBytes
[  6]   4.00-5.00   sec   393 MBytes  3.29 Gbits/sec  64.0 KBytes
[  6]   5.00-6.00   sec   392 MBytes  3.29 Gbits/sec  64.0 KBytes
[  6]   6.00-7.00   sec   395 MBytes  3.31 Gbits/sec  64.0 KBytes
[  6]   7.00-8.00   sec   396 MBytes  3.32 Gbits/sec  64.0 KBytes
[  6]   8.00-9.00   sec   396 MBytes  3.32 Gbits/sec  64.0 KBytes
[  6]   9.00-10.00  sec   391 MBytes  3.28 Gbits/sec  64.0 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  6]   0.00-10.00  sec  3.85 GBytes  3.30 Gbits/sec    0             sender
[  6]   0.00-10.00  sec  3.85 GBytes  3.30 Gbits/sec                  receiver

iperf Done.

iperf3-darwin -c 10.0.0.2 -p 5201
Connecting to host 10.0.0.2, port 5201
[  6] local 10.0.0.1 port 53648 connected to 10.0.0.2 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd          RTT
[  6]   0.00-1.00   sec  37.2 MBytes   312 Mbits/sec    0   64.0 KBytes   1ms
[  6]   1.00-2.00   sec  36.7 MBytes   308 Mbits/sec    0   64.0 KBytes   1ms
[  6]   2.00-3.00   sec  36.8 MBytes   309 Mbits/sec    0   64.0 KBytes   1ms
[  6]   3.00-4.00   sec  36.0 MBytes   302 Mbits/sec    0   64.0 KBytes   1ms
[  6]   4.00-5.00   sec  36.2 MBytes   304 Mbits/sec    0   64.0 KBytes   1ms
[  6]   5.00-6.00   sec  36.0 MBytes   302 Mbits/sec    0   64.0 KBytes   1ms
[  6]   6.00-7.00   sec  35.7 MBytes   299 Mbits/sec    0   64.0 KBytes   1ms
[  6]   7.00-8.00   sec  37.3 MBytes   313 Mbits/sec    0   64.0 KBytes   1ms
[  6]   8.00-9.00   sec  36.3 MBytes   305 Mbits/sec    0   64.0 KBytes   1ms
[  6]   9.00-10.00  sec  35.9 MBytes   301 Mbits/sec    0   64.0 KBytes   1ms
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  6]   0.00-10.00  sec   364 MBytes   305 Mbits/sec    0             sender
[  6]   0.00-10.00  sec   364 MBytes   305 Mbits/sec                  receiver

iperf Done.

@ssrlive I was reconsidering the iperf3 results and I found out that although the performance has improved, there is a problem that we have to identify and fix. Could you please check the following benchmarks?

iperf3-darwin -c 10.0.0.1 -p 5201 -R                                                                                                                                      24-04-18
Connecting to host 10.0.0.1, port 5201
Reverse mode, remote host 10.0.0.1 is sending
[  5] local 10.0.0.33 port 54299 connected to 10.0.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Rwnd
[  5]   0.00-1.00   sec   593 MBytes  4.97 Gbits/sec  64.0 KBytes
[  5]   1.00-2.00   sec   609 MBytes  5.11 Gbits/sec  64.0 KBytes
[  5]   2.00-3.00   sec   607 MBytes  5.10 Gbits/sec  64.0 KBytes
[  5]   3.00-4.00   sec   609 MBytes  5.11 Gbits/sec  64.0 KBytes
[  5]   4.00-5.01   sec   103 MBytes   857 Mbits/sec  64.0 KBytes <--- Issue
[  5]   5.01-6.00   sec  0.00 Byte s  0.00 bits/sec  64.0 KBytes <--- Issue
[  5]   6.00-7.00   sec  0.00 Byte s  0.00 bits/sec  64.0 KBytes <--- Issue
[  5]   7.00-8.00   sec  0.00 Byte s  0.00 bits/sec  64.0 KBytes <--- Issue
[  5]   8.00-9.00   sec  0.00 Byte s  0.00 bits/sec  64.0 KBytes <--- Issue
[  5]   9.00-10.00  sec  0.00 Byte s  0.00 bits/sec  64.0 KBytes <--- Issue
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec  2.46 GBytes  2.12 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  2.46 GBytes  2.11 Gbits/sec                  receiver

iperf Done.
-----
iperf3-darwin -c 10.0.0.1 -p 5201                                                                                                                                         24-04-18
Connecting to host 10.0.0.1, port 5201
[  5] local 10.0.0.33 port 54291 connected to 10.0.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd          RTT
[  5]   0.00-1.00   sec  60.7 MBytes   509 Mbits/sec    0   64.0 KBytes   1ms
[  5]   1.00-2.00   sec  62.5 MBytes   524 Mbits/sec    0   64.0 KBytes   1ms
[  5]   2.00-3.00   sec  62.3 MBytes   523 Mbits/sec    0   64.0 KBytes   1ms
[  5]   3.00-4.00   sec  62.3 MBytes   522 Mbits/sec    0   64.0 KBytes   1ms
[  5]   4.00-5.00   sec  62.5 MBytes   524 Mbits/sec    0   64.0 KBytes   1ms
[  5]   5.00-6.00   sec  62.1 MBytes   521 Mbits/sec    0   64.0 KBytes   1ms
[  5]   6.00-7.00   sec  63.0 MBytes   528 Mbits/sec    0   64.0 KBytes   1ms
[  5]   7.00-8.00   sec  62.7 MBytes   526 Mbits/sec    0   64.0 KBytes   1ms
[  5]   8.00-9.00   sec  62.2 MBytes   521 Mbits/sec    0   64.0 KBytes   1ms
[  5]   9.00-10.00  sec  62.9 MBytes   528 Mbits/sec    0   64.0 KBytes   1ms
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   623 MBytes   523 Mbits/sec    0             sender
[  5]   0.00-10.00  sec   623 MBytes   523 Mbits/sec                  receiver

[ssrlive]

Here are the testing. It looks fine.

root@e:~/tun2proxy# iperf3 -c 10.0.0.4 -P 1
Connecting to host 10.0.0.4, port 5201
Accepted connection from 10.0.0.3, port 50088
[  5] local 10.0.0.4 port 5201 connected to 10.0.0.3 port 50104
[  5] local 123.45.67.89 port 55762 connected to 10.0.0.4 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  4.12 MBytes  34.5 Mbits/sec
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  4.96 MBytes  41.6 Mbits/sec    0   39.8 KBytes
[  5]   1.00-2.00   sec  4.21 MBytes  35.3 Mbits/sec
[  5]   1.00-2.00   sec  4.19 MBytes  35.1 Mbits/sec    0   38.7 KBytes
[  5]   2.00-3.00   sec  4.17 MBytes  35.0 Mbits/sec
[  5]   2.00-3.00   sec  4.19 MBytes  35.1 Mbits/sec    0   49.2 KBytes
[  5]   3.00-4.00   sec  4.01 MBytes  33.7 Mbits/sec
[  5]   3.00-4.00   sec  3.92 MBytes  32.9 Mbits/sec    0   22.0 KBytes
[  5]   4.00-5.00   sec  3.84 MBytes  32.2 Mbits/sec
[  5]   4.00-5.00   sec  3.90 MBytes  32.7 Mbits/sec    0   20.9 KBytes
[  5]   5.00-6.00   sec  4.38 MBytes  36.8 Mbits/sec
[  5]   5.00-6.00   sec  4.45 MBytes  37.3 Mbits/sec    0   42.9 KBytes
[  5]   6.00-7.00   sec  4.16 MBytes  34.9 Mbits/sec
[  5]   6.00-7.00   sec  4.20 MBytes  35.2 Mbits/sec    0   37.7 KBytes
[  5]   7.00-8.00   sec  3.90 MBytes  32.7 Mbits/sec
[  5]   7.00-8.00   sec  3.90 MBytes  32.7 Mbits/sec    0   38.7 KBytes
[  5]   8.00-9.00   sec  4.19 MBytes  35.2 Mbits/sec
[  5]   8.00-9.00   sec  4.21 MBytes  35.3 Mbits/sec    0   49.2 KBytes
[  5]   9.00-10.00  sec  4.19 MBytes  35.2 Mbits/sec
[  5]  10.00-10.04  sec   161 KBytes  31.1 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.04  sec  41.3 MBytes  34.5 Mbits/sec                  receiver
[  5]   9.00-10.00  sec  4.19 MBytes  35.2 Mbits/sec    0   2.09 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  42.1 MBytes  35.3 Mbits/sec    0             sender
[  5]   0.00-10.04  sec  41.3 MBytes  34.5 Mbits/sec                  receiver

iperf Done.
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------

root@e:~/tun2proxy# iperf3 -c 10.0.0.4 -P 1 -R
Connecting to host 10.0.0.4, port 5201
Reverse mode, remote host 10.0.0.4 is sending
Accepted connection from 10.0.0.3, port 56050
[  5] local 123.45.67.89 port 34036 connected to 10.0.0.4 port 5201
[  5] local 10.0.0.4 port 5201 connected to 10.0.0.3 port 56056
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  18.8 MBytes   157 Mbits/sec   12   1023 KBytes
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  8.04 MBytes  67.4 Mbits/sec
[  5]   1.00-2.00   sec  8.75 MBytes  73.4 Mbits/sec    5    895 KBytes
[  5]   1.00-2.00   sec  8.49 MBytes  71.2 Mbits/sec
[  5]   2.00-3.00   sec  7.50 MBytes  62.9 Mbits/sec   10    895 KBytes
[  5]   2.00-3.00   sec  8.09 MBytes  67.9 Mbits/sec
[  5]   3.00-4.00   sec  7.50 MBytes  62.9 Mbits/sec    7    895 KBytes
[  5]   3.00-4.00   sec  7.62 MBytes  63.9 Mbits/sec
[  5]   4.00-5.00   sec  8.75 MBytes  73.4 Mbits/sec    8    895 KBytes
[  5]   4.00-5.00   sec  7.90 MBytes  66.3 Mbits/sec
[  5]   5.00-6.00   sec  7.50 MBytes  62.9 Mbits/sec    9    895 KBytes
[  5]   5.00-6.00   sec  7.87 MBytes  66.0 Mbits/sec
[  5]   6.00-7.00   sec  8.75 MBytes  73.4 Mbits/sec    5    895 KBytes
[  5]   6.00-7.00   sec  8.47 MBytes  71.0 Mbits/sec
[  5]   7.00-8.00   sec  8.75 MBytes  73.4 Mbits/sec   10    895 KBytes
[  5]   7.00-8.00   sec  8.59 MBytes  72.1 Mbits/sec
[  5]   8.00-9.00   sec  7.50 MBytes  62.9 Mbits/sec    4    895 KBytes
[  5]   8.00-9.00   sec  8.31 MBytes  69.7 Mbits/sec
[  5]   9.00-10.00  sec  8.75 MBytes  73.4 Mbits/sec    8    895 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.04  sec  92.5 MBytes  77.3 Mbits/sec   78             sender
[  5]   9.00-10.00  sec  8.06 MBytes  67.6 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.04  sec  92.5 MBytes  77.3 Mbits/sec   78             sender
[  5]   0.00-10.00  sec  81.5 MBytes  68.3 Mbits/sec                  receiver

iperf Done.
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------

root@e:~/tun2proxy# iperf3 -c 10.0.0.4 -P 1 -u
Connecting to host 10.0.0.4, port 5201
Accepted connection from 10.0.0.3, port 37398
[  5] local 10.0.0.4 port 5201 connected to 10.0.0.3 port 50241
[  5] local 123.45.67.89 port 50054 connected to 10.0.0.4 port 5201
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec   124 KBytes  1.01 Mbits/sec  0.300 ms  0/236 (0%)
[ ID] Interval           Transfer     Bitrate         Total Datagrams
[  5]   0.00-1.00   sec   128 KBytes  1.05 Mbits/sec  245
[  5]   1.00-2.00   sec   128 KBytes  1.05 Mbits/sec  0.104 ms  0/245 (0%)
[  5]   1.00-2.00   sec   128 KBytes  1.05 Mbits/sec  244
[  5]   2.00-3.00   sec   128 KBytes  1.05 Mbits/sec  0.243 ms  0/244 (0%)
[  5]   2.00-3.00   sec   128 KBytes  1.05 Mbits/sec  245
[  5]   3.00-4.00   sec   128 KBytes  1.05 Mbits/sec  0.163 ms  0/245 (0%)
[  5]   3.00-4.00   sec   128 KBytes  1.05 Mbits/sec  244
[  5]   4.00-5.00   sec   128 KBytes  1.05 Mbits/sec  0.279 ms  0/244 (0%)
[  5]   4.00-5.00   sec   128 KBytes  1.05 Mbits/sec  245
[  5]   5.00-6.00   sec   128 KBytes  1.05 Mbits/sec  0.143 ms  0/245 (0%)
[  5]   5.00-6.00   sec   128 KBytes  1.05 Mbits/sec  245
[  5]   6.00-7.00   sec   128 KBytes  1.05 Mbits/sec  0.590 ms  0/244 (0%)
[  5]   6.00-7.00   sec   128 KBytes  1.05 Mbits/sec  244
[  5]   7.00-8.00   sec   128 KBytes  1.05 Mbits/sec  0.500 ms  0/245 (0%)
[  5]   7.00-8.00   sec   128 KBytes  1.05 Mbits/sec  245
[  5]   8.00-9.00   sec   128 KBytes  1.05 Mbits/sec  0.936 ms  0/244 (0%)
[  5]   8.00-9.00   sec   128 KBytes  1.05 Mbits/sec  244
[  5]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec  0.103 ms  0/245 (0%)
[  5]  10.00-10.04  sec  4.71 KBytes  1.05 Mbits/sec  0.140 ms  0/9 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.04  sec  1.25 MBytes  1.04 Mbits/sec  0.140 ms  0/2446 (0%)  receiver
[  5]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec  245
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec  1.25 MBytes  1.05 Mbits/sec  0.000 ms  0/2446 (0%)  sender
[  5]   0.00-10.04  sec  1.25 MBytes  1.04 Mbits/sec  0.140 ms  0/2446 (0%)  receiver

iperf Done.
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------

root@e:~/tun2proxy# iperf3 -c 10.0.0.4 -P 1 -u -R
Connecting to host 10.0.0.4, port 5201
Reverse mode, remote host 10.0.0.4 is sending
Accepted connection from 10.0.0.3, port 35760
[  5] local 10.0.0.4 port 5201 connected to 10.0.0.3 port 53952
[  5] local 123.45.67.89 port 52426 connected to 10.0.0.4 port 5201
[ ID] Interval           Transfer     Bitrate         Total Datagrams
[  5]   0.00-1.00   sec   128 KBytes  1.05 Mbits/sec  245
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec   133 KBytes  1.09 Mbits/sec  0.576 ms  0/254 (0%)
[  5]   1.00-2.00   sec   128 KBytes  1.05 Mbits/sec  244
[  5]   1.00-2.00   sec   128 KBytes  1.05 Mbits/sec  1.706 ms  0/244 (0%)
[  5]   2.00-3.00   sec   128 KBytes  1.05 Mbits/sec  245
[  5]   2.00-3.00   sec   128 KBytes  1.05 Mbits/sec  0.105 ms  0/245 (0%)
[  5]   3.00-4.00   sec   128 KBytes  1.05 Mbits/sec  244
[  5]   3.00-4.00   sec   128 KBytes  1.05 Mbits/sec  1.526 ms  0/244 (0%)
[  5]   4.00-5.00   sec   128 KBytes  1.05 Mbits/sec  245
[  5]   4.00-5.00   sec   128 KBytes  1.05 Mbits/sec  0.387 ms  0/245 (0%)
[  5]   5.00-6.00   sec   128 KBytes  1.05 Mbits/sec  245
[  5]   5.00-6.00   sec   128 KBytes  1.05 Mbits/sec  0.219 ms  0/244 (0%)
[  5]   6.00-7.00   sec   128 KBytes  1.05 Mbits/sec  244
[  5]   6.00-7.00   sec   128 KBytes  1.05 Mbits/sec  0.152 ms  0/245 (0%)
[  5]   7.00-8.00   sec   128 KBytes  1.05 Mbits/sec  245
[  5]   7.00-8.00   sec   128 KBytes  1.05 Mbits/sec  0.708 ms  0/245 (0%)
[  5]   8.00-9.00   sec   128 KBytes  1.05 Mbits/sec  244
[  5]   8.00-9.00   sec   128 KBytes  1.05 Mbits/sec  0.138 ms  0/244 (0%)
[  5]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec  245
[  5]  10.00-10.04  sec  4.71 KBytes  1.03 Mbits/sec  9
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.04  sec  1.25 MBytes  1.05 Mbits/sec  0.000 ms  0/2455 (0%)  sender
[  5]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec  0.748 ms  0/244 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.04  sec  1.25 MBytes  1.05 Mbits/sec  0.000 ms  0/2455 (0%)  sender
[  5]   0.00-10.00  sec  1.25 MBytes  1.05 Mbits/sec  0.748 ms  0/2454 (0%)  receiver

iperf Done.
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------

[ssrlive]

Please look this discuss #43 @SajjadPourali

PS C:\> iperf3 -c 10.0.0.1 -t 1
Connecting to host 10.0.0.1, port 5201
[  5] local 10.0.0.33 port 56474 connected to 10.0.0.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.01   sec  12.6 MBytes   104 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.01   sec  12.6 MBytes   104 Mbits/sec                  sender
[  5]   0.00-1.02   sec  12.4 MBytes   102 Mbits/sec                  receiver

iperf Done.
PS C:\>
PS C:\>
PS C:\> iperf3 -c 10.0.0.1 -t 1 -R
Connecting to host 10.0.0.1, port 5201
Reverse mode, remote host 10.0.0.1 is sending
[  5] local 10.0.0.33 port 56478 connected to 10.0.0.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.01   sec  48.6 MBytes   402 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.02   sec  49.2 MBytes   406 Mbits/sec                  sender
[  5]   0.00-1.01   sec  48.6 MBytes   402 Mbits/sec                  receiver

iperf Done.