NDD Parameter tuning GSCBackline 2006 HewlettPackard Development Company

NDD Parameter tuning • GSC/Backline • 최원규 © 2006 Hewlett-Packard Development Company, L. P.

NDD Parameter The command ndd is a networking configuration tool used to customize the behavior of the networking kernel. To make the 11 i system more internet friendly and easier to get running "out of the box", some of the ndd tunables have been enhanced. Network parameter를 “tune”할 수 있는 tool 2

NDD / IP ip_forwarding HP hosts가 gateway로 사용시 IP을 forward할 것인가 말것인가. 0 -> forward안함. 1 -> forward함. 2 -> Interface가 2이상인 경우에만 forward함. Ver Min Max Default 11. 0 0 2 2 11. i 0 2 2 3

NDD / IP ip_ire_gw_probe Turns the Dead Gateway Detection on and off. HP-UX 11. 0에서는 default gateway를 ICMP을 이용하여 주기적으로 check하는 데 일부 router의 경우 ICMP에 응답을 하지 않는 경우가 있는데 대개의 경우 Firewall에서 default router로 ping이 않되게 setting 하는 경우가 대부분. 그때 이 시간을 넘어서면 router넘어로 network connection fail. ndd -get /dev/ip ip_ire_status | grep -e IRE_GATEWAY -e flag Ver Min Max Default 11. 0 0 1 1 11. i 0 1 1 4

NDD / IP ip_pmtu_strategy 0 -> any non-local networks have a maximum MTU of 576 1 -> All outbout packet에 DF(Don't Fragment)를 set한다. 의미는 HP machine의 outbound MTU size를 그대로 두라는 것이다. destination과 HP machine사이의 MTU size값이 틀려 fragmenetion이 필요한 경우 router로 부터 ICMP Fragmentation Needed를 받으면 IP는 MTU size을 수정한다. 이를 지원하지 않는 router의 경우는 순차적으로 작은 MTU을 사용한다. 2 -> Disable 3 -> pmtu option "-p“ Ver Min Max Default 11. 0 0 3 1 11. i 0 3 1 5

NDD / IP ip_ire_gw_probe_interval HP-UX 11. x에서는 default gateway를 ICMP을 이용하여 주기적으로 check하는데 일부 router의 경우 ICMP에 응답을 하지 않는 경우가 있는데 대개의 경우 Firewall에서 default router로 ping이 않되게 setting 하는 경우가 대부분. 그때 이 시간을 넘어서면 router넘어로 network connection fail. IBM 2212 router에 VRRP(Virtaul Router Redundancy Protocol RFC 2338)을 enable한 경우 primary router에 문제가 발생하여 backup으로 넘어가는 경우 이 backup router에서 ICMP ECHO reply을 하지 않음으로 HP machin에서 gateway를 인식하지 못함. CISCO의 HSRP(Hot Standby Routing Protocol)에서 firewall로 ICMP Reply을 disable한 경우. Ver Min Max Default 11. 0 15000 X 180000 11. i 15000 X 180000 6

NDD / IP ip_send_source_quench IP packets이 처리할 수 있는 용량보다 빨리 node에 도착하는것인데 ICMP packet을 받아서 처리 할때 만일 ICMP을 처리하는 Process가 system에 running 중인 경우 ICMP packet을 해당 process의 socket에 넣는데 socket full이 발생시 ICMP Source Quench Message가 상대방에게 보내진다. 이를 0로 하면 Source Quench을 check하지 않는다. Ver Min Max Default 11. 0 0 1 1 11. i 0 1 1 7

NDD / IP ip_max_bcast_ttl Hop count가 1로 bcast packet이 router을 넘어가지 못함. C class로 subnet하여 사용시. 199. 3. 201 199. 3. 208 20 subnet에서 braodcast가 208에 도달하게 하기위한 방법. Broadcast 하는 program에서 조정하는 방법. System gloval변수 조정. ndd -set /dev/ip ip_max_bcast_ttl 3 보통 16정도 주면 됨. 증권사에서 자주 사용함. Ver Min Max Default 11. 0 0 255 1 11. i 0 255 1 8

NDD / IP ip_check_subnet_addr Controls the subnet portion of a host address RFC 1122을 check, 즉 subnet mask 255. 192을 허용하지 않음. 0 -> RFC 1122 check을 bypass함. Host IP 205. 185. 219. 252 subnet mask 255. 192. The error he gets is: ifconfig: ioctl (SIOCSIFADDR). RFC 1122에 따른 Guide을 HP에서 준수하기 때문에 발생한 문제. Ver Min Max Default 11. 0 0 11. i 0 1 1 9

NDD / IP ip_strong_es_model • This tunable controls the support for the "Strong End-System Model" described in the RFC 1122, Section 3. 3. 4. 2. This means that the source address of a packet (that the host wants to send out) affects the selection of the gateway for the outbound packet. It is meaningful in the case that the host is multihomed (means several IP addresses for the interfaces). • • Here are the key requirements from the RFC regarding the multihomed environments for the ES model: A host MUST silently discard an incoming datagram whose destination address does not correspond to the physical interface through which it is received. A host MUST restrict itself to sending (non-sourcerouted) IP datagrams only through the physical interface that corresponds to the IP source address of the datagrams. In restricted network environments it may be a wanted behavior to know exactly which of the addresses are valid and which not. in most grown environments it is not very useful to enable this, since it will be not too easy to fulfill the restrictions always. Ver Min Max Default 11. 0 0 11. i 0 10

NDD / TCP tcp_conn_request_max Max number of outstanding connection request 동시에 Connection을 맺을 수 있는 각 socket에 대한 listen queue limit. socket program에서 listen()에서 define 할 수 있는 최대크기. Application에서의 Create한 socket port에대한 listen queue의 제한은 실제로 Application program내부의 listen()에서 사용된 size에의해서 제한을 받는다. /usr/include/sys/socket. h: #define SOMAXCONN 20 netstat –s : connect requests dropped due to full queue For a web server it might be a good idea to set this value to 1024 or higher. Ver Min Max Default 11. 0 1 undefined 20 11. i 1 undefined 4096 11

NDD / TCP tcp_recv_hiwater_def : receive buffer window size tcp_xmit_hiwater_def : send buffer window size 개별 TCP session에 대해 buffer size할당 Ver Min Max Default 11. 0 4096 Bytes undefined 32768(32 KB) 11. i 4096 Bytes undefined 32768(32 KB) 12

NDD / TCP tcp_fin_wait_2_timeout Ver Min Max Default 11. 0 0 undefined 0 11. i 0 undefined 0 13

NDD / TCP tcp_syn_rcvd_max Controls the SYN attack defense of TCP effective. SYN packet을(SYN_RECV in netstat -a) 보관하는 table space을 지정한다. 여기을 거쳐서 각 socket별로 listen queue에 들어 간다. Ver Min Max Default 11. 0 1 1000 500 connections 11. i 1 1000 500 14

NDD / TCP tcp_time_wait_interval How long stream persists in TIME_WAIT Amount of time TCP endpoints persist in TCPS_TIME_WAIT state. netstat -an으로 확인시 모든 TCP session은 정상적으로 Session이 종료시 TIME_WAIT 상태을 거치게 된다. 이 값을 조정하는 Parameter임. 10. X에서는 default로 2 MSL을 사용하며 MSL이 60 Second이기에 2분을 Waiting 하다 Session이 netstat -an시 보이지 않는다. 10. x OS에서는 조정이 불가능함. /usr/conf/h/protosw. h: #define PR_SLOWHZ 2 /usr/conf/netinet/tcp_timer. h: #define TCPTV_MSL ( 30*PR_SLOWHZ) Ver Min Max Default 11. 0 1000 (1 sec) 600000(10 min) 60000 (1 min) 11. i 1000 (1 sec) 600000(10 min) 60000 (1 min) 15

NDD / TCP tcp_status TCP dst rto mss [lport, fport] state snxt suna swnd cwnd rnxt rack rwnd 00000001015 d 1468 015. 043. 232. 202 361 e 667 e 00008000 000005 b 4 06 dda 013 00008000 24000 01460 ca 16, 9 e 9 TCP instance number destination IP-address snxt Senders next sequence number suna Data the sender has not acknowledged yet swnd Senders window (relative to suna) cwnd Congestion window rnxt Sequence number we expect to receive next rack Sequence number we have acknowledged rwnd Current receive window rto Round trip timeout mss Max segment size lport Source port fport Destination port state TCP state this connection is in 16

NDD / TCP tcp_discon ndd -get /dev/tcp tcp_status | grep -e state -e TCP_FIN_WAIT_2 tcp_discon - Terminate a TCP connection -> TCP session을 disconnection하기 위한 ndd option $ ndd -get /dev/tcp tcp_status TCP dest snxt suna swnd cwnd rnxt rack rwnd rto mss [lport, fport] state 0183 b 8 b 4 015. 043. 233. 086 533 cb 8 ce 00008000 00003000 533 bc 583 0000 02812 04096 [c 00 a, cea 9] TCP_FIN_WAIT_2 $ ndd -set /dev/tcp tcp_discon 0 x 0183 b 8 b 4 17

NDD / TCP tcp_discon_by_addr TCP session disconnection by address. Local IP: 192. 1. 2. 3 (0 xc 0010203) Local Port: 1024 (0 x 0400) Remote IP : 192. 4. 5. 6 (0 xc 0040506) Remote Port: 2049 (0 x 0801) $ ndd -set /dev/tcp tcp_discon_by_addr "c 00102030400 c 00405060801" 18

NDD / TCP retransmission(접속할때) tcp_ip_notify_cinterval [10, undefined] : R 1 timer tcp_ip_abort_cinterval [75, undefined] : R 2 timer Cinterval 같이 앞에 c가 붙는것은 보내는 것이다. 19

NDD / TCP retransmission(접속중일때) tcp_rexmit_interval_initial [3, 20] [0. 5, 20 : 11. 0] [1. 5, 20 : < PHNE_ 27730] tcp_rexmit_interval_min [0. 5, 20] tcp_rexmit_interval_max [1 m, 1 h] tcp_ip_notify_interval [10, undefined] : R 1 timer tcp_ip_abort_interval [10 m, undefined] : R 2 timer tcp_time_wait_inteval [1 m, 10 m] 20

NDD / TCP tcp_keepalive_interval [2 h] Idle connection에도 TCP상태를 가지고 있는 timer Established connection - tcp_ip_abort_interval [10 m, -] Connection establishment - tcp_ip_abort_cinterval [75 s, -] Connection terminating - tcp_keepalive_detached_interval [2 m, 10 d] setsockopt() 에서 enable해야 적용됨 21

NDD / Sockets socket_buf_max Specifies the maximum socket buffer size for AF_UNIX sockets. Ver Min Max Default 11. 0 X X X 11. i 1024 2147483647 256000 22

NDD / Sockets socket_caching_tcp (웹서버 용도로 사용될시 효과 만점) Used to enable socket caching on TCP AF_INET sockets. This value determines how many cached data structures for TCP sockets the system keeps. This could cause the system to speed up considerably if there are many short-lived connections on the system. A value between 1 and 512 will set a minimum of 512. Any number above 512 will set that value. Ver Min Max Default 11. 0 X X X 11. i 0 2147483647 0 23

Oracle DB Server Tuning Parameter Default value Suggested value ip_send_source_quench 1 0 tcp_conn_request_max 20000 tcp_rexmit_interval_max 60000 10000 tcp_rexmit_interval_min 500 1500 tcp_xmit_lowater_def 8192 24576 tcp_syn_rcvd_max 500 1024 tcp_xmit_hiwater_def 32768 1048576 24

Apache 1. 3. x Web Server Tuning Parameter Default value Suggested value tcp_conn_request_max 20 1024 tcp_fin_wait_2_timeout 0 (infinite) 900000 tcp_ip_abort_interval 600000 60000 tcp_keepalive_interval 72000000 900000 tcp_rexmit_interval_initial 500 Site-specific (see note) 25

Netscape & Iplanet Web Server Tining Parameter Default value Suggested value tcp_time_wait_interval 60000 tcp_conn_request_max 20 1024 --> 4096 tcp_ip_abort_interval 600000 60000 tcp_keepalive_interval 72000000 900000 tcp_rexmit_interval_initial 1500 tcp_rexmit_interval_max 60000 tcp_rexmit_interval_min 500 tcp_xmit_hiwater_def 32768 tcp_recv_hiwater_def 32768 26

ndd parameter for security Parameter Default value Suggested value Comment ip_forward_directed_broadcasts 1 0 Don't forward directed broadcasts ip_forward_src_routed 1 0 Don't forward packets with source route options ip_forwarding 2 0 Disable IP forwarding ip_ire_gw_probe 1 0 Disable dead gateway detection (currently no ndd help text; echo-requests interact badly with firewalls) ip_pmtu_strategy 1 1 Don't use echo-request PMTU strategy (can be used for amplification attacks and we don't want to send echo-requests anyway) ip_send_redirects 1 0 Don't send ICMP redirect messages (if we have no need to send redirects) ip_send_source_quench 1 0 Don't send ICMP source quench messages (deprecated) tcp_conn_request_max 20 500 Increase TCP listen queue maximum (performance) tcp_syn_rcvd_max 500 HP SYN flood defense ip_check_subnet_addr 1 0 Permit 0 in local network part (should be the default) ip_respond_to_address_mask_broadcast 0 0 Don't respond to ICMP address mask request broadcasts ip_respond_to_echo_broadcast 1 0 Don't respond to ICMP echo request broadcasts ip_respond_to_timestamp_broadcast 0 0 Don't respond to ICMP timestamp request broadcasts ip_respond_to_timestamp 0 0 Don't respond to ICMP timestamp requests tcp_text_in_resets 1 0 Don't send text messages in TCP RST segments (should be the default) 27

100 BT configuration file Configuration File in /etc/rc. config. d Variables btlan hpbtlanconf xxx_INTERFACE_NAME : Name of interface (lan 0, lan 1. . . ) xxx_STATION_ADDRESS : Station address of interface. This will be the MAC address of your interface. xxx_SPEED : set the card speed. Value are : 10 HD, 10 FD, 100 HD, 100 FD, auto_on. For EISA see Note 1. 11. 0 btlan 0 -> hpeisabtconf btlan 1 -> hpbasetconf btlan 3 -> hpbase 100 conf btlan 4 -> hpgsc 100 conf btlan 5 -> hppci 100 conf btlan 6 -> hpsppci 100 conf 11. I hpbtlanconf 28

Nettl trace packet 분석 tool #nettl -start #nettl -tn pduin pduout -e ns_ls_ip -s 1024 –tm 99999 -f /tmp/trace -> 이러면 /tmp/trace. TRC 00 와 trace 1. TRC 01 의 file이 생깁니다. 필요한 action 수행 #nettl -tf -e all #vi /tmp/filter ip_saddr 10. 1. 61. 212 filter ip_daddr 10. 1. 61. 212 -> 대상 장비의 IP #netfmt -c /tmp/filter -Nlf /tmp/trace. TRC 00 > /tmp/trace. txt 0 29

APA(Auto Port Aggregation) 1. Cisco’s Protocol for Automatic Trunk Discovery and Automatic Link Aggregate Configuration 2. High Availability via Multiple Links with Failover Capability 3. High Availability Servers 4. Load Balancing 5. High Throughput 6. Advanced Interoperability 7. Basic Interoperability 8. Single IP Address Capability 9. Flexibility 10. Application Transparency 30

APA(Example) /etc/rc. config. d/hp_apaconf HP_APA_INTERFACE_NAME[0]=lan 100 HP_APA_LOAD_BALANCE_MODE[0]=LB_MAC HP_APA_MANUAL_LA[0]=1, 2 HP_APA_HOT_STANDBY[0]=on /etc/rc. config. d/hp_apaportconf HP_APAPORT_INTERFACE_NAME[0]=lan 1 HP_APAPORT_GROUP_CAPABILITY[0]=5 HP_APAPORT_PRIORITY[0]=0 HP_APAPORT_CONFIG_MODE[0]=MANUAL # HP_APAPORT_INTERFACE_NAME[1]=lan 2 HP_APAPORT_GROUP_CAPABILITY[1]=5 HP_APAPORT_PRIORITY[1]=0 HP_APAPORT_CONFIG_MODE[1]=MANUAL /etc/rc. config. d/netconf INTERFACE_NAME[1]=lan 100 IP_ADDRESS[1]=150. 100. 1. 3 SUBNET_MASK[1]="255. 0. 0" BROADCAST_ADDRESS[1]="" INTERFACE_STATE[1]=up DHCP_ENABLE[1]=0 31

Utilization 측정 FTP 는 performance 측정도구로 적절하지 않다. System 에서 제공하는 command 는 lanadmin, netstat, glance 등이 있다 정확한 측정은 NMS나 analyze가 있어야 한다. 정확한 계산방법은 packet의 수가 아니라 bit 수이다. (If. In. Octet + If. Out. Octet) * 8 /If. Speed * 1/100 32

33