Setting Up nginx As a Proxy on CentOS

This post will go over how to set up nginx to proxy and load balance over multiple servers on CentOS.

Step 1:
Install nginx.  If you have not already done so, install the epel repository:

wget http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
rpm -ivh epel-release-6-8.noarch.rpm
yum install nginx

The configuration file is stored in /etc/haproxy/haproxy.cfg.  You may want to back this file up as it will be modified in the next step.  You can also set up the service to start at boot:

chkconfig nginx on

Step 2:
Set up the server configuration.  The configuration file that will be modified is /etc/nginx/conf.d/default.conf.  You may want to back up this file before proceeding.  Some of the changes that will be made to this file are shown below.

server{
     listen 1.1.1.1:80;
     server_name example.com;

     location / {
          proxy_pass http://webservice;
     }
}

upstream webservice{
     server 10.0.0.2:80;
     server 10.0.0.3:80;
}

Note that the backend servers will be apache servers.

Step 3:
Set up the backend servers defined in haproxy.cfg.  In this example, just the apache test page is being used.  Apache can be installed with a simple

yum install httpd
service httpd on

Step 4:
Verify functionality.Checking the access logs on the nginx machine at /var/logs/nginx/access.log show the client making the connection.  Checking the apache logs at /var/logs/httpd/access.log on the servers behind the proxy show the requests being load balanced across both machines.  Taking down one of the apache servers results in the site staying up, but requests only being passed to the active server.

Setting Up HAProxy on CentOS

This post will go over installing and setting up HAProxy on CentOS.  HAProxy is a load balancer, and in this case, will have a public ip of 1.1.1.1 with 10.0.0.0/8 as the private network load balancing http.

Step 1:
Install HAProxy.  If you have not already done so, install the epel repository.

wget http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
rpm -ivh epel-release-6-8.noarch.rpm
yum install haproxy

The configuration file is stored in /etc/haproxy/haproxy.cfg.  You may want to back this file up as it will be modified in the next step.  You can also set up the service to start at boot:

chkconfig haproxy on

Step 2:
Set up the haproxy.cfg file.  There are a number of configuration options for haproxy, most of which can be found here.  The below configuration is how this system will be set up.

global
  maxconn 2000 # this system only has 512MB of memory
  daemon
  nbproc 1 # specify the number of processors to use
  log 127.0.0.1 local2
  user haproxy
  group haproxy
defaults
  log global
  mode http
  option httplog
  option dontlognull
  option http-server-close
  option forwardfor except 127.0.0.0/8
  option redispatch
  retries 3
  timeout http-request 10s
  timeout queue 1m
  timeout connect 10s
  timeout client 1m
  timeout server 1m
  timeout http-keep-alive 10s
  timeout check 10s
  maxconn 1900
listen webservice 0.0.0.0:80
  mode http
  status enable
  stats uri /haproxy?stats
  stats realm Private
  stats auth admin:password
  balance roundrobin
  option httpclose
  option forwardfor
  server server2 10.0.0.2 check
  server server3 10.0.0.3 check

Once the configuration is set up, start the service:

service haproxy start

Step 3:
Set up the backend servers defined in haproxy.cfg.  In this example, just the apache test page is being used.  Apache can be installed with a simple

yum install httpd
service httpd on

Step 4:
Verify functionality.

Upon the initial connection, the stats page already shows some data.

A couple of page refreshes shows the counters going up and the load being spread across the two servers.

To get a better sense of how the system will perform, the FreeBSD tool siege will be used with 1000 concurrent hits.

Shutting down apache on one of the servers shows that the web page remains reachable, but all of the connections go to the one remaining server.

Iptables Basics IV

This post is a continuation of the previous post on some of the basic functions of the linux firewall.

To match a specific set of tcp flags, execute:

iptables -A INPUT -p tcp --tcp-flags FIN,PSH,URG FIN,PSH,URG -j DROP

--tcp-flags - Match the specified tcp flags.  The first group of flags specifies the tcp flags to examine, in this case, FIN,PSH,URG.  The second group of flags specifies which flags must be set, in this case, FIN,PSH,URG.  This statement explicitly drops the Xmas tree scan from nmap.  If you wanted to make sure a specific flag was unset, you would leave it off of the second set of flags, e.g. --tcp-flags FIN,PSH,URG FIN,PSH.  This statement would examine FIN,PSH,URG, and match if FIN and PSH were set, and URG was unset.  You can use an exclamation point to reverse the meaning of the match, e.g.  ! --tcp-flags FIN,PSH,URG FIN,PSH,URG would match when FIN,PSH,URG are unset.

To limit the number of connections, execute:

iptables -A INPUT -p tcp --tcp-flags SYN SYN -m limit --limit 3/s --limit-burst 5 -j ACCEPT
iptables -A INPUT -p tcp --tcp-flags SYN SYN -j LOG --log-prefix SYN-ATTACK:
iptables -A INPUT -p tcp --tcp-flags SYN SYN -j DROP

-m limit - use the limit module
--limit - Specifiy the maximum average matching rate.  Can also specify /m (minutes), /h(hour), /d(day).
--limit-burst - Specify the maximum initial number of packets to match.  This value gets reset every time the --limit that is specified is not reached.

The statements following the limit statement log any packets that go above the threshold and drop them.

Iptables Basics III

This post is a continuation of the previous post on some of the basic functions of the linux firewall.

To block traffic at layer 2 by mac address, execute:

iptables -A INPUT -m mac --mac-source 00:11:22:33:44:55 -j DROP

-m - specifies an extension module to test for a specific property.  You can see a list of possible command line options for a specific module by adding on a -h after the "-m mac" option, in this case, --mac-source is the only available option for this module.  You can see a list of all possible modules in the man page of iptables-extensions.
--mac-source - source mac address of the ethernet frame to drop.

To block ping requests, execute:

iptables -A INPUT -p icmp --icmp-type echo-request -j DROP

-p icmp - specifies the protocol of the packet to check
--icmp-type - the type of icmp packet to match.  A full list of possibilities can be seen by executing iptables -p icmp -h.  You can also add on --reject-with option if you want to provide more information to the source of the ping.

To block a range of addresses, execute:

iptables -A INPUT -m iprange --src-range 1.1.1.1-1.1.1.10 -j DROP

-m iprange - specifies an extension module to test for a specific property.  You can see a list of possible command line options for a specific module by adding on a -h after the "-m iprange" option, in this case, --src-range and --dst-range are the only available option for this module.

Iptables Basics II

This post is a continuation of the previous post on some of the basic functions of the linux firewall.

To block all incoming traffic, but allow outgoing traffic, execute:

iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT
iptables -A INPUT -m state --state NEW,ESTABLISHED -j ACCEPT

-m - specifies an extension module to test for a specific property.  You can see a list of possible command line options for a specific module by adding on a -h after the "-m state" option, in this case, --state is the only available option for this module.
--state - specifies state to match, which can be [INVALID|ESTABLISHED|NEW|RELATED|UNTRACKED]
-j - jump to the specified target chain.

The last rule is saying to allow NEW and ESTABLISHED connections back in; in other words, any connection that originated from the host.

To block a specific source ip address, execute:

iptables -A INPUT -s 1.1.1.1 -j DROP

To block an entire source network, execute:

iptables -A INPUT -s 1.1.1.1/8 -j DROP

To block a specific port, execute:

iptables -A INPUT -p tcp --dport 80 -j DROP

-p - specify the protocol to inspect
--dport - specify the port to match

To block traffic to a specific destination host or destination subnet, execute:

iptables -A OUTPUT -d 1.1.1.1 -j DROP
iptables -A OUTPUT -d 1.1.1.1/8 -j DROP

Iptables Basics I

This post will go over some of the basic functionality of iptables, the linux firewall.

To reset the rules of the firewall, execute:

iptables -F // delete every rule in the chain for the table
iptables -X // delete every non-builtin chain in the table
iptables -t nat -F // default table is filter, need to specify other tables.
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X

To display the rules of the firewall, as well as packet count/byte count, execute:

iptables -t <table> -L -n -v --line-numbers

-t - specify the table to act on, default is filter if nothing is specified.
-L - list the chains in the default table, which is the filter table.  Use -t to switch tables.
-n - list ip addresses and ports as numbers.
-v - verbose output.
--line-numbers - show the line number of the rule in the chain.

To dump the iptables rules to stdout, execute:

iptables-save > rules.txt

Rules that get printed out from the format above can be restored by executing:

iptables-restore < rules.txt

To delete a specific rule, use the output from above to get the line number of the rule, and execute:

iptables -D <chain> <line>
e.g. iptables -D INPUT 1

To insert a rule, execute:

iptables -I <chain> <line-number> <rule>
e.g. iptables -I INPUT 2 -s 1.1.1.1 -j DROP

To set the default policy for a chain, execute:

iptables -P <chain> <policy>
e.g. iptables -P INPUT DROP

To log a rule, execute:

iptables <rule> -j LOG --log-prefix "Log Rule: "
e.g. iptables -s 1.1.1.1 -j LOG --log-prefix "Log Rule: "

Note that --log-prefix can be up to 29 characters long, and is useful for grepping through logs.  You may want to also limit the number of logs generated.  This can be done by adding on:

-m limit --limit <average-limit> --limit-burst <burst-limit>
e.g. iptables -s 1.1.1.1 -m limit --limit 3/m --limit-burst 5 -j LOG \
--log-prefix "Log Rule: "

Note that this rule is non-terminating, meaning after executing, the next rule will be executed.

Booting into Rescue Mode in CentOS

This post will go over booting into rescue mode in CentOS.  To be able to boot into rescue mode, you will need to boot to the install media and select "Rescue installed system".

You will then need to pick a language, keyboard, and whether or not you want networking enabled.  Depending on the situation, you can then try to mount the system as read-write, read-only, or skip mounting the system.

Assuming the system has started successfully and the filesystems have been mounted successfully, you are now in runlevel 1, you now have a temporary root filesystem at /, and the actual system filesystems exist under /mnt.

You can then chroot to the filesystem to be able to execute commands as if you had booted to the actual os.  This will allow you to troubleshoot the system without the need for a full boot.

If all you need is root access to the local filesystem, you can boot into runlevel 1 from the grub boot screen by appending the word "single" at the end of the kernel line.