Handy Curl Commands

Curl is a super powerful tool that can be used for testing and scripting. It is probably one of my favorite tools and I use it on a daily basis in my work.
Posted below are some commands that I use for troubleshooting and they barely even scratch the surface of what this tool can do.

Pass a host header to an IP address to test a virtual host:
-I (Dumps HTTP response headers)
-k (ignores SSL errors)
-H (Passes host headers)

1
2
3
4
5
6
7
8
9
$ curl  -IkH 'Host: www.wafmaster.com' https://72.249.84.27
HTTP/1.1 200 OK
Date: Mon, 03 Sep 2018 11:45:37 GMT
Content-Type: text/html
Cache-Control: no-cache, no-store, must-revalidate
Pragma: no-cache
Expires: 0
Content-Length: 1668
Connection: keep-alive

Set a user agent on the request header:
-A (Set user agent)
-I (Dumps HTTP response headers)

1
2
3
4
5
6
7
8
9
$ curl  -A "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" -I https://www.wafmaster.com
HTTP/1.1 200 OK
Date: Mon, 03 Sep 2018 11:45:53 GMT
Content-Type: text/html
Cache-Control: no-cache, no-store, must-revalidate
Pragma: no-cache
Expires: 0
Content-Length: 1668
Connection: keep-alive

Test initial GET response time:
-s (Silent)
grep -v (Nix based OS excludes the HTML tags)

1
2
3
4
5
$ time curl -s  https://www.wafmaster.com | grep -v "<"
 
real	0m0.338s
user	0m0.019s
sys	0m0.007s

Run a verbose curl for extra information:
–verbose (Verbose mode)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
$ curl --verbose https://www.wafmaster.com
* Rebuilt URL to: https://www.wafmaster.com/
*   Trying 72.249.84.27...
* TCP_NODELAY set
* Connected to www.wafmaster.com (72.249.84.27) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=wafmaster.com
*  start date: Jul 14 10:23:56 2018 GMT
*  expire date: Oct 12 10:23:56 2018 GMT
*  subjectAltName: host "www.wafmaster.com" matched cert's "www.wafmaster.com"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
> GET / HTTP/1.1
> Host: www.wafmaster.com
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Mon, 03 Sep 2018 11:59:41 GMT
< Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.2k-fips PHP/5.4.16
< Last-Modified: Fri, 13 Jul 2018 10:16:12 GMT
< ETag: "eb-570dec4ee3fbe"
< Accept-Ranges: bytes
< Content-Length: 235
< Content-Type: text/html; charset=UTF-8
< 
<html>
<body bgcolor="black">
<center>
<p>Hello Nerd</p>
</center>
</html>
* Connection #0 to host www.wafmaster.com left intact

Display the version of curl you are using:
-V (version)

1
2
3
4
$ curl -V
curl 7.54.0 (x86_64-apple-darwin17.0) libcurl/7.54.0 LibreSSL/2.0.20 zlib/1.2.11 nghttp2/1.24.0
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz HTTP2 UnixSockets HTTPS-proxy

Nginx Reverse Proxy Example

In a previous post I shared a virtual host configuration for an Apache reverse proxy and thought I would do the same for Nginx.

Nginx is a very light weight and effective web server and reverse proxy!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
 server {
 
    listen 443;
    listen  [::]:443;
    server_name www.somedomain.com somedomain.com;
 
    ssl_certificate           /etc/ssl/certs/cert.pem;
    ssl_certificate_key       /etc/ssl/certs/key.pem;
 
    ssl on;
    ssl_session_cache  builtin:1000  shared:SSL:10m;
    ssl_protocols  TLSv1.1 TLSv1.2;
    ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
    ssl_prefer_server_ciphers on;
 
    access_log            /var/log/nginx/somedomain.com.access.log;
 
    proxy_set_header HOST $host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Custom-XFF $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 
 location / {
      # Fix the “It appears that your reverse proxy set up is broken" error.
      proxy_pass          https://10.0.1.10:443;
      proxy_read_timeout  90;
 
    }
  }

Why am I using WordPress? A security engineer’s perspective

There’s a very negative perception of WordPress in the security world but I’m not going to touch that topic today. What I will address is why I am running WordPress on my personal blog about security. Some folks in the information security community might find that to be ironic however there are very specific and tactical reasons for it.

I’ll outline my thoughts below..

Policy tuning: I use this site for dialing in WordPress policy for WAF deployments. Since WordPress has 59% of the CMS market share as of Q4 2017, it makes sense to invest in building these policies since the platform isn’t going to disappear anytime soon. There’s a lot you can do to protect WordPress if you build your policies correctly.

Threat Analytics: I’m able to discover the most popular attack methods threat actors are using on WordPress installations. The majority of attacks I saw in 2017 were targeted towards the  xmlrpc.php and wp-login.php scripts. That’s no surprise to be honest since these attacks are low hanging fruit. These attacks are simple in nature such as xmlrpc DDoS attempts or brute force login attempts to wp-admin. I have seen a rise in theme specific attacks and attempts to hit existing malicious payloads on hosts that are already compromised.

Geo Map Analysis: Watching the trends of where WordPress attacks originate is quite interesting. I’ve seen a recent uptick in activity originating from the United States, European Union, and South America. My belief is that the usual suspects originating from IP space with known bad reputations are using VPNs, proxies, and compromised hosts from regions with a better IP reputation to slip past the perimeter of IP based reputation filters and IDS/IPS systems. The traditional methods of geoblocking are becoming less effective and dynamic IP reputation filtering is the way to go.

Security Hardening: The best way to help clients secure WordPress is to learn the platform inside and out. Whether it’s hardening PHP, the web server,  setting proper permissions on the PHP scripts, auditing activities, obfuscating fingerprints, or enabling FIM, there are several ways you can lessen the risk posed by WordPress. The ultimate goal is to use a defense in depth strategy that drives up the skill level of the threat actor involved.

Enable X-Forwarded-For on IIS 8.5 and up

It’s critical to enable X-Forwarded-For Logging when behind a proxy or load balancer in order grab the true IP address of visitors.

To enable this in your IIS site follow the steps below:

Step 1. Select your website in the IIS management console. In this example, I have a test site called www.larmeir.com. Ensure that the logging format is W3C then click –> Select Fields.

Step 2. From the Select Fields menu click –> Add Field.

Step 3. From the Add Custom Field form add the following entries for X-Forwarded-For as shown in the screenshot.

Step 4. Click –>OK

Step 5. On the actions menu in the upper right hand corner of the IIS manager click –>Apply

Conclusion: If everything went well you will now have a second IIS log in your logging directory with will have a postfix of _x (this indicates a custom log).
This log will now contain the true IPs of visitors that are connecting to your website.

Apache Reverse Proxy Vhost Examples

For certain projects I’ll use Nginx or Apache as a reverse proxy to back end web servers. While Nginx is far more light weight and faster, Apache is the swiss army knife of web servers and has just about every feature you could need.

Here’s a couple of examples of Apache Reverse proxy vhosts.

SSL Proxy with SSL back end origin:

1
2
3
4
5
6
7
8
9
10
11
12
13
	ServerName yourdomain.com
	ServerAlias www.yourdomain.com
        ProxyPreserveHost On
        ProxyPass / http://192.10.2.11:80/
        ProxyPassReverse / http://192.10.2.11:80/
 
        ErrorLog ${APACHE_LOG_DIR}/yourdomainerror.log
        CustomLog ${APACHE_LOG_DIR}/yourdomainaccess.log combined
 
	RewriteEngine on
	RewriteCond %{SERVER_NAME} =yourdomain.com [OR]
	RewriteCond %{SERVER_NAME} =www.yourdomain.com
	RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]

Standard HTTP proxy with HTTP back end origin (Forced SSL Rewrite):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
<VirtualHost *:80>
	ServerName yourdomain.com
	ServerAlias www.yourdomain.com
        ProxyPreserveHost On
        ProxyPass / http://192.10.2.11:80/
        ProxyPassReverse / http://192.10.2.11:80/
 
        ErrorLog ${APACHE_LOG_DIR}/yourdomainerror.log
        CustomLog ${APACHE_LOG_DIR}/yourdomainaccess.log combined
 
	RewriteEngine on
	RewriteCond %{SERVER_NAME} =yourdomain.com [OR]
	RewriteCond %{SERVER_NAME} =www.yourdomain.com
	RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

I’ll do a write up in the future on the benefits of reverse proxy configurations.

Enabling X-Forwarded-For Logging In Apache 2.4

It’s critical to enable X-Forwarded-For Logging when behind a proxy or load balancer in order grab the true IP address of visitors.

To enable this in your Apache vhost configuration, simply add the following logging options:

1
2
3
4
5
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" proxy
SetEnvIf X-Forwarded-For "^.*\..*\..*\..*" forwarded
CustomLog "/var/log/httpd/youraccess.log" combined env=!forwarded
CustomLog "/var/log/httpd/youraccess.log" proxy env=forwarded

This is a simple fix to get real data on the IPs hitting your website!

Simple .htaccess rules to protect wp-login.php and xmlrpc.php

The majority of trash WordPress traffic is targeting wp-login.php with bruteforces and xmlrpc.php for pingback/dos attacks.
A simple solution? Enforce IP based ACLs via your web server.

Apache 2.4 .htaccess Example:

1
2
3
4
5
<Files wp-login.php>
 Require all denied
 # your IP below
 Require ip xxx.xxx.xxx.xxx
</Files>
1
2
3
4
5
<Files xmlrpc.php>
 Require all denied
 # your IP below
 Require ip xxx.xxx.xxx.xxx
</Files>

How to enable IPV6 and DHCP6 on a Fortigate firewall with FortiOS 5.2

Example Environment:
My IPv6 Subnet: xxxx:xxxx:0:106::1/64
My DHCP Subnet: xxxx:xxxx:0:cccc::1/64

Step 1. Configure external interface and set options:

1
2
3
4
5
6
7
8
9
10
config system interface
    edit "wan1"
        set alias "External"
            config ipv6
                set ip6-address xxxx:xxxx:0:106::2/64
                set ip6-allowaccess ping
                set ip6-manage-flag enable
                set ip6-other-flag enable
            end
    next

Step 2. Configure IPV6 Routes:

1
2
3
4
5
6
config router static6
    edit 1
        set device "wan1"
        set gateway xxxx:xxxx:0:106::1
    next
end

Step 3. Configure Internal Interface:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
 config system interface
    edit "Internal"
            config ipv6
                 set ip6-mode static
                 set ip6-address xxxx:xxxx:0:cccc::1/64
                 set ip6-allowaccess ping
                 set ip6-send-adv enable
                 set ip6-manage-flag enable
                 set ip6-other-flag enable
                 set ip6-max-interval 600
                 set ip6-min-interval 198
                 set ip6-link-mtu 0
                 set ip6-reachable-time 0
                 set ip6-retrans-time 3000
                 set ip6-default-life 1800
                 set ip6-hop-limit 0
                 set autoconf disable
                 set dhcp6-relay-service disable
            end
    next

Step 4. Configure DHCP6 Server And Options:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
config system dhcp6 server
    edit 1
        set dns-service specify
        set status enable
        set interface "internal"
            config ip-range
                edit 1
                    set end-ip xxxx:xxxx:0:cccc::6000
                    set start-ip xxxx:xxxx:0:cccc::1000
                next
            end
        set lease-time 10800
        set option1 0
        set option2 0
        set option3 0
        set rapid-commit disable
        set subnet xxxx:xxxx:0:cccc::/64
        set dns-server1 2001:4860:4860::8888
        set dns-server2 2001:4860:4860::4444
        set dns-server3 ::
    next
end