Added Response Phase for Nginx

[chaizhenhua]

Added Response Phase for Nginx

[brenosilva]

Hello,

Request body looks good. I'm doing some tests with response body and headers all looks fine. Just notice two stuffs:

1 - When the response body is compressed we use SecDisableBackendCompression On to decompress the data to the backend in reverse proxy. This is a feature for apache and i don't know if we can do something similar to Nginx ?

2 - When we finish the transaction and logging to audit log i'm seeing:

[24/Jan/2013:02:58:00 --0800] [standalone/sid#87528e0][rid#87a2808][/acao.php][4] Audit log: Logging this transaction.
[24/Jan/2013:02:58:00 --0800] [standalone/sid#87528e0][rid#87a2808][/acao.php][1] Audit log: Failed to lock global mutex: Permission denied
[24/Jan/2013:02:58:00 --0800] [standalone/sid#87528e0][rid#87a2808][/acao.php][1] Audit log: Failed to unlock global mutex: Permission denied

Any idea to fix it ?

Topics 1 and 2 should go to another pull. I will apply and close this.

Thanks

Breno

[brenosilva]

Hello,

Deny action is not working in request body phase. Looks like modsecProcessRequestBody is called but not sure if its return values is being checked ?

Thanks

Breno

[brenosilva]

Also in ngx_http_modsecurity_cody_filter. Looks like rc value is not right in the end of the functions. It is just a guess because looks like we should expect a HTTP code but it looks like not a HTTP code in the end.

If it is true and fixed it.. we just need to add something like:
if (rc < NGX_HTTP_SPECIAL_RESPONSE || rc >= 600) {
rc = NGX_HTTP_INTERNAL_SERVER_ERROR;
} else if (rc == NGX_HTTP_FORBIDDEN) {
rc = NGX_HTTP_FORBIDDEN;
}

I'm printing the rc value in debug log and it is 20014. Is it expected ?

and get deny action working for response body.

[brenosilva]

Just tested without the patch and the deny action worked fine in request body:

2013/01/24 23:55:07 [error] 3081#0: [client 192.168.0.102] ModSecurity: Access denied with code 403 (phase 2). Pattern match "price" at REQUEST_BODY. [file "/usr/local/nginx/conf/modsecurity.conf"] [line "223"] [id "113"] [hostname "standalone"] [uri "/acao.php"] [unique_id "12345"]
2013/01/24 23:55:07 [debug] 3081#0: *1 ModSecurity: status: 403, need action
2013/01/24 23:55:07 [debug] 3081#0: *1 http finalize request: 403, "/acao.php?" a:1, c:2
2013/01/24 23:55:07 [debug] 3081#0: *1 http special response: 403, "/acao.php?"
2013/01/24 23:55:07 [debug] 3081#0: *1 http set discard body
2013/01/24 23:55:07 [debug] 3081#0: *1 HTTP/1.1 403 Forbidden^M

Looks like something in the patch in not OK. My guess is something related to return codes (rc).

[chaizhenhua]

hi, brenosilva.

did the problem solved?

[brenosilva]

Hello chaizhenhua,

Looks like "yes". I was planning to release 2.7.3 soon. However i will be on vacation next week and i was contacted to release it just when i come back from vacation at 21th Feb. So we need to wait a little bit more.

Thanks

[chaizhenhua]

hi, brenosilva

I have reprodue the error, with following config:
'''
SecRequestBodyAccess Off
SecRule REQUEST_BODY "@contains attack" "phase:2,log,deny,id:1"
'''
but i can't get rc 20014 when debug. did your config look like this? did you print it in ngx_http_modsecurity_body_handler? 20014 means APR_EOF. if rc == APR_EOF we should correct it then return.

[chaizhenhua]

-rc = modsecProcessRequestBody(ctx->req);
+
+if(modsecIsRequestBodyAccessEnabled(ctx->req))
+rc = modsecProcessRequestBody(ctx->req);

may be we can move this if condition to ngx_http_modsecurity_handler like this so that we can skip read client request body phase.

-if (r->method == NGX_HTTP_POST) {
+if (modsecIsRequestBodyAccessEnabled(ctx->req)
+&&r->method == NGX_HTTP_POST) {
rc = ngx_http_read_client_request_body(r, ngx_http_modsecurity_body_handler);

[chaizhenhua]

1 - When the response body is compressed we use SecDisableBackendCompression On to decompress the data to the backend in reverse proxy. This is a feature for apache and i don't know if we can do something similar to Nginx ?
if we copy request_rec's out_headers to ngx_http_headers_out_t, i think it might work.

[brenosilva]

Just added a patch. Looks like fixed the issue when RequestBodyAccess Off.

Related to SecDisableBackendCompression On I still don't know how to do it in Nginx. Did you test this ngx_http_headers_out_ idea ?

[chaizhenhua]

if SecDisableBackendCompression set to On,
modsecurity unset "Accept-Encoding" and "TE" header in after request body phase, and then send to backend,
when recevied from backend, modsecurity recover "Accept-Encoding" and "TE" header. and nginx will compress body for us. is it right?

[brenosilva]

Yes. This is how it works with Apache. However i think we need to work with request_rec nginx version struct.

[brenosilva]

Hello chaizhenhua,

Looks like we have an issue #56
The user said it is seeing a PCRE version warn message. So we still don't know if the 100% cpu issue can be caused by some pcre rule or not. I ask this info, but we will have it later.

I cannot reproduce the issue. Let me know if you can.

Thanks

Breno

[brenosilva]

Looks like loading all CRS it trigger the issue

[brenosilva]

I can reproduce it just one time. Looking at error.log ... a few rules was executed in a kind a loop (960017 and 960032):

2013/04/04 07:09:26 [error] 2608#0: [client 192.168.0.101] ModSecurity: Warning. Pattern match "^[\d.:]+$" at REQUEST_HEADERS:Host. [file "/usr/local/nginx/conf/owasp-modsecurity-crs-2.2.6/base_rules/modsecurity_crs_21_protocol_anomalies.conf"] [line "98"] [id "960017"] [rev "2"] [msg "Host header is a numeric IP address"] [data "192.168.0.102"] [severity "WARNING"] [ver "OWASP_CRS/2.2.6"] [maturity "9"] [accuracy "9"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/IP_HOST"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] [tag "http://technet.microsoft.com/en-us/magazine/2005.01.hackerbasher.aspx"] [hostname ""] [uri "/index.html"] [unique_id "12345"]
2013/04/04 07:09:26 [error] 2609#0: [client 192.168.0.101] ModSecurity: Warning. Pattern match "^[\d.:]+$" at REQUEST_HEADERS:Host. [file "/usr/local/nginx/conf/owasp-modsecurity-crs-2.2.6/base_rules/modsecurity_crs_21_protocol_anomalies.conf"] [line "98"] [id "960017"] [rev "2"] [msg "Host header is a numeric IP address"] [data "192.168.0.102"] [severity "WARNING"] [ver "OWASP_CRS/2.2.6"] [maturity "9"] [accuracy "9"] [tag "OWASP_CRS/PROTOCOL_VIOLATION/IP_HOST"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] [tag "http://technet.microsoft.com/en-us/magazine/2005.01.hackerbasher.aspx"] [hostname ""] [uri "/index.html"] [unique_id "12345"]

[chaizhenhua]

did it relate to some rules in response headers/body phase?

[brenosilva]

I'm not sure any more. Looks like the multiple execution of OK because it is in different uris. So Shouldn't be the problem. And those rules are not in phase 3 and 4.

[brenosilva]

It is hard to reproduce. I cannot reproduce it anymore. Not sure if it could be a uninitialized variable problem ?

[brenosilva]

Ignore the slow request info. I notice my backend disk was full. Maybe this is the reason

[brenosilva]

Hello chaizhenhua,

Are you be able to reproduce the 100% cpu issue ?

Thanks

[chaizhenhua]

I can not reproduce and have no idea about it.