Multiple component wwsympa_url with mod_proxy_fcgi is broken

[ikedas]

Version

n/a

Installation method

RPM (EPEL 7)

Expected behavior

wwsympa_url with multiple component (e.g. http://host.domain/aa/bb) will be allowed.

Actual behavior

It is reported that configuration with mod_proxy_fcgi does not allow such URL: mod_fcgid allows it.

syslog message: wwsympa[pid_num]: info main:: ... "Unknown action bb"
pop-up message on the web interface: ERROR(bb) - unknown action

Additional information

Details: https://bugzilla.redhat.com/show_bug.cgi?id=1770783

[ikedas]

Current version of Sympa relys on PATH_INFO variable by which Sympa determines "extra-path" in script-URI. If a client accesses to http://server/aa/bb/home, following values of CGI environment variables are expected by Sympa:

REQUEST_URI=/aa/bb/home
SCRIPT_NAME=/aa/bb
PATH_INFO=/home

However, by the configuration with mod_proxy_fcgi:

    <Location /aa/bb>
        SetHandler "proxy:unix:/var/run/sympa/wwsympa.socket|fcgi://"
        Require all granted
    </Location>

If a client accesses to http://server/aa/bb/home, following CGI environment variables are set:

REQUEST_URI=/aa/bb/home
SCRIPT_NAME=/aa
PATH_INFO=/bb/home

I.e., the server takes the first component of requested URI as SCRIPT_NAME and the remainder as PATH_INFO. From my understanding, this is one of possible behaviors of the server (See RFC 3875, 3.3 and related sections).

---

Question: Should we deal with this problem by Sympa itself? If otherwise, which?

[bruncsak]

The first question, may we rely that the web server parses properly the REQUEST_URI, and sets the values for SCRIPT_NAME and PATH_INFO? I think, the answer is: no, as the problem shows. Unless, there is a configuration option for the web server (that case apache) to tell where to cut the REQUEST_URI.

I think the proper way is not to rely on SCRIPT_NAME and PATH_INFO. Use only REQUEST_URI and parse it based on the value of the wwsympa_url configuration variable(s).
Any of the main or robots wwsympa_url must match the REQUEST_URI. If none matches, the request is not for the sympa service at all. The longest match length decides it goes to which robot. (Theoretically, one robot's wwsympa_url can be substring of another one.) Cutting off the wwsympa_url part from the request, that must be a valid sympa command (for example, home, review, list, etc...)

[ikedas]

The first question, may we rely that the web server parses properly the REQUEST_URI, and sets the values for SCRIPT_NAME and PATH_INFO? I think, the answer is: no, as the problem shows. Unless, there is a configuration option for the web server (that case apache) to tell where to cut the REQUEST_URI.

Indeed we could write the code to parse REQUEST_URI by ourselves. However, in fact, REQUEST_URI is not a part of CGI specification. From view of CGI, the applications are expected to use SCRIPT_NAME, PATH_INFO (they are parts of the specification) and so on parsed by the CGI (web server).

The process to split script-URI into PATH_INFO etc. looks essentially in charge of the CGI (web server). So my opinion at the moment is that --- we'd be better to find the way to configure httpd, or to repair mod_proxy_fcgi.

[ikedas]

FYI.

Apache HTTPd 2.4.26 or later has ProxyFCGISetEnvIf Directive. Though I've not confirmed, it looks able to repair SCRIPT_NAME / PATH_INFO variable.

For RHEL/CentOS, RHSCL / SCL repository has httpd24-httpd package (currently providing httpd 2.4.27).

[bruncsak]

Yes, I think it might be a feasible solution for the problem. After positive testing, may it be a general suggestion for CentOS 7 never use the shipped version of apache server, but always use httpd24-httpd? If the answer is no, it falls back to a workaround status, not a solution.

[ikedas]

... After positive testing, may it be a general suggestion for CentOS 7 never use the shipped version of apache server, but always use httpd24-httpd? ...

No. There are many ways including SCL to use httpd 2.4.26+.

EDIT: In other words, using recent version of httpd is not a workaround but a legitimate solution.

[ikedas]

FYI.

nginx has similar problem, however it also has solution.

server {
    server_name web.example.org;
    ...

    location /aa/bb {
        include       /etc/nginx/fastcgi_params;
        fastcgi_pass  unix:/var/run/sympa/wwsympa.socket;

        fastcgi_split_path_info ^(/aa/bb)(.*)$;
        fastcgi_param PATH_INFO $fastcgi_path_info;
    }
    ...

By this configuration, SCRIPT_NAME and PATH_INFO will have expected values. If fastcgi_split_path_info line was not there, full path of (decoded) URI will be set to SCRIPT_NAME and PATH_INFO will be empty.
By this reason, if we want to change path of Sympa web interface, we have to change two parameters in nginx configuration: location and regular expression.

[ikedas]

lighttpd does not have problem.

...
$HTTP["url"] =~ "^/aa/bb" {
fastcgi.server = ( "/aa/bb" =>
    ((    "check-local"    =>    "disable",
        "bin-path"    =>    "/usr/libexec/sympa/wwsympa-wrapper.fcgi",
        "socket"    =>    "/var/run/lighttpd/sympa.sock",
        "max-procs"    =>     2,
        "idle-timeout"    =>     20,
    ))
)
}

Same as nginx: If we want to change path of Sympa web interface, we have to change two parameters in configuration.

[ikedas]

httpd using ProxyPass (or ProxyPassMatch)

ProxyPass "/aa/bb" "unix:/var/run/sympa/wwsympa.socket|fcgi://localhost"
<Location /aa/bb>
    Require all granted
</Location>

has the same problem as nginx without fastcgi_split_path_info: full path of (decoded) URI will be set to SCRIPT_NAME and PATH_INFO will be empty.

httpd 2.4.11 and later may "guess" better PATH_INFO using proxy-fcgi-pathinfo env-var, however it looks not so useful for our purpose. ProxyFCGISetEnvIf of httpd 2.4.26+ is desirable.

[ikedas]

In conclusion: Although most of recent HTTPd software can adjust PATH_INFO, adjusting it by WWSympa can simplify configuration.

@bruncsak , could you please apply this patch and check if it works as expected?

[ikedas]

@bruncsak , if you have difficulty to apply patch, please check the next beta (perhaps released in a few months): Changes may be included in it.

[xavierba]

@ikedas, I'm unsure if @bruncsak and the Red Hat bugzilla reporter are the same person, but I provided a link to prebuilt EL7 RPMs.

[bruncsak]

@xavierba,
Yes it is me, just a different e-mail address at registration of Red Hat bugzilla, the account is shared with my colleague.

Thanks for the packaging!

@ikedas,

It is always returning with the following HTTP status code and message in the HTTP header:
HTTP/1.1 421 Misdirected Request and no data.
The maximum message in /var/log/sympa.log I have is ( with log_level 5):

wwsympa[12345]: debug2 Sympa::List::get_robots()

I tried to reconfigure to wwsympa_url to be just /sympa (of course same as location in /etc/httpd/conf.d/sympa.conf), the same thing as with multiple component path. Sympa service seems to be totally unreachable.

[ikedas]

@ikedas,

It is always returning with the following HTTP status code and message in the HTTP header:
HTTP/1.1 421 Misdirected Request and no data.
The maximum message in /var/log/sympa.log I have is ( with log_level 5):

wwsympa[12345]: debug2 Sympa::List::get_robots()

I tried to reconfigure to wwsympa_url to be just /sympa (of course same as location in /etc/httpd/conf.d/sympa.conf), the same thing as with multiple component path. Sympa service seems to be totally unreachable.

@bruncsak , please show us ---

• Configuration of related components: Both httpd and sympa.
• What you operated step-by-step, e.g. "launch web browser, open URL xxx, click xxx, ..."
• The log from when you started accessed to web server to when the server finished returning the result.
• The result, e.g. what shown on the browser.

And, you'd be better to show the things above both when the patch has not been applied yet and has already been applied.

[ikedas]

@ikedas,

It is always returning with the following HTTP status code and message in the HTTP header:
HTTP/1.1 421 Misdirected Request and no data.
The maximum message in /var/log/sympa.log I have is ( with log_level 5):

wwsympa[12345]: debug2 Sympa::List::get_robots()

I tried to reconfigure to wwsympa_url to be just /sympa (of course same as location in /etc/httpd/conf.d/sympa.conf), the same thing as with multiple component path. Sympa service seems to be totally unreachable.

Your httpd looks not yet been configured. At first, please check the configuration to work without multiple component (I guess you have not set ServerName and/or ServerAlias correctly). Please see the documentation.

[bruncsak]

Thanks, I found the incorrect configuration. The host part of the wwsympa_url parameter in the sympa.conf did not match the domain of the URL in the browser, and I did not restart the wwsympa service either. Sorry. Now all looks good with multiple component path both with single domain configuration as well with virtual robot configuration.

[ikedas]

@bruncsak , thank you for reporting problem and confirming fixes! I'll merge the PR above.