NotionCommotion Posted May 9, 2020 Share Posted May 9, 2020 Ran the following but got a segfault as it doesn't seem to support php7.4. ./vendor/bin/concrete5 c5:install -i So I changed ./vendor/bin/concrete5 from: #!/usr/bin/env php <?php ... to #!/usr/bin/env php72 <?php ... I am sure I will forget I did this and would rather configure either the directory or the user to use php72. Is this possible? Also, should I be making any other changes? For instance, maybe: "config": {"platform": {"php": "7.2.30"}} Thanks Quote Link to comment Share on other sites More sharing options...
NotionCommotion Posted May 9, 2020 Author Share Posted May 9, 2020 Ugg.. Got a little further, but another segfault. Had unrelated segfaults a while back and investigated backtraces, but never got them working and gave up. Guess I will be revisiting and figuring out once and for all how to investigate segfaults. kernel: php72[65109]: segfault at 7ff0c9e9ef00 ip 000055d9e88616f4 sp 00007ffee72c70a0 error 4 in php[55d9e8542000+473000] mysqld[61840]: 2020-05-09 12:09:36 19578 [Warning] Aborted connection 19578 to db: 'concrete5_ws' user: 'concrete5' host: 'localhost' (Got an error reading communication packets) Quote Link to comment Share on other sites More sharing options...
NotionCommotion Posted May 9, 2020 Author Share Posted May 9, 2020 Starting from scratch... $ echo '/tmp/coredump-%e.%p' | sudo tee /proc/sys/kernel/core_pattern # Yep, it is there $ cat /proc/sys/kernel/core_pattern /tmp/coredump-%e.%p # /tmp is writable $ ls -lat / total 40 dr-xr-xr-x 13 root root 0 May 9 13:02 sys drwxrwxrwt. 11 root root 4096 May 9 12:58 tmp drwxr-xr-x 33 root root 920 May 8 11:26 run ... And I created a segfault php -r 'posix_kill(posix_getpid(), SIGSEGV);' But nothing new is in /tmp. $ ls -lat /tmp total 12 drwxrwxrwt. 11 root root 4096 May 9 12:58 . srwxrwxrwx 1 postgres postgres 0 May 9 12:30 .s.PGSQL.5432 -rw------- 1 postgres postgres 49 May 9 12:30 .s.PGSQL.5432.lock drwx------ 3 root root 16 May 8 11:27 systemd-private-809bd1f0c8db495c945df5ac824728c0-php72-php-fpm.service-XXgFV3 drwx------ 3 root root 16 May 8 11:26 systemd-private-809bd1f0c8db495c945df5ac824728c0-php-fpm.service-HuwtDT drwx------ 3 root root 16 May 8 11:26 systemd-private-809bd1f0c8db495c945df5ac824728c0-httpd.service-IizJQ2 drwx------ 3 root root 16 Apr 9 18:29 systemd-private-809bd1f0c8db495c945df5ac824728c0-ntpd.service-iTmi5U dr-xr-xr-x. 17 root root 4096 May 26 2018 .. drwxrwxrwt. 2 root root 6 Apr 8 2016 .font-unix drwxrwxrwt. 2 root root 6 Apr 8 2016 .Test-unix drwxrwxrwt. 2 root root 6 Apr 8 2016 .X11-unix drwxrwxrwt. 2 root root 6 Apr 8 2016 .ICE-unix drwxrwxrwt. 2 root root 6 Apr 8 2016 .XIM-unix $ Note that https://bugs.php.net/bugs-generating-backtrace.php states the following, however, requinix early stated it wasn't required. Just making sure... Quote Important! To get a backtrace with correct information you must have PHP configured with --enable-debug! When running $ gdb `where php-fpm`, I get "-bash: where: command not found". Did you mean whereis? Probably for this case shouldn't even be worrying about php-fpm, and just php, right? A clue! Missing debuginfo. $ gdb php -r 'posix_kill(posix_getpid(), SIGSEGV);' ... Reading symbols from /usr/bin/php...Reading symbols from /usr/bin/php...expanding to full symbols...(no debugging symbols found)...done. expanding to full symbols...(no debugging symbols found)...done. /var/www/greenbean_ws/posix_kill(posix_getpid(), SIGSEGV);: No such file or directory. Missing separate debuginfos, use: debuginfo-install php-cli-7.4.5-1.el7.remi.x86_64 (gdb) So I installed it and tried again... debuginfo-install php-cli-7.4.5-1.el7.remi.x86_64 php -r 'posix_kill(posix_getpid(), SIGSEGV);' # Have two new folders $ ls -lt /tmp total 0 drwx------ 3 root root 16 May 9 13:37 systemd-private-809bd1f0c8db495c945df5ac824728c0-httpd.service-Xy4O5a drwx------ 2 root root 6 May 9 13:37 vmware-root_40848-1770131046 drwx------ 3 root root 16 May 9 13:18 systemd-private-809bd1f0c8db495c945df5ac824728c0-php72-php-fpm.service-ANw4wj drwx------ 3 root root 16 May 9 13:18 systemd-private-809bd1f0c8db495c945df5ac824728c0-php-fpm.service-OYXAU4 drwx------ 3 root root 16 Apr 9 18:29 systemd-private-809bd1f0c8db495c945df5ac824728c0-ntpd.service-iTmi5U But they are empty $ sudo ls -l /tmp/systemd-private-809bd1f0c8db495c945df5ac824728c0-httpd.service-Xy4O5a total 0 drwxrwxrwt 2 root root 6 May 9 13:37 tmp $ sudo ls -l /tmp/systemd-private-809bd1f0c8db495c945df5ac824728c0-httpd.service-Xy4O5a/tmp total 0 $ sudo ls -l /tmp/vmware-root_40848-1770131046 total 0 $ Maybe should be using dbg? If so recommendations how? Thanks Quote Link to comment Share on other sites More sharing options...
NotionCommotion Posted May 9, 2020 Author Share Posted May 9, 2020 Still don't have a dump in /tmp, but what is odd is the folder dates sometimes change with no rhyme or reason but they are always empty. Maybe the sub-directories need to be writable. Tried running php as sudo as well as making the directory 0777 but nothing. With gdb, however, now I am getting some visibility even though I expect I am not using it right. Maybe __GI__IO_file_underflow () from /lib64/libc.so.6? Don't know where to go from here... $ gdb php72 GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-119.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /opt/remi/php72/root/usr/bin/php...Reading symbols from /usr/lib/debug/opt/remi/php72/root/usr/bin/php.debug...done. done. (gdb) run Starting program: /usr/bin/php72 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". ^C Program received signal SIGINT, Interrupt. 0x00007ffff4c249a0 in __read_nocancel () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install audit-libs-2.8.5-4.el7.x86_64 bzip2-libs-1.0.6-13.el7.x86_64 cyrus-sasl-lib-2.1.26-23.el7.x86_64 expat-2.1.0-11.el7.x86_64 fontconfig-2.13.0-4.3.el7.x86_64 freetype-2.8-14.el7.x86_64 fribidi-1.0.2-1.el7_7.1.x86_64 gd-last-2.3.0-1.el7.remi.x86_64 glib2-2.56.1-5.el7.x86_64 glibc-2.17-307.el7.1.x86_64 graphite2-1.3.10-1.el7_3.x86_64 harfbuzz-1.7.5-2.el7.x86_64 jbigkit-libs-2.0-11.el7.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-46.el7.x86_64 libX11-1.6.7-2.el7.x86_64 libXau-1.0.8-2.1.el7.x86_64 libXpm-3.5.12-1.el7.x86_64 libc-client-2007f-16.el7.x86_64 libcap-ng-0.7.5-4.el7.x86_64 libcom_err-1.42.9-17.el7.x86_64 libcurl-7.29.0-57.el7.x86_64 libgcrypt-1.5.3-14.el7.x86_64 libgpg-error-1.12-3.el7.x86_64 libidn-1.28-4.el7.x86_64 libjpeg-turbo-1.2.90-8.el7.x86_64 libmcrypt-2.5.8-13.el7.x86_64 libpng-1.5.13-7.el7_2.x86_64 libraqm-0.7.0-4.el7.x86_64 libselinux-2.5-15.el7.x86_64 libssh2-1.8.0-3.el7.x86_64 libtiff-4.0.3-32.el7.x86_64 libuuid-2.23.2-63.el7.x86_64 libwebp7-1.0.3-1.el7.remi.x86_64 libxcb-1.13-1.el7.x86_64 libxml2-2.9.1-6.el7.4.x86_64 libxslt-1.1.28-5.el7.x86_64 libzip5-1.6.1-1.el7.remi.x86_64 nspr-4.21.0-1.el7.x86_64 nss-3.44.0-7.el7_7.x86_64 nss-softokn-freebl-3.44.0-8.el7_7.x86_64 nss-util-3.44.0-4.el7_7.x86_64 oniguruma5-6.9.4-1.el7.remi.x86_64 openldap-2.4.44-21.el7_6.x86_64 pam-1.1.8-23.el7.x86_64 pcre-8.32-17.el7.x86_64 php72-php-pecl-mcrypt-1.0.3-1.el7.remi.x86_64 php72-php-pecl-zip-1.18.2-1.el7.remi.x86_64 sqlite-3.7.17-8.el7_7.1.x86_64 xz-libs-5.2.2-1.el7.x86_64 (gdb) bt #0 0x00007ffff4c249a0 in __read_nocancel () from /lib64/libc.so.6 #1 0x00007ffff4bb0cc4 in __GI__IO_file_underflow () from /lib64/libc.so.6 #2 0x00007ffff4baf778 in __GI__IO_file_xsgetn () from /lib64/libc.so.6 #3 0x00007ffff4ba415f in fread () from /lib64/libc.so.6 #4 0x0000555555851f1d in zend_stream_getc (file_handle=0x7fffffffd120, file_handle=0x7fffffffd120) at /usr/src/debug/php-7.2.30/Zend/zend_stream.c:147 #5 zend_stream_read (file_handle=0x7fffffffd120, buf=0x7ffff3880000 "", len=4096) at /usr/src/debug/php-7.2.30/Zend/zend_stream.c:159 #6 0x00005555558521ef in zend_stream_fixup (file_handle=file_handle@entry=0x7fffffffd120, buf=buf@entry=0x7fffffffa868, len=len@entry=0x7fffffffa870) at /usr/src/debug/php-7.2.30/Zend/zend_stream.c:252 #7 0x00005555557f7146 in open_file_for_scanning (file_handle=file_handle@entry=0x7fffffffd120) at Zend/zend_language_scanner.l:514 #8 0x00005555557f74a7 in compile_file (file_handle=0x7fffffffd120, type=8) at Zend/zend_language_scanner.l:628 #9 0x00007fffe51c5b7d in phar_compile_file (file_handle=<optimized out>, type=<optimized out>) at /usr/src/debug/php-7.2.30/ext/phar/phar.c:3333 #10 0x00007ffff7e64fce in ?? () from /opt/remi/php72/root/usr/lib64/php/modules/dbg-php-7.2.so #11 0x0000555555833e6c in zend_execute_scripts (type=type@entry=8, retval=retval@entry=0x0, file_count=file_count@entry=3) at /usr/src/debug/php-7.2.30/Zend/zend.c:1492 #12 0x00005555557ce578 in php_execute_script (primary_file=primary_file@entry=0x7fffffffd120) at /usr/src/debug/php-7.2.30/main/main.c:2599 #13 0x00005555558e8adf in do_cli (argc=1, argv=0x555555c76c50) at /usr/src/debug/php-7.2.30/sapi/cli/php_cli.c:1011 #14 0x000055555563ddcb in main (argc=1, argv=0x555555c76c50) at /usr/src/debug/php-7.2.30/sapi/cli/php_cli.c:1403 (gdb) quit A debugging session is active. Inferior 1 [process 64503] will be killed. Quit anyway? (y or n) y $ Quote Link to comment Share on other sites More sharing options...
kicken Posted May 9, 2020 Share Posted May 9, 2020 2 hours ago, NotionCommotion said: Still don't have a dump in /tmp Did you check the ulimit setting? If your script is something you just run from the CLI you don't really need a core dump though, just run it using gdb. kicken@web1:~$ gdb /usr/bin/php7.3 GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git [...] Reading symbols from /usr/bin/php7.3...(no debugging symbols found)...done. (gdb) run yourScript.php Starting program: /usr/bin/php7.3 yourScript.php Quote Link to comment Share on other sites More sharing options...
NotionCommotion Posted May 9, 2020 Author Share Posted May 9, 2020 Haven't checked ulimit_settings, but I will. Ah, I needed to add the php script to run. Thanks, will give it a try. Quote Link to comment Share on other sites More sharing options...
requinix Posted May 9, 2020 Share Posted May 9, 2020 7 hours ago, NotionCommotion said: With gdb, however, now I am getting some visibility even though I expect I am not using it right. Maybe __GI__IO_file_underflow () from /lib64/libc.so.6? Don't know where to go from here... You started up php through gdb but then killed it with a ^C. The backtrace is what it is because you killed it. If running concrete5 is what crashes then you have to run that. As in gdb `/usr/bin/php72` and then run ./vendor/bin/concrete5 c5:install -i (or whatever). Quote Link to comment Share on other sites More sharing options...
NotionCommotion Posted May 10, 2020 Author Share Posted May 10, 2020 Thanks to both of your help, I am definitely closer. Turned out ulimit settings prevented me from generating a core dump. I created a php script designed to generate a segfault, ran it from the cli, and a core dump was created. But when I run the script through the webserver, I still generate the segfault but there is no core dump file created. Tried using multiple different users but nothing. Any thoughts? Thanks [php73] prefix = /run/php-fpm user = michael group = apache listen.owner = apache listen.group = apache listen.mode = 0600 listen = $pool.sock pm = ondemand pm.max_children = 50 ;php_flag[display_errors] = off php_admin_value[error_log] = /var/opt/remi/php73/log/php-fpm/error.log php_admin_flag[log_errors] = on ;php_admin_value[memory_limit] = 128M php_value[session.save_handler] = files php_value[session.save_path] = /var/opt/remi/php73/lib/php/session php_value[soap.wsdl_cache_dir] = /var/opt/remi/php73/lib/php/wsdlcache ;php_value[opcache.file_cache] = /var/opt/remi/php73/lib/php/opcache ;Used to create core backtraces rlimit_core = unlimited Quote Link to comment Share on other sites More sharing options...
kicken Posted May 10, 2020 Share Posted May 10, 2020 If your using systemd, you may need to configure the ulimit equivalent there, LimitCORE = infinity. Create /etc/systemd/system/php7.3-fpm.service.d/coredump.conf (may need to adjust directory name if your service name is not php7.3-fpm) with the content: [Service] LimitCORE = infinity Reload the configuration with /bin/systemctl daemon-reload then restart the FPM service. Quote Link to comment Share on other sites More sharing options...
NotionCommotion Posted May 11, 2020 Author Share Posted May 11, 2020 13 hours ago, kicken said: If your using systemd, you may need to configure the ulimit equivalent there, LimitCORE = infinity. Yes, systemd, but tried and no go. Wouldn't there be some sort of error log if an attempt to create a large core dump exceeded some setting? Its like php-fpm isn't even trying to do so because I didn't have some setting correct instructing it to do so. On 5/9/2020 at 2:28 PM, requinix said: If running concrete5 is what crashes then you have to run that. As in gdb `/usr/bin/php72` and then run ./vendor/bin/concrete5 c5:install -i (or whatever). This file caused crash on first PHP 7.4 and then PHP 7.2, but not PHP7.3. After install, still having crashes when making post request via web server. Don't think it is related but am using the composer version. As an interim fix, thinking of writing a script which serializes request and saves to file. Then I could run from the cli some script which populates the request body, etc and runs index.php. Pretty sad that I need to resort to doing so. If I can't figure out how to create core dump using php-fpm, any better options? Related topic. "unlimitted" sounds scary. Any risk of some infinite core being created? Quote Link to comment Share on other sites More sharing options...
NotionCommotion Posted May 11, 2020 Author Share Posted May 11, 2020 I don't know whether this is relevant. The request which causes the segfault is a POST request. My original thought was to capture the input stream, save it to a file, and then somehow (haven't figure this part out yet) write it to the body when executing the script on the CLI so that I would generate a core dump. The strange this is while $_POST is populated, stream_get_contents(fopen(php://input, r)) is empty. Does this make any sense? I ended up trying to just write to $_POST, however, I got some other error, and think I will abandon this idea and focus on how to property create a core dump when using Apache and php-fpm. <?php $server=$_SERVER;; if (php_sapi_name() == "cli") { $request=json_decode(file_get_contents('request.json'), true); $_SERVER=$request['$_SERVER']; $_GET=$request['$_GET']; $_POST=$request['$_POST']; $_COOKIE=$request['$_COOKIE']; } elseif($_SERVER['REQUEST_METHOD'] !=='GET') { syslog(LOG_ERR, '$_POST: '.json_encode($_POST)); //For unknow reasons, $_POST is populated but php://input is not when using Concrete5??? $stream = fopen('php://input', 'r'); $content = stream_get_contents($stream); syslog(LOG_ERR, 'stream_get_contents(fopen(php://input, r)): '.$content); $content = file_get_contents('php://input'); syslog(LOG_ERR, 'file_get_contents(php://input): '.$content); file_put_contents('request.json', json_encode(['$_GET'=>$_GET, '$_POST'=>$_POST, '$_COOKIE'=>$_COOKIE, '$_SERVER'=>$_SERVER])); } require('concrete/dispatcher.php'); Quote Link to comment Share on other sites More sharing options...
NotionCommotion Posted May 11, 2020 Author Share Posted May 11, 2020 (edited) According to http://httpd.apache.org/docs/current/mod/mpm_common.html#coredumpdirectory, looks like I must explicitly set CoreDumpDirectory. I tried "CoreDumpDirectory /tmp/coredumps", but got an httpd error that the directory didn't exist. Turns out I needed to set PrivateTmp=false in /etc/systemd/system/httpd.service. Tried but no good. Then went back to PrivateTmp=true and made systemd-private-7c09e14e22c34aaca6abb409f543b500-httpd.service-Txh4xo as writable by all (0777) but still nothing. https://cwiki.apache.org/confluence/display/HTTPD/CoreDump shows the following, however, I haven't done it as I don't know if it is required. sysctl -w kernel.core_pattern=/some/core/pattern Edited May 11, 2020 by NotionCommotion Quote Link to comment Share on other sites More sharing options...
kicken Posted May 11, 2020 Share Posted May 11, 2020 On 5/9/2020 at 9:53 AM, NotionCommotion said: posix_kill(posix_getpid(), SIGSEGV); I had to change that to posix_kill(posix_getpid(), 11); as the SIGSEGV constant was not recognized. After doing that, and the rest of the stuff mentioned here (ulimit -c unlimited, systemd LimitCORE=infinity, core_pattern change) I was able to successfully get a core dump on ubuntu. root@web1:/tmp# ls -al core* -rw------- 1 root www-data 278056960 May 11 11:24 coredump-php-fpm7.3.19660 root@web1:/tmp# gdb -c ./coredump-php-fpm7.3.3256 /usr/sbin/php-fpm7.3 [...] Core was generated by `php-fpm: pool kicken '. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007f0b38a02187 in kill () at ../sysdeps/unix/syscall-template.S:78 78 ../sysdeps/unix/syscall-template.S: No such file or directory. (gdb) bt #0 0x00007f0b38a02187 in kill () at ../sysdeps/unix/syscall-template.S:78 #1 0x00007f0b2a4ddde3 in ?? () from /usr/lib/php/20180731/posix.so #2 0x000055d5894844eb in ZEND_DO_ICALL_SPEC_RETVAL_UNUSED_HANDLER () at ./Zend/zend_vm_execute.h:649 #3 execute_ex (ex=0xcb8) at ./Zend/zend_vm_execute.h:55503 #4 0x000055d5894886f3 in zend_execute (op_array=op_array@entry=0x7f0b356800e0, return_value=0x0, return_value@entry=0x7f0b19e77b60) at ./Zend/zend_vm_execute.h:60939 #5 0x000055d5893f93a2 in zend_execute_scripts (type=type@entry=8, retval=0x7f0b19e77b60, retval@entry=0x0, file_count=895606832, file_count@entry=3) at ./Zend/zend.c:1568 #6 0x000055d589399380 in php_execute_script (primary_file=0x7ffd8f44a570) at ./main/main.c:2639 #7 0x000055d58925299b in main (argc=<optimized out>, argv=<optimized out>) at ./sapi/fpm/fpm/fpm_main.c:1951 (gdb) Quote Link to comment Share on other sites More sharing options...
NotionCommotion Posted May 11, 2020 Author Share Posted May 11, 2020 Agree SIGSEGV isn't recognized by php7.3 but only 7.4, and also had to find a different way to generate a segfault. So, no setting of CoreDumpDirectory in httpd.conf required for you? Also, are you running nginx or apache? Thanks Quote Link to comment Share on other sites More sharing options...
NotionCommotion Posted May 11, 2020 Author Share Posted May 11, 2020 Eureka! – Set fs.suid_dumpable for setuid or otherwise protected/tainted binaries. # vi /etc/sysctl.conf fs.suid_dumpable = 2 Following is the meaning of each predefined value: 0 – (default): traditional behaviour. Any process which has changed privilege levels or is execute only will not be dumped. 1 – (debug): all processes dump core when possible. The core dump is owned by the current user and no security is applied. This is intended for system debugging situations only. 2 – (suidsafe): any binary which normally not be dumped is dumped readable by root only. This allows the end-user to remove such a dump but not access it directly. For security reasons, core dumps in this mode will not overwrite one another or other files. This mode is appropriate when administrators are attempting to debug problems in a normal environment. Quote Link to comment Share on other sites More sharing options...
NotionCommotion Posted May 11, 2020 Author Share Posted May 11, 2020 After creating a core dump and getting a backtrace, I saw that my ide debugger somehow prevented getting the info. Disabled the ide and no more segfault. Generated a fake segfault and was able to view with gdb. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.