Zend FW rediculously slow at including resources.

[trq]

Part way through developing a rather large CMS system for my employer and have just started benchmarking load times. It would seem Zend FW is terribly slow at loading bootstrap resources.

 

I have stripped the bootstrap process back to this.

 

index.php

<?php
    
defined('APPLICATION_PATH')
    || define(
        'APPLICATION_PATH',
        realpath(dirname(__FILE__) . '/../application')
    );

defined('APPLICATION_ENV')
    || define(
        'APPLICATION_ENV',
        'development'
    );

require_once realpath(APPLICATION_PATH . '/bootstrap/Setup.php');

$application->bootstrap('acl');

 

/bootstrap/Setup.php

<?php

set_include_path(implode(PATH_SEPARATOR, array(
    get_include_path(),
    realpath(APPLICATION_PATH . '/library')
)));

require_once 'Zend/Config/Ini.php';
require_once 'Zend/Registry.php';
require_once 'Zend/Application.php';

$config = new Zend_Config_Ini(APPLICATION_PATH . '/configs/application.ini', APPLICATION_ENV);

Zend_Registry::set('conf', $config);

$application = new Zend_Application(
    APPLICATION_ENV,
    array(
        'autoloaderNamespaces' => array('This_'),
        'pluginPaths' => array(
            'This_Bootstrap_Resource' => 'This/Bootstrap/Resource'
        ),
        'resources'     => array(
            'Acl'       => array(),
            'Log'       => array(),
            'Session'   => array(),
            'Orm'       => array(),
            'Front'     => array(),
            'Layout'    => array(),
            'View'      => array(),
        ),
        'phpSettings'   => array(
            'display_startup_errors' => $config->php->display_startup_errors,
            'display_errors' => $config->php->display_errors
        ),
    )
);

 

This/Bootstrap/Resource/Acl.php

<?php

class This_Bootstrap_Resource_Acl extends Zend_Application_Resource_ResourceAbstract
{
    public function init()
    {
    }

}

 

This setup is so stripped back it doesn't actually start the application. Results are not good.

 

thorpe $ ab -n 100 -c 10 http://phpdev/

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking phpdev (be patient).....done


Server Software:        Microsoft-IIS/6.0
Server Hostname:        phpdev
Server Port:            80

Document Path:          /
Document Length:        0 bytes

Concurrency Level:      10
Time taken for tests:   18.387 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      36800 bytes
HTML transferred:       0 bytes
Requests per second:    5.44 [#/sec] (mean)
Time per request:       1838.686 [ms] (mean)
Time per request:       183.869 [ms] (mean, across all concurrent requests)
Transfer rate:          1.95 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   2.7      0      16
Processing:  1468 1812 238.2   1765    2562
Waiting:     1468 1812 238.2   1765    2562
Total:       1468 1812 238.1   1765    2562

Percentage of the requests served within a certain time (ms)
  50%   1765
  66%   1812
  75%   1843
  80%   1875
  90%   2250
  95%   2437
  98%   2546
  99%   2562
100%   2562 (longest request)

 

Now, if I add a call to require_once to include Acl.php within Setup.php ....

require_once 'Zend/Config/Ini.php';
require_once 'Zend/Registry.php';
require_once 'Zend/Application.php';
require_once 'This/Bootstrap/Resource/Acl.php';

 

My results....

thorpe $ ab -n 100 -c 10 http://phpdev/

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking phpdev (be patient).....done


Server Software:        Microsoft-IIS/6.0
Server Hostname:        phpdev
Server Port:            80

Document Path:          /
Document Length:        199 bytes

Concurrency Level:      10
Time taken for tests:   1.250 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      37400 bytes
HTML transferred:       19900 bytes
Requests per second:    80.02 [#/sec] (mean)
Time per request:       124.974 [ms] (mean)
Time per request:       12.497 [ms] (mean, across all concurrent requests)
Transfer rate:          29.22 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   3.4      0      16
Processing:    47  118 115.0    109    1234
Waiting:       47  118 114.8    109    1234
Total:         47  119 114.9    109    1234

Percentage of the requests served within a certain time (ms)
  50%    109
  66%    109
  75%    109
  80%    109
  90%    156
  95%    156
  98%    187
  99%   1234
100%   1234 (longest request)

 

As you can see, much better. The problem is (obviously) I have allot of other libraries that need to be dynamically included when the actual application is properly boostraped.

 

My include path is as clean as can be....

C:\php\include\ZendFramework\library;C:\Inetpub\sites\phpdev\application\library

 

Im running php under fastcgi on IIS6 (don't have a choice in the matter) with Zend FW 1.9. I don't have apc or memcache installed though I do have full control of the server. Testing on ZendServer CE over the last few days (which does have APC along with Zend Optimizer) however didn't improve the results and I had to actually remove it due to issues getting it to work with MSSQL (again, out of my hands).

 

Another site (our current sites dev version) running on the exact same server although writtin in asp 2.0 results in....

 

thorpe $ ab -n 100 -c 10 http://webdev/

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking webdev (be patient).....done


Server Software:        Microsoft-IIS/6.0
Server Hostname:        webdev
Server Port:            80

Document Path:          /
Document Length:        22499 bytes

Concurrency Level:      10
Time taken for tests:   0.547 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      2284900 bytes
HTML transferred:       2249900 bytes
Requests per second:    182.87 [#/sec] (mean)
Time per request:       54.684 [ms] (mean)
Time per request:       5.468 [ms] (mean, across all concurrent requests)
Transfer rate:          4080.42 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   4.0      0      16
Processing:    31   50   9.2     47      62
Waiting:        0   24  13.0     31      47
Total:         31   51   9.7     47      78

Percentage of the requests served within a certain time (ms)
  50%     47
  66%     62
  75%     62
  80%     62
  90%     62
  95%     62
  98%     62
  99%     78
100%     78 (longest request)

 

I'm aware that PHP under fastCGI is going to take a performance hit and that ZendFW is going to add yet more overhead but this is the first time Iv'e bothered to benchmark any of my ZendFW sites and to be honest, this is ridiculous. this application will be expected to respond to 30,000+ hits / day, and I'm just not sure where to go from here.

 

Any ideas what could be done to speed up the boostrap process? Really quite desperate for some advice here as I'm about two months into this project and dropping zend at this stage is pretty well out of the question.

[trq]

Oh, by the way. Actual figures of load times with all resources loaded are....

 

thorpe $ ab -n 100 -c 10 http://phpdev/

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking phpdev (be patient).....done


Server Software:        Microsoft-IIS/6.0
Server Hostname:        phpdev
Server Port:            80

Document Path:          /
Document Length:        1188 bytes

Concurrency Level:      10
Time taken for tests:   6.937 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      136400 bytes
HTML transferred:       118800 bytes
Requests per second:    14.41 [#/sec] (mean)
Time per request:       693.737 [ms] (mean)
Time per request:       69.374 [ms] (mean, across all concurrent requests)
Transfer rate:          19.20 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   4.0      0      16
Processing:   422  682  76.9    687     875
Waiting:      422  682  77.0    687     875
Total:        422  683  77.2    687     875

Percentage of the requests served within a certain time (ms)
  50%    687
  66%    703
  75%    703
  80%    719
  90%    812
  95%    844
  98%    844
  99%    875
100%    875 (longest request)

 

Thanks.

[trq]

Sorry, that last post isn't right. Here are the actual figures.

 

thorpe $ ab -n 100 -c 10 http://phpdev/

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking phpdev (be patient).....done


Server Software:        Microsoft-IIS/6.0
Server Hostname:        phpdev
Server Port:            80

Document Path:          /
Document Length:        13756 bytes

Concurrency Level:      10
Time taken for tests:   48.699 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      1412800 bytes
HTML transferred:       1375600 bytes
Requests per second:    2.05 [#/sec] (mean)
Time per request:       4869.928 [ms] (mean)
Time per request:       486.993 [ms] (mean, across all concurrent requests)
Transfer rate:          28.33 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   4.0      0      16
Processing:  3937 4825 444.9   4750    6578
Waiting:     3937 4820 444.1   4734    6578
Total:       3937 4826 444.2   4750    6578

Percentage of the requests served within a certain time (ms)
  50%   4750
  66%   4812
  75%   4890
  80%   4937
  90%   5531
  95%   5765
  98%   6109
  99%   6578
100%   6578 (longest request)

[sKunKbad]

Zend has a bad reputation for being slow. Sorry to hear you have done so much work already. I'd guess that you have more experience than most people who come here, so the chances of a miracle/answer are very low at best.

[trq]

Thanks for the interest. Id still like to hear from anyone with any ideas.

 

Ive now implemented quite a bit from the performance guide including striping calls to require once and setting up[ an include file cache.

 

Still, no performance benefits where gained at all.

[thehippy]

So far you've profiled your application externally measuring its run entirety, how about you try running it through a debugger and profiling the application internals.  Perhaps that path will lead you to your answer.

[Zyx]

One of the main reasons that Zend Framework is slow is the fact of using include_path. It is one of the worst solutions for including files, especially if the list of available paths is longer. This is what PHP does when it has to execute include('foo.php') file, having five paths in the list:

 

1. Check if the file exists under the first location (disk operation - slow!). If not, go to step 2.

2. Check if the file exists under the second location (another disk operation). If not, go to step 3.

3. Check if the file exists under the third location... and so on until it will find a file.

 

If the most commonly used paths are provided at the end of the list, your application wastes lots of time to search the files in different locations. Unfortunately, using include_paths is hard-coded in Zend Framework (many files are included manually), so the smarter autoloader will not help much. Remember that every single disk operation costs a lot, because the disk access is quite slow. You should try to minimize the number of them.

 

Actually, I'm against using include_paths in any form in a production software, but if you already have an application, there is little you can do...

[trq]

My path (as noted) is tiny.

 

C:\php\include\ZendFramework\library;C:\Inetpub\sites\phpdev\application\library

[Zyx]

Is this the result of

 

echo get_include_path();

 

?

 

Usually include_paths are much longer. Anyway, it is still include_path, especially for the files from the second group. Why are you using it, if you have the new, smarter ZF autoloader? Try to configure it without include_path - it should help a bit.

 

I've noticed now that your code performs some disk operations twice. For example:

 

defined('APPLICATION_PATH')
    || define(
        'APPLICATION_PATH',
        realpath(dirname(__FILE__) . '/../application')
    );

 

Now, APPLICATION_PATH is realpathed, but a couple of lines later you do:

 

require_once realpath(APPLICATION_PATH . '/bootstrap/Setup.php');

 

realpath() again on a path that is already absolute. And later, when configuring include_paths - another. These just two lines are not so critical, but I haven't seen the rest of your code. Maybe there are much more such unnecessary disk (and other stuff) calls in your application and the sum of them gives you a very bad exectution times?

[trq]

Is this the result of

 

echo get_include_path();

 

?

 

Yeah. I minimised the path as much as I could. This project needs nothing but the Zend framework and my own libraries (and extensions to ZF) contained within C:\Inetpub\sites\phpdev\application\library

 

I've noticed now that your code performs some disk operations twice. For example:

 

defined('APPLICATION_PATH')
    || define(
        'APPLICATION_PATH',
        realpath(dirname(__FILE__) . '/../application')
    );

 

Now, APPLICATION_PATH is realpathed, but a couple of lines later you do:

 

require_once realpath(APPLICATION_PATH . '/bootstrap/Setup.php');

 

realpath() again on a path that is already absolute. And later, when configuring include_paths - another. These just two lines are not so critical, but I haven't seen the rest of your code. Maybe there are much more such unnecessary disk (and other stuff) calls in your application and the sum of them gives you a very bad exectution times?

 

I'll fix those up, but these are the only places that require and the like are called. I'm not sure what you mean by....

 

Anyway, it is still include_path, especially for the files from the second group. Why are you using it, if you have the new, smarter ZF autoloader? Try to configure it without include_path - it should help a bit

 

Surely the necessary libraries need to be on my path somewhere.

[Daniel0]

This may sound stupid, but are you running an opcode cache like APC? Are you caching things using Zend_Cache? If yes, do you use the filesystem as backend, or something that stores it in memory like APC or memcached?

[trq]

To be honest, I had allot of trouble getting apc running within IIS. Never tried memcache.

 

As you can see however, even without actually getting to my application, just bootstraping acl via Zend_Application completely killed performance.

 

I have been granted an extension to have this application finished by Febuary, in doing so have decided that Zend simply isn't going to perform well enough to be relied upon and I am now part way through rolling my own (much more minimal) mvc. I really loved the framework but the numbers simply don't stack up.

[Daniel0]

It sounds weird. The numbers I get for PHP Freaks are not even near the numbers you get, and that is for the entire application, not just the bootstrapping process. I'll try to profile our code some time and see where the bottlenecks are here.

[trq]

Yeah, it doesn't make much sense to me either. I might try actually moving the application to another server on Monday and have another look. I'd really like to stick with it if possible.