2011-12-01 19:01:15
Views(15265)Case
Test Simple "Hello World" page output, Simple MVC logic(Router -> Controller -> Viewer), No Database Connections, No complex logic.
Hardware platform
CPU: Intel Core i5 750 (2.67GHz x4)
RAM: 4GB
Software Environment
Debian 6.0.4 x86_64 (2.6.32-41)
apache 2.2.16 (mpm-prefork,mod-php5)
php 5.3.10
php-apc 3.1.9 (Optimization for include/require)
Copy all projects to /dev/shm/* (Optimization for files read/write)
Frameworks list
* zf: Zend Framework
Site: http://framework.zend.com
Ver: 1.11.11
* zf2: Zend Framework
Site: http://framework.zend.com
Site: https://github.com/zendframework/ZendSkeletonApplication
Ver: 2.0.0-beta1
* symfony: symfony
Site: http://www.symfony-project.org
Ver: 2.0.0
* cakephp: CakePHP
Site: http://cakephp.org
Ver: 2.0.4
* ci: CodeIgniter
Site: http://codeigniter.com
Ver: 2.1.0
* yii: Yii Framework
Site: http://www.yiiframework.com
Ver: 1.1.8
* micromvc4: MicroMVC
Site: http://www.micromvc.com
Ver: 4.0.0
* laravel: Laravel, A PHP Framework For Web Artisans
Site: http://laravel.com
Ver: 2.0.2
* slim: Slim Framework
Site: http://www.slimframework.com
Ver: 1.5.0
* yaf: Yet Another Framework in PHP Extension
Site: http://code.google.com/p/yafphp
Ver: 2.1.3-beta (yaf.cache_config=1)
* FuelPHP: Simple, Flexible, Community driven Web Framework
Site: http://fuelphp.com/
Ver: 1.1
* ColaPHP:
Site: http://code.google.com/p/colaphp/
Ver: 1.2-ga
# XHProf: A Hierarchical Profiler for PHP
Site: http://pecl.php.net/xhprof
Ver: 0.9.2
# The source code of this testing:
Source: https://github.com/eryx/php-framework-benchmark
1. Apache Benchmark
Requests pre second (-c 100 -n 30000), the bigger is better
Requests pre second (-c 200 -n 50000), the bigger is better
2. System LoadAvg
System LoadAvg in 1 Minute when Apache Benchmark Complete,
the smaller is better (-c 100 -n 30000)
3. Memory usage
How many memory usage in one "hello world" page. the smaller is better.
4. Response time
The time of page request to response.
5. Number of function calls (Facebook XHProf)
The number of functions calls in one "hello world" page.
The detail of function calls map
| Symfony2 |  |
Zend Framework 1.11.1 |  |
| Zend Framework 2.0.0-beta1 |  |
CakePHP 2.0.4 |  |
| CodeIgniter 2.1.0 |  |
Yii Framework 1.1.8 |  |
| Slim 1.5 |  |
Laravel 2.0.2 |  |
| MicroMVC 4.0.0 |  |
Yaf 2.1.3-beta |  |
| FuelPHP 1.1 |  |
ColaPHP 1.2 |  |
6. Number of Files
The number of files included or required in one "hello world" page. the smaller is better.
Changelog:
* v20111201 @2011-12-01
- release
* v20111201-2 @2011-12-05
- bugfix: Fix a stupid logic error in bench.php
- bugfix: Symfony2, Change running mode from 'dev' to 'prod'
* v20111201-3 @2012-02-29
- New: new frameworks added: fuelphp, colaphp
* v20111201-4 @2012-05-18
- Optimized configuration for symfony2 (Commit By https://github.com/Smart-Core)
Patch https://github.com/eryx/php-framework-benchmark/pull/2
--EOF--
Tags:
Comments
thanks for your work~ :)
Why would you benchmark Symfony 2 in dev mode?
@Matthias Breddin
Just the beginning, I tried to use app.php to boot, but failed (500 error); at the same time app_dev.php work well. In order to output a simple "hello world" page, I modified the config_dev.yml([web_profiler] toolbar: false; intercept_redirects: false).
I think, although the mode name is "dev", but because the debug was disabled, in fact the "app.php" was equal with "app_dev.php" ?
I was the first time using symfony, If the configuration can be optimized, expect you to submit patch :)
https://github.com/eryx/php-framework-benchmark (symfony2)
@rui
According to the function calls map, the profiler was activated, which slows the app and increase memory usage.
The AsseticBundle routing was also enabled, which makes checking the routing cache more expensive.
The app was clearly in debug mode. Putting toolbar:false in the config does not disable the debug mode (which is not even handled in the config file) but simply disable displaying the toolbar.
@stof
Thanks for your reminder, I will modify the configuration and redo the test for symfony.
you forgot to include two more great frameworks:
1. index.php: <?php echo "Hello world;" ?>
2. index.html:
Hello world
your tests are totally useless
It doesnt seem like the laravel framework is excecuting a controller in the test.
how do I run this benchmark on my own server? I tried downloading the code from github but that didnt work too well
guest above: you can use routes instead of controllers in laravel.
I love how the big framework guys moan at the shitty performance of symfony and zend.
Why not FuelPHP?
"how do I run this benchmark on my own server?"
i have just update the "README" file, How to run this benchmark
it's good
This is very interesting. Can you benchmark more frameworks? I would like to see you test these others:
Silex: http://silex.sensiolabs.org/
Flight PHP: http://flightphp.com/
Phraw PHP: http://phraw.dav-muz.net/
Lithium: http://lithify.me/
FuelPHP: http://fuelphp.com/
apache benchamark, seriously?
"Hello world" ? come on! Why would you use a framework for that?
Testing mode is some frameworks?
I feel like this benchmark has no real value, data is hard to trust in.
documentation on yaf is kinda lacking but http://yaf.dashexamples.com/ seems to be your best bet. The guy has been pretty good about responding to my questions.
Amazing work. Thx so much!
@rui it seems the security component for Symfony is enabled (app.security.isGranted() called in a template), not sure if the others have ACL and user checking enabled, but according to the callgraph it accounts for 35% of the request, whereas others are probably unauthenticated hello worlds.
@beberlei Thanks for your suggestion, I will try to turn off this parameter in the next time.
Good job Rui. This is a great general overview. 谢谢你的分享.
Let's say we have a stick, a gun, a tank and a bomb. Lets do some benchmarks on which one is better to kill an ant. Don't forget to count the time it takes to load each if necessarily.
This benchmarks are very good, but don't chose a framework solely on this.
Excelent work with those Benchmarks. Thank you very much.
By the way, Laravel 3.0 (stable) was released a week ago. It would be interesting to include in the benchmark.
Finally of the most popular Framewoks, only Kohana is missing.
http://kohanaframework.org/
PD: Sorry for my bad English.
Doophp should be in that list
Thank you very much, keep up the good work :)
Your fuelphp comparison is off :(
Looking through the code, your CI app, loads a view that is pure HTML. The FuelPHP controller, does a LOT more. There is CSS, decoding an image, etc. You might want to redo it with the same view that you used for CI :)
the fuelphp is also use the "pure HTML", no css/image etc.
see "fuel/app/views/welcome/index.php"
You stated that APC was running
php-apc 3.1.9 (Optimization for include/require)
But you did not change symfony to use ApcUniversalClassLoader.php
"For better performance, you can use the APC based version of the universal class loader:
require_once __DIR__.'/src/Symfony/Component/ClassLoader/UniversalClassLoader.php';
require_once __DIR__.'/src/Symfony/Component/ClassLoader/ApcUniversalClassLoader.php';
use Symfony\Component\ClassLoader\ApcUniversalClassLoader;
$loader = new ApcUniversalClassLoader('apc.prefix.');
Furthermore, the component provides tools to aggregate classes into a single file, which is especially useful to improve performance on servers that do not provide byte caches."
@jminkler
Thank you for reading the source code, I will continue to optimize the configuration of framework, and update the test data next time.
Excellent work...
Can you test with nginx http server?
A.
Please add kohana 3.0 and kohana 3.2
Awesome! Would love to see MODX (modx.com) in there
Would like to see Laravel 3.1.9 here
Hello world comparison worth nothing if you need more than hello world "application", if you don't need more than that why are you looking at frameworks?
frameworks should help you develop big applications, that's why they have overhead. Bigger overhead and more functions are just signs of a frameworks that does more for you.
If you want a real comparison, compare a real app!
Thank you, great benchmark.
Can you also add Lithium, FuelPHP and Fat Free?
Thanks!
wow Thank you
very very very usefull :D
add Kohana plz
Thank you for your attention. I will wait for a period of time (6-12 months) in order to wait for each framework can have a greater milestone updated and optimized. of course will add more frameworks :)
Thank you, great benchmark.
Can you also add DooPHP http://doophp.com/
Thanks!
With Yii is possible to include yiilite.php instead which made it both faster and include less files.. If I get the time I will run the benchmark again.
Please upload my optimized configuration for "hello world" and update the benchmarks.
https://github.com/Smart-Core/php-framework-benchmark/commit/b0895581229f87845264bb4ba2e3cfd6087f6840
oops :) this is correct pull request:
https://github.com/eryx/php-framework-benchmark/pull/2
About use yiilite.php in Yii:
I try to use yiilite.php replace yii.php, the "include files" reduced from 19 to 4, but the memory-usage is increased from 1180KB to 2389KB, the QPS lower.