PHP Framework MVC Benchmark - v20111201-4

laruence

thanks for your work~ :)

Matthias Breddin

Why would you benchmark Symfony 2 in dev mode?

rui

@Matthias Breddin
Just the beginning, I tried to use app.php to boot, but failed (500 error); at the same time app_dev.php work well. In order to output a simple "hello world" page, I modified the config_dev.yml([web_profiler] toolbar: false; intercept_redirects: false).
I think, although the mode name is "dev", but because the debug was disabled, in fact the "app.php" was equal with "app_dev.php" ?
I was the first time using symfony, If the configuration can be optimized, expect you to submit patch :)
https://github.com/eryx/php-framework-benchmark (symfony2)

stof

@rui

According to the function calls map, the profiler was activated, which slows the app and increase memory usage.
The AsseticBundle routing was also enabled, which makes checking the routing cache more expensive.

The app was clearly in debug mode. Putting toolbar:false in the config does not disable the debug mode (which is not even handled in the config file) but simply disable displaying the toolbar.

rui

@stof

Thanks for your reminder, I will modify the configuration and redo the test for symfony.

mindaugas

you forgot to include two more great frameworks:

1. index.php: <?php echo "Hello world;" ?>
2. index.html:

Hello world

your tests are totally useless

guest

It doesnt seem like the laravel framework is excecuting a controller in the test.

guest

how do I run this benchmark on my own server? I tried downloading the code from github but that didnt work too well

guest

guest above: you can use routes instead of controllers in laravel.

I love how the big framework guys moan at the shitty performance of symfony and zend.

Zack Kitzmiller

Why not FuelPHP?

rui

"how do I run this benchmark on my own server?"

i have just update the "README" file, How to run this benchmark

srikanth

it's good

guest

This is very interesting. Can you benchmark more frameworks? I would like to see you test these others:

Silex: http://silex.sensiolabs.org/
Flight PHP: http://flightphp.com/
Phraw PHP: http://phraw.dav-muz.net/
Lithium: http://lithify.me/
FuelPHP: http://fuelphp.com/

Jose Lorenzo

apache benchamark, seriously?
"Hello world" ? come on! Why would you use a framework for that?
Testing mode is some frameworks?

I feel like this benchmark has no real value, data is hard to trust in.

bobby

documentation on yaf is kinda lacking but http://yaf.dashexamples.com/ seems to be your best bet. The guy has been pretty good about responding to my questions.

EnZo

Amazing work. Thx so much!

beberlei

@rui it seems the security component for Symfony is enabled (app.security.isGranted() called in a template), not sure if the others have ACL and user checking enabled, but according to the callgraph it accounts for 35% of the request, whereas others are probably unauthenticated hello worlds.

rui

@beberlei Thanks for your suggestion, I will try to turn off this parameter in the next time.

guest

Good job Rui. This is a great general overview. 谢谢你的分享.

daniel

Let's say we have a stick, a gun, a tank and a bomb. Lets do some benchmarks on which one is better to kill an ant. Don't forget to count the time it takes to load each if necessarily.

This benchmarks are very good, but don't chose a framework solely on this.

John

Excelent work with those Benchmarks. Thank you very much.

By the way, Laravel 3.0 (stable) was released a week ago. It would be interesting to include in the benchmark.

Finally of the most popular Framewoks, only Kohana is missing.

http://kohanaframework.org/

PD: Sorry for my bad English.

guest

Doophp should be in that list

Jim

Thank you very much, keep up the good work :)

guest

Your fuelphp comparison is off :(
Looking through the code, your CI app, loads a view that is pure HTML.   The FuelPHP controller, does a LOT more.   There is CSS, decoding an image, etc.   You might want to redo it with the same view that you used for CI :)

rui

the fuelphp is also use the "pure HTML", no css/image etc.

see "fuel/app/views/welcome/index.php"

jminkler

You stated that APC was running

php-apc 3.1.9 (Optimization for include/require)

But you did not change symfony to use ApcUniversalClassLoader.php

"For better performance, you can use the APC based version of the universal class loader:

require_once __DIR__.'/src/Symfony/Component/ClassLoader/UniversalClassLoader.php';
require_once __DIR__.'/src/Symfony/Component/ClassLoader/ApcUniversalClassLoader.php';

use Symfony\Component\ClassLoader\ApcUniversalClassLoader;

$loader = new ApcUniversalClassLoader('apc.prefix.');

Furthermore, the component provides tools to aggregate classes into a single file, which is especially useful to improve performance on servers that do not provide byte caches."

rui

@jminkler
Thank you for reading the source code, I will continue to optimize the configuration of framework, and update the test data next time.

Andrea

Excellent work...

Can you test with nginx http server?

A.

aktuba

Please add kohana 3.0 and kohana 3.2

guest

Awesome! Would love to see MODX (modx.com) in there

guest

Would like to see Laravel 3.1.9 here

guest

Hello world comparison worth nothing if you need more than hello world "application", if you don't need more than that why are you looking at frameworks?

frameworks should help you develop big applications, that's why they have overhead. Bigger overhead and more functions are just signs of a frameworks that does more for you.

If you want a real comparison, compare a real app!

Shay Ben Moshe

Thank you, great benchmark.

Can you also add Lithium, FuelPHP and Fat Free?

Thanks!

guest

wow Thank you
very very very usefull :D

guest

add Kohana plz

rui

Thank you for your attention. I will wait for a period of time (6-12 months) in order to wait for each framework can have a greater milestone updated and optimized. of course will add more frameworks :)

guest

Thank you, great benchmark.
Can you also add DooPHP http://doophp.com/
Thanks!

guest

With Yii is possible to include yiilite.php instead which made it both faster and include less files.. If I get the time I will run the benchmark again.

smart-core

Please upload my optimized configuration for "hello world" and update the benchmarks.

https://github.com/Smart-Core/php-framework-benchmark/commit/b0895581229f87845264bb4ba2e3cfd6087f6840

smart-core

oops :) this is correct pull request:

https://github.com/eryx/php-framework-benchmark/pull/2

rui

About use yiilite.php in Yii:
I try to use yiilite.php replace yii.php, the "include files" reduced from 19 to 4, but the memory-usage is increased from 1180KB to 2389KB, the QPS lower.

Abdullah Al Mamun

Great work. Thanks a lot.
But where is Kohana (most elegant one)?

guest

Thanks for an awesome comparison!
Kohana is the only one missing, it would be great if you could add it.
If you could keep this up to date with any major stable releases in the future this would be an awesome reference!

rui

The next major update will be at the end of this year.
the Kohana, DooPHP framework will be added :)

Salih BiLGiN

I helped my choices. This comparison is that you do. So which framework you use? Which is also good in a photo processing?

Andrei

The benchmark is excellent. What a framework must basically do is organize code. What this benchmark does is it tests the MVC part. "Hello world" ensures that the network card isn't the bottleneck. If yaf lacks fancy email and ACL, take it from Zend, but for the most part, a framework must dispatch requests.
PS: on such a server, 7000 requests/second on Apache is all you can hope for. Even an index.html will be as slow. If you want to take PHP to the limit, set AllowOverride None in httpd.conf and move the rewrite rules to a entry. The whole .htaccess finding and applying routine is lagging Apache terribly. Then yaf might output something like 15000 requests/second.

SCIF

Thanks for very interest and explained benchmark!

Can you retest these frameworks with new versions of it?

System load diagram is very interest. I think that you must use nginx + PHP-FPM for better productivity. I can help with setting it up.

Thank!!

guest

I was tested my ZF app with ab and discovered diferent numbers like 2493 conections per second.
My machine? not more wich a windows xp 32bits dualcore 2.53Ghz with 2gb of RAM.

Riyan

Ruby on Rails is great. It's a very good thing to learn if you're a PHP coder without MVC exinpreece. Frameworks to PHP are (sort of) what Rails is to Ruby    it makes you organize your code.I remember giving up on Code Igniter because it supported PHP4, effectively ruining the whole object-oriented paradigm. Symfony has impressive features (hello there, profiler) and great tutorial; but I gave up on Symfony when I realized it wasn't all that easy to learn    using relatively exotic languages (like YAML) and annotations (just   ugh), which seemed unnecessarily complicated. Zend Framework on the other hand is nicely structured and loose enough not to be the central problem in the application, but has a very poorly written Quickstart Tutorial, which needs a few days to complete, and not a few hours as it claims. CakePHP has a large   cookbook   database, but most of it is user-uploaded and poorly written.There is no   best ever   framework, it all depends on what you need. Don't seek perfection, young grasshopper. Frameworks are like hammers; they all are made to bang on things, but come in various sizes and shapes, because you wouldn't want to carve stone with a carpenter's hammer.

guest

Any chance you could include KohanaPHP latest version? I have had a long time soft spot for Kohana and it realy looks like it has come a long way and now offers a lovely HMVC light approach.

aris

What about compare with php limonade too?

https://github.com/sofadesign/limonade

PHP limonade is MVC/VC framework use procedural style programming. As it not use any object I think it will result similar performance with Yaf or out perform Yaf :).

Limonade size: 2671 lines (2397 sloc) 75.841 kb (low memory usage winner)
Limonade watch: 479, fork: 56

-thanks

guest

I'd be curious to see how AgilePHP stacks up against other full stacks.   http://agilephp.org/

guest

Just came across this framework built as a C extension. Would be interested to see how it fairs against everything else   http://phalconphp.com/

guest

Guys, don't choose a framework based on those benchmarks.

I quote Harro Verton :
http://fuelphp.com/forums/topics/view/8182

ou always have to be careful with interpreting results like this. Every framework is designed different, and that design can have an influence on the test results in a specific test case. Same for the way the "Hello World" example is coded.

For example, FuelPHP 1.x loads quite a few core files on startup, which might be noticable in benchmarks on systems with relatively slow I/O. But this number doesn't really increase when you build your app.

Another example: FuelPHP's "Hello World" example contains a fully functional controller, including loading css, images, and twitter bootstrap. It also uses a ViewModel. Compare that to CI's version, which only echo's a few lines of text.

And finally, performance isn't everything. By definition every layer of abstraction adds complexity and makes things a bit slower. We feel that is acceptable (within limits off course) if there are enough gains elsewhere. FuelPHP is a framework designed for large/corporate applications. In such environments, the speed of development is a big part of the total cost, so that the cost of a slightly faster server isn't really significant.

guest

You should include Silex

shama

I pity the fool who thinks frameworks can be benchmarked against each other in this way.

DAE51D

I agree with "mindaugas" that you need to also have a 'control' benchmark to index.html and index.php for comparison.

guest

The Photon framework is missing.

guest

what about compare with Phalcon? http://phalconphp.com/

guest

would love to see Kohana added to this list

http://www.kohanaframework.org

guest

Agrh! One more voice in the crowd cheering for inclusion of Kohana in this benchmark!

Kohana, Kohana, Kohana!

Kohanians, unite and conquer!

guest

Where is Kohana, the most elegant one?

guest

This bencmark is a joke... Zend Framework 2 beta1 is ancient... ZF2 is up to RC4, about to go stable. Beta1 had many known performance issues, which have long since been addressed.

And Symfony 2 in development mode? Seriously?

You should seriously consider removing such a misleading post from your blog until you can learn to do a proper benchmark. Have a look at some of Paul Jones stuff, http://paul-m-jones.com/category/programming/benchmarks

Also consider that ZF1, for example, has no layout enabled and uses no view helpers by default. ZF2'sdefault skeleton application, on the other hand illustrates practical usage of the URL view helper, gettext transations, layouts, etc. Even a basic hello-wold example is nowhere close to a fair comparison.

guest

ZF2 is at RC7 now, and still has so many performance issues that it's an absolute joke.

This is a pretty fair benchmark IMO and accurately represents the 'overhead and bloatedness' that different ones bring to the table.

And from personal experience ZF2 is indeed like Jabba the Hut in comparison to everything else.

Bruno Batista

Please update the benchmark for Laravel 3.2.7 :)

guest

Just came across this framework built as a C extension. Would be interested to see how it fairs against everything else    http://phalconphp.com/

guest

great benchmark, please uptodate framework version.

guest

PLEASE update the benchmark with latest versions.
also PLEASE include Phalcon PHP.

thanks!

Igor

I agree this bencmark is joke. ZF2 is now full realised.

guest

yes, please add PhalconPHP!!!

masterblaster

UPDATE!!!!

http://www.laruence.com/2012/09/16/2791.html

guest

Need an update with all last released framework

Symfony 2.1.x
ZF2
silex
etc...

guest

What these benchmarks do show is if your writing simple apps don't use the big frameworks.

It shows ZF2 beta1 did have major performance issues and whilst yes it might be loading everything but the kitchen sink. With a larger application it would still be slow.

Good work on the article, whilst there has been a few bugs in the testing procedure (as already pointed out), the article still has quite a bit of significance.

And for those people complain about out of date versions of the framework - looking at the date of the article before ignorantly blasting the author.

Keep up the good work!

guest

waiting for the test with updated frameworks!

guest

Hi Rui,

This is good. May I know what application was used to get the include functions map?

guest

First of all thanks very much.

If you repeat the benchmark, please add flow3 to your framework list

guest

More recent benchmarks (NOV. 2012):

http://www.comentum.com/web-application-framework-comparison.html

Alex Belyaev

Please, add to comparison - phalconphp - php framework fully compiled inc C extension for PHP. In benchmarks from phalcon's authors - hi is the fastest php framework ever

Alex Belyaev

and ofcourse very interesting to be added - is YAF (http://www.php.net/manual/en/intro.yaf.php)

guest

I really appreciate the time and effort that was put into doing this.   Does Hello World really indicate framework performance, I guess that's up for debate, however, I still this it's a valid benchmark to generally see how these frameworks handle the basics.

zf2 is in production, so these tests really, to be fair, shouldn't be run against a beta that was in development.   It would be nice to see the real numbers of the production release.   zend claims it's twice as fast as zf1.

guest

I'd like to see the Zend Framework production version on there. It'd be interesting to see if they've managed to reduce load etc since the tests,

guest

There is another benchmark in terms of page generation speed here: http://phpixie.com/blog/php-framework-benchmark/

Lito

Hi! can you add the phpCan PHP framework? https://github.com/eusonlito/PHPCan

I can send you a controller file with your requirements.

Regards,
Lito.

DimkOf

Please add Kohana Framework version >=3.2
http://kohanaframework.org

guest

Please add: Laravel 3.2.13

Xenon

Is there an Update to this tests ?
Laravel is now 3.2 version.

arobase

aktuba if you want to do a performance test kohana you can easily do yourself there are sources on github, kohana and the passage does not worth to be tested for the framework as it was managed is certainly not the framework to use professionally I said! therefore a performance test kohana would be completely useless here no thank you

guest

Hello, I really enjoyed this comparison. Yesterday I met the Phalcon php framework, would be interesting if you add it in this comparison.

http://phalconphp.com/

guest

Really? Hello world. Whats the point.   IF you really want to bench mark frameworks you need to include more in depth test like routes, db layers etc. Even then they will never quite beat out pure PHP.   Half the point of frameworks is developer speed opposed to application speed