performance - PHP MongoDB driver opens many connections on slow query -

we encounter semi-weird behaviour in mongodb php driver (v1.3) when executing slow query. driver seems keep opening connections when requests slow , don't understand why. maybe guys have suggestions here.

here facts first:

• website , database run on single ubuntu 13.04 server
• server high-end 8 core 16gb ram server
• mongodb v2.2.4
• website runs php 5.4
• apache 2 web server
• php mongo driver 1.3.x (should newest 1.3)
• website uses doctrine odm
• website has 50 100 concurrent users @ moment
• ulimit open files (ulimit nofile) = 64000

once day memcache record expires , slow query done. leads php opening 800 connections mongodb (normally have 10 open connections according logs). our website memcached, our database doesn't have other significant load. 800 open connections make website have 30 second loading times @ first , throw several types of mongoexceptions (too many connections / socket exceptions) later on.

it's ugly query group by. clear, understand query slow , idiotic, , we're removing query today. it's not clear why screws entire website. use doctrine abstraction layer, actual query on 200,000 document database (3 fields per document: id/product/date)) according logs:

{"group":true,"keys":{"product":1},"initial":{"count":0},"reduce":"function (obj, prev) { prev.count++; }","options":[],"db":"orders","collection":"history"}

after query done, results written memcache 24h. new requests memcache, not mongodb. still, sticks around 800 connections, problem not solve , website doesn't respond anymore after while. takes 10 minutes open these 800 connections.

it feels typical race condition. query doesn't feel heavy enough cause race condition on server load. mean, feels shouldn't.

okay, questions are:

1. why php keep opening many connections?
2. why can't mongodb handle (it shouldn't big of problem, right?)
3. any other suggestions of should do?
4. should set timeouts on connections , queries solve or else?

reason ask because our website growing fast , we're expecting way more traffic , mongodb load in future.

thanks lot in advance!

given you're invoking group command instead of performing basic read query, may fighting against javascript interpreter in mongodb 2.2. it's not until 2.4 javascript interpreter enhanced support concurrent execution. if each of these group operations requires js evaluation (at least reduce function), you're looking @ widespread resource starvation.

i don't have explanation "too many connection" exceptions. 800 concurrent connections falls below mongodb's limit of 20,000 (note: being removed 2.6 in server-8943).

one idea refactor application , avoid group race condition use single document lock php process recompute result , refill cache. using findandmodify, have single document string _id (e.g. "order.history group") , active field. when php process gets cache miss , need recompute result, can first attempt execute findandmodify , find appropriate _id active false, updating active true in same, atomic operation. after getting lock document should proceed group command. other php processes can't find lock document (because active not false) instructed sleep bit, return stale data, or abort web request.