Seriously, who is out there queuing for expeditions and walking away or declining causing time outs?!

This seems to be happening way too frequently on both mutated random expeditions and normal expeditions. Is there a penalty for those who queue and then cause time outs? It's highly frustrating to sit in queue for upwards of 20 minutes or more at times just to have a group start and 1 person decline or time out...