When the earth started to move beneath his feet, he wasn't that worried. He had guards and soldiers with him, and they could handle anything that Yu Dao could throw at them. (Even an Avatar, and even if they hadn't actually won he had to give them points for their bravery. In all honesty they'd done quite well.)
When the earth suddenly shot up and swallowed him up to his shoulders, and his undignified scramble to escape only left him with one free arm, is when Zuko decided to start worrying. Around him his entourage was in varied states of burial; up to shoulders and necks and poor Chen had toppled over and was stuck in an odd position, bent over, his torso above the ground but his arms sunk to his shoulders and his legs sunk up to his thighs. There were shouts of alarm as they tried to free themselves, shouts to him over what to do, is the Fire Lord safe, where are they coming from?
The last question was answered rather promptly by the appearance of a certain short earthbender