We study the first-order model checking problem on two generalisations of pushdown graphs. The first class is the class of nested pushdown trees. The other is the class of collapsible pushdown graphs. Our main results are the following. First-order logic with reachability is uniformly decidable on nested pushdown trees. Considering first-order logic without reachability, we prove decidability in doubly exponential alternating time with linearly many alternations. First-order logic with regular reachability predicates is uniformly decidable on level 2 collapsible pushdown graphs. Moreover, nested pushdown trees are first-order interpretable in collapsible pushdown graphs of level 2. This interpretation can be extended to an interpretation of the class of higher-order nested pushdown trees in the collapsible pushdown graph hierarchy. We prove that the second level of this new hierarchy of nested trees has decidable first-order model checking. Our decidability result for collapsible pushdown graph relies on the fact that level 2 collapsible pushdown graphs are uniform tree-automatic. Our last result concerns tree-automatic structures in general. We prove that first-order logic extended by Ramsey quantifiers is decidable on all tree-automatic structures.