[Purpose] The changing trend of passenger flow at urban bus stops is inseparable from the temporal and spatial characteristics. This paper aims to capture the temporal and spatial characteristics in passenger flow of bus stops. [Methods] Firstly, the graph convolutional network (GCN) was used to capture the spatial characteristics of the passenger flow; secondly, the gated recurrent unit (GRU) was employed to capture the temporal characteristics of the passenger flow; thirdly, a passenger flow prediction model of bus stops based on deep learning was developed and named gated-graph convolutional network (G-GCN). Finally, the passenger flow data of 512 bus stops in Zhumadian City were divided into three time granularities of 30, 45, and 60 min, and the G-GCN model was utilized for prediction. The prediction results were compared with those of the baseline model. [Findings] The root mean square errors of the G-GCN model are 2.35, 3.00, and 3.57 at the three time granularities, reducing by 19.60%, 24.40%, and 26.40% compared with those of the other baseline models. [Conclusions] The research results break through the limitations of predicting passenger flows in public transport within regular areas and provide technical support for the optimization of urban public transport organizations.