Although I don't read the manga but it's my understanding it's only the fillers that have make it look like he hasn't had much of a break. He could have skipped the shit happening in the fillers and used the time for training. Actually he should have trained himself in the SS. Magic training from Rukia and others and attitude training from Kenpachi. He would have beaten Grimmjaw and Nnoitra in no time. He probably would have still lost the fight against Ulquiorra but at least it wouldn't have looked so miserable.