Honestly? If you want my theory, Gale never left and never went to District Two. That was all a lie fed to Katniss to keep the Mockingjay sane. 
I strongly believe, as much as it breaks my heart, that Gale died during that break-in at the Capitol when he was taken away. That they took him away and killed him before that bomb that he was blamed for even went off. 

Besides this theory though and if you asked my second opinion if he really did just up and leave Katniss for District 2, I agree with Charlie. Knowing it was his bomb, he was overwhelmed with guilt and the thought that his creation killed someone close as family. He loved Prim, took care of her when Katniss wasn't around, made sure she didn't starve. 
He knew Katniss was unstable enough as it was and seeing him would probably only further set her off. It was best for him to stay away. If she really wanted to see him, really wanted him back, she could have gone to District 2. Love is a two-way street after all and Gale did all he could for Katniss. 

Though it did mention her sending Gale letters, which furthers my belief that Gale is actually dead. 

Source 

Nunca, nunca tal coisa me tinha passado pela cabeça. O Gale morto.
Será?
Esperem um bocadinho enquanto acabo de chorar.