Side effects permenant??

I have been taking Neurontin for almost 2 years now. I was up to 3200 mg/day, but in the past several months I have been weaning myself down, and now take 2400/day. I am stuck and can't seem to go any further without severe pain. My side effects have been memory loss and some weight gain. I'm gradually losing weight, but my memory issues are terrible. My question: Will I ever be able to get off this medication, and is my mind ever going to work right again? I function well on this dosage, but when I try to go down to less, I am miserable. But sometimes, I feel like I have brain damage.