There are two obvious directions to approach this problem: either continuously add information storage matter, continuously improve the storage efficiency of the information, or continuously optimize the useful information.
The first option is certainly cheating. Eventually we'd have giant heads wobbling on our thin little necks. However, for some creatures, this is a valid approach. Consider if the internet was alive. Its approach to continuing to learn is to keep adding harddrives all along its nodes.
The second option is less cheating. If we could gzip the information in our heads, we could compress it better. Unlike the first option, I consider it a fair direction to explore because in theory we could learn new compression algorithms, so it would make sense that you could keep growing this way.
The third option is where things get interesting.
Consider that we don't have to remember everything. Most humans do fine, even though we don't remember much from when we were 2. We kept the parts that were useful (walking is awesome, we should keep doing that), but we generally forgot all of the memories. They weren't very helpful anyways (unless you found a way to puke on your mom's favorite dress without getting her angry... that will be a useful skill in those upcoming frat days).
The trick is that we collect a large amount of information, not knowing if it's actually useful or not. Eventually some of it condenses into smaller nuggets which have long term potential, and we keep those. We don't really need to know the exact wind speed or time of day when we fell off the bike and skinned our knees. The part that was really useful was the fact that bikes are not 100% stable. Sometimes we do remember the whole event "like it was yesterday," but even that word choice should suggest that that is the exception not the rule.
There are unlimited ways you can go about this, but some of them are simple, so it makes sense to look at them before exploring the real creativity that is human existence.
One simple approach is time-boxing. Your creature could decide "I remember the last 100 years worth of events, and after that I forget." This approach is nice because it's easy. However, it removes one of the major advantages to living a long time: the ability to collect experience. Pretty soon our creature would forget WWII, and would be on the same footing as any individual from the modern era who never went through it.
The next step along that line is repeated time-boxing. In this solution, the unusually long lived being "forgets" everything it knows, and then lives out a normal existence until some trigger "wakes it up." In then processes everything in that life, assimilates it, and then does the process over again. This pattern is common in science fiction as a "solution to getting bored," ensuring each life is as new and exciting as the last. As long as the trigger invokes your full capacity before you get yourself, in trouble, you're safe.
Also interesting is that this process can be recursive. Eventually you could "fill up" the larger mind, and then you're stuck. But if that lifespan is also time-boxed, it could pass that information on to a longer lived one. Consider saving half of your mind for "the long existence," that you wake up to at the end of each "forgetful" lifetime. The other half is allocated to life. You could then divide this half mind in half again, devoting one part to "the middle existence," and one to "the short existence" of a lifetime. This path guarantees that your longest existence has complete unfettered control over at least half of your resources at all times, while the rest of the resources cycle useful information up towards it.
Of course, this pattern has a catch. It is remarkably hard to start. You have to hold back a remarkably large amount of your mental resources as you walk the earth using a fraction of your brain. The rest is slowly filled... very slowly. If you get in trouble during a "short" life, you won't have a whole lot of experience saved up because you chose to live such a small existence (using only a fraction of your mind for each life).
So let's flip it around. Lets use as much mind as we possibly can at all times. That's much more effective while we are young. When our "life" is full, we'll split our mind into two bins, half for holding onto "old" memories, and half for making "new" memories. We then fill the "new" memories bucket again, until it's full, and grab the juicy bits to keep in the "old bucket."
Eventually the old bucket gets full. Here's where we change it up. In the previous approach, we had subdivided the "new" bucket, so our brain for holding onto new memories got smaller and smaller as we tried to live longer and longer, leading to thinner and thinner lives. In this approach, we subdivide the old bucket. We now have 1/2 of our brain devoted to "new life," 1/4 devoted to "middle life," and 1/4 devoted to "all life." When we subdivide again, we end up with 1/2 new, 1/4 middle, 1/8 older, and 1/8 "all the rest." Each time we divide this way, we keep half of our brain for new experiences, so life continues to be new and exciting, every time we live it.
Now lets take this to the extreme. Buckets are nice, but nature loves continuous processes. If we stretch ourselves out to be reaaaaly old, we start to see a really strong exponential pattern form, 1/2, 1/4, 1/8, 1/16... Mathematics has a continuous version of this, the exponential curve: a*e^-kt.
Time to cook with some real gas. Let's make a model for our memories. We're going to show a pattern for compressing memories. Let's make a Y axis be "amount of our mind we're using for this data" and X axis be "how old the data is." Draw an exponential curve in the first quadrant (above and to the right of the axes).
The unknown of the world appears in the negative X axis. This is the data which we cannot know because it is "-12 days old," i.e. 12 days in the future. As we move through the world, these unknown values move to the right, towards "0 days old." This is the only point where we can interact with the world, and experience it. We can't experience things that happened in the past, we can only remember them. So we can measure the size of our life experience by how much of the world fits below the exponential curve at X=0 (now). The more that fits under the curve, the more of our mind we've devoted to being in the present.
From here on out, the data is no longer experience, it's simply memories. Data moves to the right, becoming older. As it does, the exponential curve clamps down, forcing us to compress the data. We analyze the data, and pick the "most useful" bits to keep going along. Maybe we don't need the full 1080p version of someone throwing a water balloon full of shaving cream at us... maybe the 320x200 youtube video will do. Maybe we realize that all 5 of our Engineering courses actually only taught us one thing, in different ways (differential equations), and that we can remember them all in much less space if we refactor them to all use the same space.
Eventually the oldest parts of us remember the most important little details. Perhaps it's a religion. Perhaps it's a thing that drives us to be alive.
A long lived creature can modify the exponential they use to store data. However, generally speaking they have finite resources. They can't simply say "give me a larger 'a' term so I can live more of life to the fullest." There's a tradeoff between 'a' and 'k.' The more of life you experience, the faster you have to compress it to stay under the bounding limit of the curve. The less of life you live, the more you can drag out that compression process, giving more fidelity to the older memories, and decreasing the risk of compressing out something you needed.
So those are the easy versions, which have nice clean shapes. There is a final one that I will write down, which builds off of the exponential. In the exponential memory described above, the march of time was relentless, crushing memories and hopes and dreams alike. It doesn't account for us remembering important events like our wedding night for years afterwards like it was just yesterday.
Change up the X axis. It is no longer "time," but "subjective time." The more subjective time has passed, the more compressed an idea is. Other than that, very little changes. The world still meets us at x=0 where we experience the world. However, let's let the process of remembering be a bit more chaotic. Lets let ideas go in both directions: getting compressed as before, and getting 'inflated,' doing our best to reconstitute the original memory from the compressed version. Now we can explain how we remember our wedding night: if we let a memory float in the wide open space near X=0, we can keep its full beauty alive. We simply never let it get compressed. Of course, this forces us to compress the rest of our life faster (making our life a bit thinner), but maybe it gives us purpose.
Of course, all good things must come to an end. Under the pressure of all of the incomming experiences, the memory ebbs and flows, compressing and decompressing ever so slightly. Eventually, the memory starts to fade, and we are forced to let time take its course, compressing the memory so we can still hold onto it.
This model should be sufficient for your long lived creature to learn. They live the balance: compressing memories as little as possible to keep them beautiful, but compressing them enough to avoid clogging their mind and stopping them from learning and experiencing. As long as they do a good job of this balance, there is no reason they cannot live for as long as they please, and learn the entire time. However, if they ever get "caught in the past," they could clog their mind and miss out on key opportunities to learn.