Pi very definitely does not encode infinite information. As you notice, we can write a very short computer program that will produce all the digits of pi (eventually).
In any case, you can do lots and lots of mathematics with either only finite objects, or if you allow limits as you suggest, you can do almost all of math.
Only a vanishingly small part of math deals with actual infinities in a way that cannot be re-written in terms of limits.
I thought it encoded all information but the catch is the index into it contains as much information as what you are trying to get out so it is effectively a very slow, negatively efficient zip format.
That's basically equivalent to saying that the natural numbers encode all information [0], it's just that the index to say which natural number you mean is rather long.
However for natural numbers, the 'decompression' is very fast. Much faster than for Pi.
[0] You can make this precise via Bijective Numeration https://en.wikipedia.org/wiki/Bijective_numeration to handle leading zeroes nicely.
That's right. But it does give a feel for what infinity is to our finite minds. It ain't just big numbers. It's all computation! (For a certain definition of computation)
Enumerating the natural numbers is only a single trivial computation. Enumerating the digits of Pi is a bit more involved, but it ain't universal, either.