Nope, a little piece of of what ever in there is ok. SO long as it does not melt or break from the heat. You can even use our soft glass diffuser beads as terp pearls.
The main thing of a terp pearl is fun and entertainment. The do help hold more heat as they add more mass to the banger. They also ahh, kinda help the vaporization process, but a good carb cap will move the oils also, so this is why we find terp pearls to be mostly for fun, but that is what life is all about!!