For the Vortex this is not a problem but it should not become too much.
The material is chosen so that magnetite hardly settles on the inside, or not at all.
If the installation is full of magnetite, you may have other problems (flow profile etc), hence it should not get too excessive.