Integrality of iterated averages and pointwise prime powers

Suppose that we repeatedly replace the values of an integer-valued function on $\Z$ by averages over $p$ consecutive arguments, where $p$ is prime. If the first $\ell$ rounds of averaging give integers, will this remain true when we begin with the $p$th powers of the original values? We prove that the answer is yes for $\ell\le p$. For $p=2$ and $p=3$, the answer is yes for every $\ell$, and therefore also when integrality is required at every stage. For larger primes, a known counterexample first fails at round $p+1$. The proofs use finite differences and polynomial interpolation. We also explain connections with earlier work on divisibility of finite differences and Stirling numbers.