If I reassure him it’s nothing is it actually reinforcing that there is something to be concerned about?
No. But it might not help, either.
I wouldn't guarantee that he'll grow out of it. He might, or it might stay the same, or get worse. In general, the more a behaviour is practiced, the more it's likely to happen again in the future.
My approach would be the same as any alert barking - trying to pair the trigger with something relaxing and reinforcing. The easiest thing to do is a food scatter. Chuck food on the floor by your feet (give a "scatter" cue if you need) so that he starts to come to you on hearing the noise, and snuffles around at your feet. For many dogs, the act of sniffing and eating is calming. If it's not (Shadow can stay aroused), then once he's snuffled out that food, I deliberately place individual pieces of food on the floor. Because those pieces aren't bouncing, they're less exciting and that brings the arousal down.
Over time, you'll likely find that his barking will become less at each instance. What this progression looked like for Shadow with noises outside was a bit like this:
> Noise outside, Shadow runs to the door barking manically. I throw food at my feet and say "scatter". Shadow would continue to bark, sometimes run between me and the door, mainly ignored the food, until he calmed down, at which point, I would continue to feed at my feet.
> Noise outside, Shadow runs to the door, barking manically. I throw food at my feet and say "scatter". Shadow would break off to run to me, grab some food, run back to the door, come back etc. Calming would happen a little more quickly.
> Noise outside, Shadow runs to the door barking, I say "scatter", and he come to me to eat the food. He might gruff in place a few times. I continue to feed until he is calm (no looking at the door, no ear flicks towards it etc).
> Noise outside, Shadow runs to ME barking. No need to say "scatter", I just feed on the floor, as above, until calm.
> Noise outside, Shadow looks at the door and gruffs, then comes to me. No "scatter" cue, just feeding until he dismisses.
That's pretty much where we got to before I left Andorra. The level of noise he was responding to changed over the process, too. At first, he would react to any sound of any door closing, any footsteps etc. By the end, he wouldn't respond to a group of kids shouting in the hallway. It was more unexpected noises, but the general apartment block noise just became normalised to him.
I did also set up scenarios where I would knock on furniture, the door etc as "training", but I think this was actually less effective than simply responding to the world as it happened; he definitely cottoned on quickly to the fact it was me making the noises! That said, it's worth trying that approach of making the noises yourself to see if it helps.You preferably want to set it up with noises that are under his response threshold, so you don't get a response at all, and you can just pair unexpected noises with a scatter by your feet.