This is not a new noise - I've heard it a couple of times before after doing a rapid stop from high speeds when the brakes were already hot - it would make this intermittent scraping noise which would fade away over a couple of minutes. But I have never heard it before when driving casually at 30mph not doing any significant braking so I decided it was time to strip the caliper down, especially when I have suspected the front right caliper of intermittently sticking on slightly as previously discussed.
So off the caliper came and the problem was immediately apparent:

The edge of the inner half of the disc has exploded with flaky rust all the way around which has increased its diameter enough to scrape against the edge of the pad retaining bracket...

I started to file the rust back around the outside edge and discovered that it fell off in chunks, so I just tapped all the loose rust off around the edge and then gave it a clean up with a file and wire brush. That half of the disc is now about 4mm smaller in diameter than the other side but its a nice clean smooth edge with no rust.
Clearly on previous occasions when the noise appeared after hard braking it was due to the disc heating up and expanding just enough to make the rust rub against the bracket, why it happened today without hard braking I'm not sure - perhaps hitting a bump in the roadworks caused some of the flaky rust to start coming away, it really was just falling off with the slightest tap when I cleaned it up...
One thing that flummoxed me today is that I noticed, as you can see in the picture above, the rubber on the piston seemed to be stretched beyond its limit with the seal lifting up away from the piston. On investigating I realised that since I had lifted the caliper up and had been working on the rust problem the piston had extended itself out about 10mm past where it had previously been - and past the normal working range of the piston judging by the rubber boot.
How this happened I have no idea - I did not press the brake pedal nor did I bump the handbrake lever arm on the caliper - which was already disconnected before I lifted the caliper up. I did pivot the caliper assembly up and down many times on its pivot while filing the rusty disc so the only thing I can think of (besides a leaky doseur valve or a collapsing brake hose) is that perhaps bending the hose back and forth pushed some fluid into the piston and moved it out... but I'm not convinced by that theory.
As I have previously suspected this caliper of sticking on sometimes naturally I'm concerned that the pressure to the caliper may not always be 100% releasing... anyone have any idea what might have pushed the caliper out ? It did not happen on the other side when I did that one. Did I bend the hose back on itself too tightly ? I wound the piston right back in and it didn't come out again so its still a bit of a mystery...
When I tried to remove the pads I discovered a likely cause of the brake sticking on and "clicking" when letting go the first use each day that I reported earlier - the "ears" on the edge of the outer pad on the front right were well and truly wedged in by corrosion in the curve that the ears sit in - even with a hammer and screwdriver against the backing plate of the pad I struggled to get the pad to let go, it looks like the edge of the pad had actually corroded fast to the pad carrier bracket. The inner pad was pretty bad as well.
Once I had the pads out I gave the mounting bracket curve and pad ears some wire brushing and a little filing to clean them up so that the pads can move in and out freely, I then added copperslip to the mounting bracket in all four locations:

I don't know if other people do this but it seemed like a good idea. After this I put the pads back and found they can move in and out freely, yet the anti-rattle spring built into the pad still keeps the pad from rattling sideways at all. Perfect.
I also did the left hand side to make sure there wasn't any brake imbalance - the left side wasn't too bad at all - the disc wasn't rusty in the same way and just had a little bit of roughness around the edge that needed wire brushing, and the pads were not rusted in like the other side but I cleaned the ears up and greased it in the same way all the same.
So far so good.
So, the mystery scraping is solved, and I suspect the intermittent light sticking of the front right may also be solved - with the pad rusted into the holder it would not have released properly. I'm still a little bit concerned about why the piston extended itself 10mm after I flipped the caliper up so I will keep an eye on things.
Although the right hand disc looked pretty bad initially (the rust made it look like the rusty half was splitting down the middle) after cleaning up both disk edges with a wire brush and file it looks very tidy now, the lips at the edge are relatively minor and there is still about 8mm of pad thickness left, so I think I'll get another 10k at least out of these discs/pads