Does breaking broken builds count as griefing?

I was wondering if breaking the destroyed remains of a build would count as griefing.

Here is an example:

See this Topic.

Does that mean that I can’t remove it regardless of it’s state?

Yes, you can’t